Mobile platform control system and method based on multi-source sensor data

Through multi-source sensor data analysis, the driving mode of the mobile platform is automatically identified and adjusted, which solves the problem of difficulty in selecting modes in different environments of the agricultural mobile platform, and realizes efficient and safe driving without the need for experienced drivers.

CN113805589BActive Publication Date: 2024-11-12CHINA COAL TECH & ENG GRP CHONGQING RES INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111107633.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-08-31
Filing Date
2021-09-22
Publication Date
2024-11-12
Estimated Expiration
2041-09-22

AI Technical Summary

Technical Problem

In agricultural applications, agricultural mobile platforms need to automatically identify suitable wheeled or pavement drive modes according to different environments and states when transitioning. Currently, they rely on experienced drivers, and inexperienced drivers may lead to wrong mode selection. , causing inefficiency or safety hazards.

Method used

Design a mobile platform control system based on multi-source sensor data, including scene acquisition module, scene analysis module, work plan module, status detection module and driving control module, and identify environmental features through image recognition and neural network analysis to generate suitable work Modes and routes are adjusted in real time during driving to adapt to different scenarios and states.

Benefits of technology

No experienced drivers are required to automatically identify and select suitable working modes and routes, improve the driving efficiency and safety of the mobile platform, reduce labor costs, and ensure normal driving in different environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113805589B_ABST
    Figure CN113805589B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of mobile platform control technology, and specifically discloses a mobile platform control system and method based on multi-source sensor data, including a scene acquisition module, which is used to acquire images around the mobile platform and send them to a remote control terminal. A scene analysis module is used to analyze the images and identify feature models and application scenarios. A route planning module is used to identify the working area based on the feature model and application scenario, and to formulate a working route for the mobile platform within the working area, and send the working route to a receiving terminal. A mode control module is preset with working modes and moving speed ranges corresponding to each application scenario, and the corresponding working mode and moving speed range are given according to the identification results and sent to the receiving terminal. It can automatically identify the appropriate working mode and plan the working route according to the environment and the state of the mobile platform.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of mobile platform control, and in particular to a mobile platform control system and method based on multi-source sensor data. Background Art

[0002] Mobile platforms refer to unmanned aerial vehicles, robots, mobile cars, mobile ships or underwater mobile equipment, etc., which currently play an important role in agriculture, industry, film and television, search and rescue, police, military and other fields.

[0003] In the application of modern agriculture, for agricultural mobile platforms, it is necessary to pass through roads when transferring, and there are gravel, pits, swamps, steep slopes, woods and other obstacles in the farmland itself, which cause great obstacles to the movement of the mobile platform. Based on this, the inventor has designed an agricultural mobile platform that can switch between two working modes, wheel drive and crawler drive, to cope with different application scenarios. However, as to when to use wheel drive and when to use crawler drive, it is still necessary to rely on the operator's own experience and judgment, so corresponding experienced operators are required to control, and the manpower cost is high. For inexperienced drivers, after adopting the wrong working mode, it is also possible to get half the result with twice the effort. Therefore, there is an urgent need for a method that can identify the appropriate working mode according to the working environment and the state of the mobile platform itself, to make up for the problem of not being able to choose the correct working mode caused by the driver's own lack of experience. Summary of the invention

[0004] The technical problem solved by the present invention is to provide a mobile platform control system and method based on multi-source sensor data, which can identify the current application scenario according to the working environment and determine the appropriate working mode.

[0005] The basic solution provided by the present invention is: a mobile platform control system based on multi-source sensor data, comprising a mobile platform, the mobile platform having two working modes of wheel drive and track drive, and also comprising a receiving end and a remote control end connected to the network, the receiving end is arranged on the mobile platform body, and comprises a scene acquisition module, a control execution module and a state detection module, the remote control end comprises a scene analysis module, a working scheme module and a driving control module;

[0006] The scene acquisition module is used to acquire the surrounding images of the mobile platform and send them to the remote control end;

[0007] The scene analysis module is used to analyze images and identify feature models and application scenarios;

[0008] A work plan module, used to generate a work plan for the mobile platform according to the feature model and application scenario, and send the work method to the mobile platform, wherein the work plan includes a work route, a work mode, and a moving speed range;

[0009] A control execution module is used to control the mobile platform according to the working plan given by the remote control end;

[0010] The state detection module integrates multiple sensors to detect the state of the mobile platform when the mobile platform is driving and send the state of the mobile platform to the remote control end;

[0011] The driving control module is used to control the driving of the mobile platform according to the state of the mobile platform.

[0012] The principle and advantage of the present invention are: the receiving end obtains the surrounding images through the scene acquisition module, and sends the images and real-time to the remote control end. After receiving the data sent by the receiving end, the remote control end analyzes the image, identifies the feature model and application scenario according to the image, and the working mode module gives a working plan through the feature model and application scenario, and gives the planning of the working route, working mode and moving speed range of the mobile platform under the current environment, and sends the planned working plan to the receiving end. The receiving end controls the mobile platform through the control execution module according to the working plan. At the same time, during the driving process of the mobile platform, the state of the mobile platform is detected by the state detection module, and the driving control module controls the mobile platform in real time according to the state of the mobile platform to ensure the normal driving of the mobile platform. Compared with the prior art, the experience of the driver is not required, and the suitable working mode and the planned working route can be automatically identified according to the environment and the state of the mobile platform, and the mobile platform is controlled according to the feedback state of the mobile platform during the driving process of the mobile platform.

[0013] Further, the scene analysis module includes a feature extraction module and a feature analysis module;

[0014] A feature extraction module is used to extract feature models from images through image recognition;

[0015] The feature analysis module is used to analyze the feature model and identify the application scenario through the trained neural network model;

[0016] The working scheme module includes a route planning module and a mode control module;

[0017] The route planning module is used to identify the working area according to the feature model and application scenario, and to formulate the working route of the mobile platform within the working area;

[0018] The mode control module is preset with the working mode and moving speed range corresponding to each application scenario, and gives the corresponding working mode and moving speed range according to the recognition result.

[0019] In different application scenarios and different environments, the working plans required by the mobile platform are different. Therefore, by extracting the feature model and identifying the application scenario based on the feature model, the working area of ​​the mobile platform is identified based on the feature model and the application scenario, and the working route is specified in the working area. At the same time, the most suitable working mode and the moving speed range are identified according to the application scenario, so as to formulate a complete working plan for the mobile platform.

[0020] By extracting the feature model from the image and analyzing the feature model to identify the application scenario, the application scenario is identified and analyzed through the multi-dimensional feature model to ensure the accuracy of the recognition and select the correct working mode.

[0021] Further, the state detection module includes a vibration sensor and a tilt sensor;

[0022] Vibration sensor, used to detect the vibration frequency of the mobile platform;

[0023] A tilt sensor for detecting the tilt angle of the mobile platform;

[0024] During the traveling of the mobile platform, the traveling control module controls the traveling speed of the mobile platform according to the vibration frequency, and controls the traveling speed and the traveling direction of the mobile platform according to the tilt angle.

[0025] The detected states include vibration frequency and tilt angle. The vibration frequency reflects the flatness of the road surface, and the tilt angle reflects the slope of the road. The moving speed and direction of the mobile platform are controlled according to the vibration frequency and tilt angle to prevent damage to the mobile platform components caused by bumps and overturning caused by the slope.

[0026] Furthermore, the mode control module is also preset with vibration thresholds and tilt angle thresholds corresponding to the vibration frequency and tilt angle respectively. The mode control module is also used to control the working mode to switch to track drive when the working mode is wheel drive and the vibration frequency or tilt angle exceeds the vibration threshold or the tilt angle threshold.

[0027] When the mobile platform is in wheel drive, if the vibration frequency or tilt angle exceeds the vibration threshold or tilt angle threshold, it means that the current scene is no longer suitable for wheel drive, and it switches to track drive. In this way, when the application scene is suitable for wheel drive in the large environment, but track drive is required in some areas, timely adjustments can be made, and the impact of blind spots in image acquisition and emergencies can be avoided.

[0028] Furthermore, the state detection module also includes a speed detection module, which is used to detect the speed of the mobile platform. The mode control module is also used to control the mobile platform to switch the working mode when the speed of the mobile platform is not within the moving speed range.

[0029] When the wheel drive is used, if the moving speed is less than the moving speed range, it may be because the tire is stuck in a pit, or because the road surface is slipping and the friction coefficient is too low. By switching to the crawler drive, it can be effectively solved. If the moving speed is greater than the moving speed range, because the speed is too fast, the working efficiency in the current application scenario will be unsatisfactory. For example, in the plowing scenario, the speed is too fast, and the plowing effect may not achieve the expected effect. Under the same power, the crawler drive moves slower. By switching to the crawler drive, the moving speed is guaranteed not to exceed the moving speed range. When in crawler drive, if the moving speed is less than the moving speed range, it may be caused by insufficient traction, and the wheel drive has greater traction, which can effectively solve it. If the moving speed under crawler drive is greater than the moving speed range, it may be because the current road surface is not particularly bumpy, and there is no need to use crawler drive, so switch to wheel drive. By changing the working mode when the moving speed is not within the moving speed range, different problems can be solved.

[0030] Furthermore, the remote control terminal further includes a route optimization module, which includes a signal detection module, an area classification module, and a route redrawing module;

[0031] A signal detection module is used to detect the signal strength between the remote control terminal and the receiving terminal;

[0032] The area classification module is used to divide the working area into areas with low, medium and strong signal strength according to the signal strength;

[0033] The route replanning module is used to replan the moving route of the mobile platform according to the signal strength area, thereby reducing the time the mobile platform is in the low signal strength area.

[0034] Signal strength areas are divided according to signal strength, and the route is replanned according to the signal strength areas to reduce the time the mobile platform is in the low signal strength area, and to prevent the remote control end and the receiving end from being disconnected due to being in the low signal strength area for too long.

[0035] The present invention also discloses a mobile platform control method based on multi-source sensor data, using the mobile platform control system based on multi-source sensor data, the method comprises the following steps:

[0036] Scene acquisition steps: The receiving end acquires the image around the mobile platform and sends it to the remote control end;

[0037] Scenario analysis steps: including the following steps:

[0038] Step 1: Extract feature models from images through image recognition;

[0039] Step 2: Analyze the feature model and identify the application scenario through the trained neural network model;

[0040] Work plan step: Generate a work plan for the mobile platform based on the feature model and application scenario, and send the work plan to the receiving end;

[0041] The working scheme steps include the following steps:

[0042] Step 1: The remote control terminal identifies the work area based on the feature model and application scenario, and formulates a work route within the work area;

[0043] Step 2: According to the preset working modes and moving speed ranges corresponding to each application scenario, the corresponding working modes and moving speed ranges are given;

[0044] Driving control steps: Control the mobile platform according to the working plan given by the remote control end;

[0045] Status detection step: During the driving process of the mobile platform, the status of the mobile platform is detected and sent to the remote control end;

[0046] Driving control steps: During the moving driving process, the driving of the mobile platform is controlled according to the state of the mobile platform.

[0047] Further, the state detection step includes the following steps:

[0048] Step 1: Detect the vibration frequency of the mobile platform;

[0049] Step 2: Detect the tilt angle of the mobile platform;

[0050] Step 3: Detect the moving speed of the mobile platform;

[0051] The driving control step comprises the following steps:

[0052] Step 1: Control the moving speed of the mobile platform according to the vibration frequency;

[0053] Step 2: Control the moving speed and direction of the mobile platform according to the tilt angle.

[0054] Furthermore, the mode control step further includes the following steps:

[0055] Step 1: Match the working mode corresponding to the current application scenario;

[0056] Step 2: When the mobile platform is in wheel drive, determine whether the vibration frequency is greater than the vibration threshold. If so, switch the working mode to track drive;

[0057] Step 3: When the mobile platform is in wheel drive, determine whether the tilt angle is greater than the tilt angle threshold, and if so, switch the working mode to track drive;

[0058] Step 4: Determine whether the moving speed of the mobile platform is within the moving speed range. If not, control the mobile platform to switch the working mode.

[0059] Furthermore, the method further includes a route optimization step, wherein the route optimization step includes the following steps:

[0060] Step 1: Detect the signal strength between the receiving end and the remote control end;

[0061] Step 2: Divide the working area into three areas: low, medium, and strong according to the signal strength;

[0062] Step 3: Based on the signal strength area, the moving route of the mobile platform is replanned to reduce the time the mobile platform is in the low signal strength area. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 The present invention is a logic block diagram of a mobile platform control system based on multi-source sensor data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following is further described in detail through specific implementation methods:

[0065] The embodiment is basically as shown in the attached Figure 1 As shown:

[0066] It includes a mobile platform body and a mobile platform control system. The mobile platform control system includes a receiving end and a remote control end. The remote control end includes a state detection module, a scene acquisition module, and a control execution module. The remote control end includes a scene analysis module, a work plan module, a driving control module, and a route optimization module.

[0067] The receiving end is installed on the mobile platform, and the scene acquisition module is a camera used to capture images around the mobile platform. The receiving end and the remote control end are connected through the LORA network, and the receiving end sends the acquired images to the remote control end in real time.

[0068] After the remote control receives the image, the scene analysis module analyzes the image. The scene analysis module includes a feature extraction module and a feature analysis module. The feature extraction module uses existing image recognition technology to extract feature models in the image, such as stones, trees, pits, steep slopes, fences, roads, and pendant tools carried by the mobile platform. The feature analysis module identifies the application scenario by analyzing the feature model. Specifically, the feature analysis module analyzes through the trained BP neural network model. The input layer of the BP neural network is the feature model extracted by the feature extraction module. The hidden layer includes the parameters of the feature model, such as stone size, tree density, soil color, fence range, road surface flatness, type of pendant tools, etc. The output layer is a specific application scenario, such as plowing scenes, sowing scenes, transportation scenes, sowing scenes, etc.

[0069] The working modes of the mobile platform include wheel drive and crawler drive. The working plan module includes a mode control module and a route planning module. The mode control module is preset with working modes corresponding to various application scenarios and the working modes and moving speed ranges of the mobile platform under the application scenarios. The mode control module identifies the required working mode according to the identified application scenario. For example, in the plowing scenario, due to the large ground contact pressure of the wheel drive, the tires always walk in the furrows during plowing operations. The tires will cause shearing effects on the soil, causing the soil structure of the plow layer to be destroyed. This will cause a hard bottom layer to form at the soil plow line for a long time, which is not conducive to soil water storage and moisture retention and crop growth. It will also have an adverse effect on the sowing quality and bring difficulties to watering operations. Therefore, when the application scenario is plowing operations, the corresponding working mode should preferably be crawler drive, and the moving speed during operation should be within the range of 6-10km / h. In the transportation scenario, since the traction of the wheel drive is greater than that of the crawler drive, the corresponding working mode should preferably be wheel drive in the transportation scenario.

[0070] At the same time, the route planning module identifies the working area of ​​the mobile platform according to the model and application scenario, and designs the working route. For example, in the plowing scenario, the working area is close to a square, and the working route should cover the entire working area. In the transportation scenario, the working area is strip-shaped, and the working route goes back and forth between the two endpoints. When designing the working route, avoid obstacles such as stones and trees in the feature model.

[0071] The remote control terminal gives the working route, working mode and moving speed range and sends them to the receiving terminal. The receiving terminal controls the mobile platform to drive in the given working mode on the working route and keep the moving speed within the moving speed range according to these data. The remote control terminal is also provided with a display, which is used to display the working route, the current working mode and the moving speed range.

[0072] However, by using image recognition application scenarios, giving the working mode and moving speed is only a vague judgment. There may be some adjustments in the actual operation process of the mobile platform due to obstructed vision or sudden situations. Therefore, when the mobile platform is in operation, the state detection module detects the state of the mobile platform and sends it to the remote control end in real time. The state detection module includes a vibration sensor and a tilt sensor. The vibration sensor is used to detect the vibration frequency of the mobile platform. The driving control module controls the moving speed of the mobile module according to the vibration frequency. The larger the vibration frequency, the slower the moving speed, and it changes linearly within the range of the moving speed. Because the faster the vibration frequency, the bumpier the road surface, the moving speed is reduced. In addition, the mode control module is preset with a vibration threshold. If the working mode is wheel drive, when the vibration frequency exceeds the vibration threshold, the mode control module controls the mobile platform to convert to crawler drive, because when the vibration threshold is exceeded, it means that the road surface is very bumpy. Since the center of gravity of crawler drive is low and the adhesion coefficient is large, the vibration generated on the bumpy road surface is also small, reducing the damage to the mobile platform components. Similarly, the larger the detected tilt angle is, the greater the road slope is. The driving control module controls the moving speed to decrease, and controls the mobile platform to run in the opposite direction according to the direction of the tilt. The mode control module is provided with a tilt angle threshold. If the working mode is wheel drive, when the tilt angle is greater than the tilt angle threshold, the mobile platform is controlled to switch to track drive. The track drive has good anti-rollover performance and slope stability to prevent the mobile platform from overturning.

[0073] The state detection module also includes a speed detection module for detecting the moving speed of the mobile platform, and the driving control module is also used to switch the mode when the moving speed of the mobile platform is not within the moving speed range. Because when the wheel drive is used, if the moving speed is less than the moving speed range, it may be because the tire is stuck in a pit, or because the road surface is slipping, and the friction coefficient is too low. By switching to the crawler drive, it can be effectively solved. If the moving speed is greater than the moving speed range, because the speed is too fast, the working efficiency in the current application scenario will be unsatisfactory. For example, in the plowing scene, the speed is too fast, and the plowing effect may not achieve the expected effect. Under the same power, the moving speed of the crawler drive is slower. By switching to the crawler drive, it is ensured that the moving speed does not exceed the moving speed range. When in the crawler drive, if the moving speed is less than the moving speed range, it may be caused by insufficient traction, and the wheel drive has greater traction, which can be effectively solved. If the moving speed under the crawler drive is greater than the moving speed range, it may be because the current road surface is not particularly bumpy, and there is no need to use the crawler drive, then switch to the wheel drive.

[0074] The remote control terminal also includes a route optimization module, which includes a signal detection module, a region classification module and a route redrawing module. The signal detection module is used to detect the signal strength between the receiving terminal and the remote control terminal. The region classification module divides the working area into areas with low, medium and strong signal strengths according to the signal strength. The route redrawing module replans the route of the mobile platform according to the signal strength area to reduce the time the mobile platform is in the low signal strength area to prevent the connection between the receiving terminal and the remote control terminal from being disconnected due to being in the signal strength area for too long. Specifically, in this embodiment, when replanning the route, the number of times the mobile platform alternates between the medium signal strength area and the low signal strength area is increased to reduce the time the mobile platform is in the low signal strength area at a time.

[0075] This embodiment also discloses a mobile platform control method based on multi-source sensor data, which uses the mobile platform control system based on multi-source sensor data. The method is specifically as follows:

[0076] Scene acquisition steps: The receiving end acquires the image around the mobile platform and sends it to the remote control end;

[0077] Scenario analysis steps: including the following steps:

[0078] Step 1: Extract feature models from images through image recognition;

[0079] Step 2: Analyze the feature model and identify the application scenario through the trained neural network model;

[0080] Work plan step: Generate a work plan for the mobile platform based on the feature model and application scenario, and send the work plan to the receiving end;

[0081] The working scheme steps include the following steps:

[0082] Step 1: The remote control terminal identifies the work area based on the feature model and application scenario, and formulates a work route within the work area;

[0083] Step 2: According to the preset working modes and moving speed ranges corresponding to each application scenario, the corresponding working modes and moving speed ranges are given;

[0084] Driving control steps: Control the mobile platform according to the working plan given by the remote control end;

[0085] Status detection step: During the driving process of the mobile platform, the status of the mobile platform is detected and sent to the remote control end;

[0086] Driving control steps: During the moving driving process, the driving of the mobile platform is controlled according to the state of the mobile platform.

[0087] Further, the state detection step includes the following steps:

[0088] Step 1: Detect the vibration frequency of the mobile platform;

[0089] Step 2: Detect the tilt angle of the mobile platform;

[0090] Step 3: Detect the moving speed of the mobile platform;

[0091] The driving control step comprises the following steps:

[0092] Step 1: Control the moving speed of the mobile platform according to the vibration frequency;

[0093] Step 2: Control the moving speed and direction of the mobile platform according to the tilt angle.

[0094] Furthermore, the mode control step further includes the following steps:

[0095] Step 1: Match the working mode corresponding to the current application scenario;

[0096] Step 2: When the mobile platform is in wheel drive, determine whether the vibration frequency is greater than the vibration threshold. If so, switch the working mode to track drive;

[0097] Step 3: When the mobile platform is in wheel drive, determine whether the tilt angle is greater than the tilt angle threshold, and if so, switch the working mode to track drive;

[0098] Step 4: Determine whether the moving speed of the mobile platform is within the moving speed range. If not, control the mobile platform to switch the working mode.

[0099] Furthermore, the method further includes a route optimization step, wherein the route optimization step includes the following steps:

[0100] Step 1: Detect the signal strength between the receiving end and the remote control end;

[0101] Step 2: Divide the working area into three areas: low, medium, and strong according to the signal strength;

[0102] Step 3: Based on the signal strength area, the moving route of the mobile platform is replanned to reduce the time the mobile platform is in the low signal strength area.

[0103] Embodiment 2

[0104] The difference between this embodiment and the first embodiment is that the remote control end also includes a user interaction module, and the user interaction module includes a direction module, a speed module and a mode module. The direction module is used to output operating instructions for controlling the moving direction of the mobile platform according to the input of the operator, the speed module is used to output operating instructions for controlling the moving speed of the mobile platform according to the input of the operator, and the mode module is used to output operating instructions for controlling the working mode of the mobile platform according to the input of the operator, and the manual control module sends the control instructions to the receiving end.

[0105] The control execution module is also used to execute only the operation instructions issued by the manual control module when the manual control module issues a control instruction.

[0106] Specifically, in this embodiment, the direction module is a steering wheel, the speed module is an accelerator and a brake, and the mode module is a button. The direction module can control the moving direction of the mobile platform, the speed module controls the moving speed of the mobile platform, and the mode module controls the working mode of the mobile platform.

[0107] The remote control terminal also includes a user adaptation module, which includes a user portrait module and a sensitivity adjustment module;

[0108] A user portrait module, used to generate a user portrait of an operator using the user interaction module according to the control instructions issued by the user interaction module and the mobile platform status detected by the status detection module, wherein the user portrait is the user's operation proficiency and operation habits;

[0109] The sensitivity adjustment module is used to adjust the sensitivity of the direction module and the speed module according to the user portrait.

[0110] Specifically, when different operators use the user interaction module to manually control the mobile platform, due to their own habits and different operating proficiency, there will be differences in the control of the mobile platform. For example, when an operator with low proficiency is operating, he may make a sudden acceleration or a sharp turn. At this time, the moving speed of the mobile platform will increase or decrease sharply, and the moving route will also change suddenly. If the road is bumpy, the vibration frequency of the mobile platform will also increase or decrease sharply. Therefore, the user portrait can be generated through the mobile platform state and the control instructions issued by the user interaction module to understand the operator's operating habits. According to the user portrait, the sensitivity of the direction module and the speed model is adjusted. For example, if the operator often accelerates suddenly, the sensitivity of the throttle or brake is reduced. If the operator often turns untimely, the sensitivity of the steering wheel is increased, thereby helping users not to change their operating habits and can operate the mobile platform normally when their proficiency is low.

[0111] The route optimization module is also used to generate a suggested route based on the user portrait and display it on the display.

[0112] If the user's operating proficiency is low, the time spent in low signal strength areas will be reduced when generating and establishing routes to avoid users driving in low signal strength areas for too long and being unable to exit low signal strength areas in time, causing the mobile platform to lose connection. If the user often makes sharp turns, a route with a larger curve angle will be generated when generating recommended routes. If the user often accelerates or decelerates suddenly, a straight-line route will be generated first when generating recommended routes.

[0113] This embodiment also discloses a mobile platform control method based on multi-source sensor data, which uses the mobile platform control system based on multi-source sensor data. The method is specifically as follows:

[0114] User operation steps: Output the operation instructions for the direction, speed and working mode of the mobile platform according to the input quantity operated by the operator;

[0115] User portrait step: Generate a user portrait of the operator based on the mobile platform status and operation instructions;

[0116] Sensitivity adjustment steps: According to the user portrait, adjust the sensitivity of the mobile platform direction and speed response to the operator's input.

[0117] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A mobile platform control system based on multi-source sensor data, comprising a mobile platform, wherein the mobile platform has two working modes: wheel drive and track drive, and is characterized in that: It also includes a receiving end and a remote control end connected to the network, wherein the receiving end is arranged on the mobile platform body and includes a scene acquisition module, a control execution module and a state detection module, and the remote control end includes a scene analysis module, a work plan module and a driving control module; The scene acquisition module is used to acquire the surrounding images of the mobile platform and send them to the remote control end; The scene analysis module is used to analyze images and identify feature models and application scenarios; A work plan module, used to generate a work plan for the mobile platform according to the feature model and application scenario, and send the work method to the mobile platform, wherein the work plan includes a work route, a work mode, and a moving speed range; A control execution module is used to control the mobile platform according to the working plan given by the remote control end; The state detection module integrates multiple sensors to detect the state of the mobile platform when the mobile platform is driving and send the state of the mobile platform to the remote control end; A driving control module, used to control the driving of the mobile platform according to the state of the mobile platform; The remote control terminal also includes a route optimization module, which includes a signal detection module, an area classification module and a route redrawing module; A signal detection module is used to detect the signal strength between the remote control terminal and the receiving terminal; The area classification module is used to divide the working area into areas with low, medium and strong signal strength according to the signal strength; The route replanning module is used to replan the mobile platform's moving route according to the signal strength area to reduce the time the mobile platform is in the low signal strength area; The remote control terminal also includes a user interaction module, which includes a direction module, a speed module and a mode module. The direction module is used to output an operation instruction for controlling the moving direction of the mobile platform according to the input of the operator, the speed module is used to output an operation instruction for controlling the moving speed of the mobile platform according to the input of the operator, and the mode module is used to output an operation instruction for controlling the working mode of the mobile platform according to the input of the operator. The manual control module sends the control instruction to the receiving terminal; The control execution module is also used to execute only the operation instruction issued by the manual control module when the manual control module issues a control instruction; The remote control terminal also includes a user adaptation module, which includes a user portrait module and a sensitivity adjustment module; A user portrait module, used to generate a user portrait of an operator using the user interaction module according to the control instructions issued by the user interaction module and the mobile platform status detected by the status detection module, wherein the user portrait is the user's operation proficiency and operation habits; Sensitivity adjustment module, used to adjust the sensitivity of the direction module and speed module according to the user portrait; The route optimization module is also used to generate a suggested route based on the user portrait and display it on the display.

2. The mobile platform control system based on multi-source sensor data according to claim 1, characterized in that: The scene analysis module includes a feature extraction module and a feature analysis module; A feature extraction module is used to extract feature models from images through image recognition; The feature analysis module is used to analyze the feature model and identify the application scenario through the trained neural network model; The working scheme module includes a route planning module and a mode control module; The route planning module is used to identify the working area according to the feature model and application scenario, and to formulate the working route of the mobile platform within the working area; The mode control module is preset with the working mode and moving speed range corresponding to each application scenario, and gives the corresponding working mode and moving speed range according to the recognition result.

3. The mobile platform control system based on multi-source sensor data according to claim 2, characterized in that: The state detection module includes a vibration sensor and a tilt sensor; Vibration sensor, used to detect the vibration frequency of the mobile platform; A tilt sensor for detecting the tilt angle of the mobile platform; During the traveling of the mobile platform, the traveling control module controls the traveling speed of the mobile platform according to the vibration frequency, and controls the traveling speed and the traveling direction of the mobile platform according to the tilt angle.

4. The mobile platform control system based on multi-source sensor data according to claim 3, characterized in that: The mode control module also presets vibration thresholds and tilt angle thresholds corresponding to the vibration frequency and tilt angle respectively. The mode control module is also used to control the working mode to switch to track drive when the working mode is wheel drive and the vibration frequency or tilt angle exceeds the vibration threshold or the tilt angle threshold.

5. The mobile platform control system based on multi-source sensor data according to claim 4, characterized in that: The state detection module also includes a speed detection module, which is used to detect the speed of the mobile platform. The mode control module is also used to control the mobile platform to switch the working mode when the speed of the mobile platform is not within the moving speed range.

6. A mobile platform control method based on multi-source sensor data, using the mobile platform control system based on multi-source sensor data according to any one of claims 1 to 5, characterized in that: The following steps are involved: Scene acquisition steps: The receiving end acquires the image around the mobile platform and sends it to the remote control end; Scenario analysis steps: including the following steps: Step 1: Extract feature models from images through image recognition; Step 2: Analyze the feature model and identify the application scenario through the trained neural network model; Work plan step: Generate a work plan for the mobile platform based on the feature model and application scenario, and send the work plan to the receiving end; The working scheme steps include the following steps: Step 1: The remote control terminal identifies the work area based on the feature model and application scenario, and formulates a work route within the work area; Step 2: According to the preset working modes and moving speed ranges corresponding to each application scenario, the corresponding working modes and moving speed ranges are given; Driving control steps: Control the mobile platform according to the working plan given by the remote control end; Status detection step: During the driving process of the mobile platform, the status of the mobile platform is detected and sent to the remote control end; Driving control steps: During the moving driving process, the driving of the mobile platform is controlled according to the state of the mobile platform.

7. The mobile platform control method based on multi-source sensor data according to claim 6, characterized in that: The state detection step comprises the following steps: Step 1: Detect the vibration frequency of the mobile platform; Step 2: Detect the tilt angle of the mobile platform; Step 3: Detect the moving speed of the mobile platform; The driving control step comprises the following steps: Step 1: Control the moving speed of the mobile platform according to the vibration frequency; Step 2: Control the moving speed and direction of the mobile platform according to the tilt angle.

8. The mobile platform control method based on multi-source sensor data according to claim 7, characterized in that: The mode control step further comprises the following steps: Step 1: Match the working mode corresponding to the current application scenario; Step 2: When the mobile platform is in wheel drive, determine whether the vibration frequency is greater than the vibration threshold. If so, switch the working mode to track drive; Step 3: When the mobile platform is in wheel drive, determine whether the tilt angle is greater than the tilt angle threshold, and if so, switch the working mode to track drive; Step 4: Determine whether the moving speed of the mobile platform is within the moving speed range. If not, control the mobile platform to switch the working mode.

9. The mobile platform control method based on multi-source sensor data according to claim 8, characterized in that: The route optimization step also includes the following steps: Step 1: Detect the signal strength between the receiving end and the remote control end; Step 2: Divide the working area into three areas: low, medium, and strong according to the signal strength; Step 3: Based on the signal strength area, the moving route of the mobile platform is replanned to reduce the time the mobile platform is in the low signal strength area.