Movable platform control method and system, and movable platform

Abstract

A movable platform control method and system, and a movable platform. The method comprises: acquiring (a) a calibration sample determined by a user and (b) indication information of the user; generating an optimized AI model on the basis of the calibration sample and (b) the indication information, wherein the indication information is used for determining an architecture of the AI model, and the AI model can be recognized by a movable platform; and controlling the movable platform to load the AI model and perform a corresponding function of the AI model. A user can construct, according to requirements thereof, an AI model deployed in a movable platform, so that the requirements of the movable platform in different application scenarios are met, thereby improving the scalability of the movable platform.

Description

Control methods, systems, and mobile platforms for mobile platforms Technical Field

[0001] This application relates to the field of mobile platform technology, and more specifically, to a control method, system, and mobile platform for a mobile platform. Background Technology

[0002] Mobile platforms are widely used in agriculture, logistics, security, and other fields. To make mobile platforms more intelligent in performing various tasks, AI models can be deployed within them to achieve certain functions. Currently, however, these AI models are embedded within the mobile platform, limiting its functionality to the functions corresponding to pre-defined AI models and failing to meet the diverse application scenarios available to mobile platforms. Summary of the Invention

[0003] In view of this, this application provides a control method, system, and mobile platform for a mobile platform.

[0004] According to a first aspect of this application, a control method for a mobile platform is provided, comprising:

[0005] Obtain (a) the calibration sample determined by the user and (b) the user's instruction information;

[0006] Based on the calibration samples and the indication information (b), an optimized AI model is generated, wherein the indication information is used to determine the architecture of the AI ​​model, and the AI ​​model can be recognized by the mobile platform; and

[0007] The mobile platform is controlled to load the AI ​​model and execute the corresponding functions of the AI ​​model.

[0008] According to a second aspect of this application, a control method for a mobile platform is provided, comprising:

[0009] Obtain an optimized AI model, wherein the AI ​​model is generated based on (a) user-defined calibration samples and (b) user-instructed information, wherein the instruction information is used to determine the architecture of the AI ​​model; and

[0010] Load the AI ​​model and execute the corresponding functions of the AI ​​model.

[0011] The solution described in this application allows for the acquisition of information related to the AI ​​model architecture configured by the user based on their own needs, as well as the calibration samples selected by the user. Then, based on this AI model architecture information and the calibration samples, an optimized AI model capable of running on a mobile platform can be generated, enabling the mobile platform to utilize this optimized AI model to achieve relevant functions. In this way, users can define the architecture of the AI ​​model themselves and select appropriate calibration samples for training or optimizing the AI ​​model based on the application scenario of the mobile platform. This results in AI models applicable to different scenarios, meeting the specific needs of the mobile platform in various application scenarios and improving the scalability of the mobile platform.

[0012] According to a third aspect of this application, a control method for a mobile platform is provided, comprising:

[0013] Obtain user instruction information, wherein the instruction information is related to the correlation between the AI ​​model and the airborne sensors of the mobile platform;

[0014] Based on the indicated information, the AI ​​model is associated with one or more airborne sensors of the mobile platform to generate corresponding functional modules, wherein the AI ​​model can be selectively associated with at least one of the airborne sensors to be configured into different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the AI ​​model in the first functional module is at least partially different from the airborne sensor associated with the AI ​​model in the second functional module; and;

[0015] The mobile platform is controlled to load the functional module and execute the corresponding functions of the functional module.

[0016] According to a fourth aspect of this application, a control method for a mobile platform is provided, comprising:

[0017] A functional module is acquired to obtain user instruction information, wherein the functional module is generated by associating the AI ​​model with one or more airborne sensors of the mobile platform based on the instruction information, wherein the AI ​​model can be selectively associated with at least one of the airborne sensors to configure different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the first AI model in the first functional module is at least partially different from the airborne sensor associated with the second AI model in the second functional module; and

[0018] Load the functional module and execute the corresponding functions of the functional module.

[0019] By applying the solutions of this application, AI models can be selectively associated with different types or numbers of airborne sensors in a mobile platform. This allows the AI ​​model to process and analyze the data collected by these airborne sensors to achieve different functional requirements. In this way, the different airborne sensors of the mobile platform can be used to meet the needs of different application scenarios, improve the scalability of the mobile platform, or improve the accuracy of the mobile platform in analyzing and processing the data collected by these sensors, thereby enhancing the accuracy of the mobile platform's operations.

[0020] According to a fifth aspect of this application, a control method for a mobile platform is provided, comprising:

[0021] Obtain user instructions;

[0022] Based on the indicated information, determine the AI ​​model; and

[0023] The mobile platform is controlled to perform functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model, and the second functional module is an onboard functional module fixed to the mobile platform. The second functional module includes the path planning module of the mobile platform.

[0024] According to a sixth aspect of this application, a control method for a mobile platform is provided, comprising:

[0025] Acquiring AI models; and

[0026] The AI ​​model-related functions are executed, wherein the execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model; the second functional module is an airborne functional module fixed to the mobile platform, and the second functional module includes the path planning module of the mobile platform.

[0027] The solution implemented in this application restricts configuration permissions for security-related functional modules within the mobile platform, preventing users from modifying, updating, or customizing these modules. However, for security-independent functional modules, configuration permissions can be granted, allowing users to define their own AI models. This enables the mobile platform to meet the needs of different application scenarios or improve the accuracy of operations within the same work environment. Simultaneously, the security of the mobile platform remains unaffected.

[0028] According to a seventh aspect of this application, a mobile platform includes:

[0029] At least one processor; and

[0030] At least one memory including computer program code, wherein the at least one memory and the computer program code, together with at least one processor, are configured to enable the mobile platform to perform at least the following operations:

[0031] Obtain an optimized AI model, wherein the AI ​​model is generated based on (a) calibration samples determined by the user and (b) user instruction information, wherein the instruction information is used to determine the architecture of the AI ​​model; and load the AI ​​model and execute the corresponding functions of the AI ​​model;

[0032] and / or

[0033] The system acquires a functional module corresponding to user instruction information, wherein the functional module is generated by associating the AI ​​model with one or more airborne sensors of the mobile platform based on the instruction information, wherein the AI ​​model can be selectively associated with at least one of the airborne sensors to configure different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the first AI model in the first functional module is at least partially different from the airborne sensor associated with the second AI model in the second functional module; and loads the functional module and executes the corresponding function of the functional module.

[0034] and / or

[0035] The system acquires an AI model and executes functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model. The second functional module is an airborne functional module embedded in the mobile platform and includes a path planning module of the mobile platform.

[0036] According to an eighth aspect of this application, a control system for a mobile platform is provided, comprising:

[0037] One or more control terminals are configured to acquire (a) calibration samples determined by a user and (b) user instruction information, and generate an optimized AI model based on the calibration samples and (b) the instruction information, wherein the instruction information is used to determine the model architecture of the AI ​​model, and the optimized AI model can be recognized by the mobile platform; and

[0038] A mobile platform, communicatively connected to at least one of the control terminals.

[0039] The mobile platform is used to load the AI ​​model and execute functions related to the AI ​​model.

[0040] According to a ninth aspect of this application, a control system for a mobile platform is provided, comprising:

[0041] One or more control terminals are used to acquire user instruction information, wherein the instruction information is related to the association between the AI ​​model and the airborne sensors of the mobile platform;

[0042] Based on the indicated information, the AI ​​model is associated with one or more airborne sensors of the mobile platform to generate corresponding functional modules. The AI ​​model can be selectively associated with at least one of the airborne sensors to be configured into different functional modules, including a first functional module and a second functional module. The airborne sensor associated with the AI ​​model in the first functional module is at least partially different from the airborne sensor associated with the AI ​​model in the second functional module.

[0043] A mobile platform, communicatively connected to at least one of the control terminals.

[0044] The mobile platform is used to acquire the functional module, load the functional module, and execute the corresponding functions of the functional module.

[0045] According to a tenth aspect of this application, a control system for a mobile platform is provided, comprising:

[0046] One or more control terminals are used to acquire user instruction information; and to determine an AI model based on the instruction information.

[0047] A mobile platform, communicatively connected to at least one of the control terminals.

[0048] The mobile platform acquires the AI ​​model and performs functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model, and the second functional module is an onboard functional module fixed to the mobile platform. The second functional module includes the path planning module of the mobile platform.

[0049] According to the eleventh aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed, implements the methods mentioned in any one of the first to sixth aspects above.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 is a schematic diagram of an application scenario of an embodiment of this application.

[0053] Figure 2 is a schematic diagram of an application scenario of another embodiment of this application.

[0054] Figure 3 is a schematic diagram of an application scenario of another embodiment of this application.

[0055] Figure 4 is a schematic diagram of an application scenario of another embodiment of this application.

[0056] Figure 5 is a schematic diagram of an application scenario of another embodiment of this application.

[0057] Figure 6 is a flowchart of a control method for a mobile platform according to an embodiment of this application.

[0058] Figure 7(a) is a schematic diagram of a piping identification scheme according to an embodiment of this application.

[0059] Figure 7(b) is a schematic diagram of a piping identification scheme according to another embodiment of this application.

[0060] Figure 7(c) is a schematic diagram of deploying an AI model in a mobile platform according to another embodiment of this application.

[0061] Figure 8 is a flowchart of a control method for a mobile platform according to another embodiment of this application.

[0062] Figure 9 is a flowchart of a control method for a mobile platform according to another embodiment of this application.

[0063] Figure 10 is a schematic diagram of the logical structure of a mobile platform according to an embodiment of this application. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] Mobile platforms are widely used in agriculture, logistics, security, and other fields. To make these platforms more intelligent in performing various tasks, AI models can be deployed within them to achieve specific functions. In some technologies, deploying AI models on mobile platforms involves pre-installing general-purpose AI models at the factory. However, these general-purpose models can only perform general functions and cannot meet the specific needs of different application scenarios. For example, taking target recognition models as an example, general-purpose target recognition models can typically be deployed at the factory. These models can only identify targets such as people, vehicles, and ships. However, different users have different needs for mobile platforms. For instance, some users need to use mobile platforms for pipeline inspection, while others need them for power line inspection. In other words, different users may need to use AI models to identify different types of targets, and current AI model deployment solutions cannot meet the diverse needs of users.

[0066] Some approaches involve providing an interface within the mobile platform, allowing users to connect to a self-developed computing power box. This box can store AI models that the user has developed or trained. The mobile platform can then call upon these AI models to perform certain functions during task execution. However, this approach requires users to possess AI model development capabilities, making it unsuitable for ordinary users. Furthermore, the user-developed computing power box can significantly impact the mobile platform's battery life and security.

[0067] Therefore, in order to meet the needs of different users, it is necessary to provide a solution that allows for the flexible deployment of various AI models on mobile platforms, so that mobile platforms can meet the needs of users in different application scenarios and improve the scalability of mobile platforms.

[0068] Based on this, one embodiment of this application provides a control method for a mobile platform. This method can acquire information related to the AI ​​model architecture configured by the user based on their own needs, as well as calibration samples selected by the user. Then, based on the AI ​​model architecture information and calibration samples, an optimized AI model capable of running on the mobile platform can be generated, allowing the mobile platform to utilize this optimized AI model to achieve relevant functions. In this way, users can define the architecture of the AI ​​model themselves and select appropriate calibration samples for training or optimizing the AI ​​model based on the application scenario of the mobile platform. This results in AI models applicable to different scenarios, meeting the specific needs of the mobile platform in various application scenarios and improving the scalability of the mobile platform.

[0069] Figure 1 illustrates an application scenario according to an embodiment of this application. The mobile platform control system may include a mobile platform and one or more control terminals, wherein the mobile platform is communicatively connected to at least one of the one or more control terminals. The one or more control terminals can be used to acquire calibration samples determined by the user and instruction information issued by the user to determine the architecture of the AI ​​model. Based on this information, an optimized AI model that can be recognized by the mobile platform is generated and sent to the mobile platform. The mobile platform then acquires the optimized AI model and loads it during task execution to perform AI model-related functions.

[0070] For example, as shown in Figure 2, in some embodiments, the one or more control terminals may include a developer front-end, a model training and / or quantization platform. The developer front-end can provide a user interface through which the user can determine the aforementioned calibration samples and input instruction information for determining the architecture of the AI ​​model. The developer front-end can send this information to the model training and / or quantization platform, which can generate an optimized AI model that can be recognized by the mobile platform based on this information. The optimized AI model can then be directly sent to the mobile platform so that the mobile platform can load the optimized AI model.

[0071] In some embodiments, to allow users to decide whether and when to deploy the optimized AI model on the mobile platform, the model training and / or quantization platform can send the generated optimized AI model to the ground control device of the mobile platform. When the user wishes to deploy the optimized AI model on the mobile platform, they can input a confirmation command through the ground control device, which will then send the optimized AI model to the mobile platform. Therefore, as shown in Figure 3, the one or more control terminals may include a developer front-end, a model training and / or quantization platform, and a ground control device. The developer front-end can provide a user interface through which the user can determine the calibration samples and input instructions for determining the architecture of the AI ​​model. The developer front-end can send this information to the model training and / or quantization platform, which can then generate an optimized AI model that can be recognized by the mobile platform and then distribute it to the ground control device of the mobile platform. Once the ground control equipment detects the confirmation command input by the user, it can send the optimized AI model to the mobile platform, so that the mobile platform can store the optimized AI model and load it during the execution of the task.

[0072] In some embodiments, the functionality of the developer's front-end can be integrated into the ground control device. For example, as shown in Figure 4, the one or more control terminals may include the ground control device, a model training and / or quantization platform. The ground control device can provide a user interface through which the user can determine the calibration samples and input indication information for determining the architecture of the AI ​​model. The ground control device can send this information to the model training and / or quantization platform. After the model training and / or quantization platform generates an optimized AI model based on the calibration samples and the indication information for determining the architecture of the AI ​​model, it can send the optimized AI model to the ground control device so that the ground control device can send it to the mobile platform.

[0073] In some embodiments, the functions of the developer front-end and the model training and / or quantization platform can be integrated into the ground control device, meaning the control terminal includes only one ground control device. For example, as shown in Figure 5, the control terminal may consist only of the ground control device, which can provide a user interface. The user can use this interface to determine the calibration samples and input instruction information for determining the architecture of the AI ​​model. The ground control device can then generate an optimized AI model based on this information and send the optimized AI model to the mobile platform.

[0074] The mobile platform in this application embodiment may be at least one of aircraft, vehicles, ships, and mobile robots, but is not limited thereto.

[0075] Aircraft can include rotorcraft, fixed-wing aircraft, or hybrid fixed-wing / rotorcraft. Rotorcraft can be single-rotor, dual-rotor, tri-rotor, quadcopter, hexacopter, octocopter, decacopter, or dodeccopter. Aircraft can include, but are not limited to, manned aircraft, logistics aircraft, aerial photography aircraft, agricultural plant protection aircraft, and industry rescue aircraft. The above are merely illustrative examples, and this application does not specifically limit the type of aircraft. Aircraft include unmanned aerial vehicles (UAVs) and manned aircraft; aircraft can be used for tasks such as aerial photography, aerial reconnaissance, geographic mapping, environmental monitoring, and security patrol.

[0076] The ground control equipment in this application embodiment can be various devices that are communicatively connected to the mobile platform for controlling the mobile platform. For example, it can be a remote control, mobile phone, tablet, computer, head-mounted glasses, ground base station, etc. that are associated with the mobile platform. This application embodiment does not impose any restrictions.

[0077] In this application embodiment, the developer frontend can be any device running a developer frontend client, such as a user's mobile phone, computer, etc. This developer frontend client can be an app or a web client, providing an interactive interface for the user to select calibration samples and receive instructions related to the AI ​​model architecture.

[0078] In the embodiments of this application, the model training and / or quantization platform can be various devices with model training and / or quantization functions, such as a single server or a server cluster. The model training and / or quantization platform can provide multiple functions such as AI model generation, training, optimization, and quantization.

[0079] As shown in Figure 6, the mobile platform control method provided in this application embodiment may include the following steps:

[0080] S602, One or more control terminals acquire the calibration samples determined by the user and the user's instruction information, the instruction information being used to determine the architecture of the AI ​​model;

[0081] In step S602, the one or more control terminals can acquire the calibration samples determined by the user and the indication information input by the user for determining the architecture of the AI ​​model. For example, the one or more control terminals can provide an AI model configuration interface, where the user can configure information related to the model architecture as the aforementioned indication information based on their own needs. The indication information can be various types of information used to determine the AI ​​model architecture, and this embodiment does not impose any limitations.

[0082] For example, in some scenarios, the AI ​​model configuration interface can provide a variety of task scenarios that can be executed on mobile platforms, such as power inspection tasks, piping detection tasks, traffic inspection tasks, etc. For each task scenario, a model architecture adapted to the task scenario can be predefined. Users only need to select the corresponding task scenario based on their own needs. As the above-mentioned instruction information, one or more control terminals can determine the corresponding AI model architecture based on the task scenario selected by the user.

[0083] In some scenarios, users can also directly input the requirements of the task scenario in the configuration interface as the aforementioned instructions. The one or more control terminals can then automatically determine a suitable AI model architecture for the user based on these requirements.

[0084] In some scenarios, users can also select the type and number of AI models through the configuration interface as instruction information, and one or more control terminals can then determine the architecture of the AI ​​models based on the information selected by the user.

[0085] In some scenarios, users can also select the type of sensors mounted on the mobile platform associated with the AI ​​model through the configuration interface. For example, if the mobile platform is equipped with sensors such as infrared cameras, visible light cameras, and LiDAR, the user can select the infrared camera if the AI ​​model is to be used to recognize infrared images captured by the infrared camera, and select the LiDAR if the AI ​​model is to be used to recognize point cloud data captured by the LiDAR. The one or more control terminals can then automatically determine the appropriate AI model architecture based on the sensor types selected by the user.

[0086] In addition, users can determine calibration samples based on their own needs. These calibration samples are used to train and / or optimize AI models. Calibration samples are samples carrying labels that indicate the true result corresponding to that sample. For example, if the calibration sample is an image, and this calibration sample is used to train a model that can identify cattle and sheep in an image, then the label is the image region corresponding to the cattle and sheep that has been pre-marked in the image.

[0087] S604. One or more control terminals generate an optimized AI model based on the calibration sample and the user's instruction information, wherein the optimized AI model can be recognized by the mobile platform.

[0088] In step S604, after obtaining the calibration sample and indication information, the one or more control terminals can generate an optimized AI model that can be recognized by the mobile platform based on the calibration sample and indication information. The ability to be recognized by the mobile platform means that the AI ​​model can run on the mobile platform.

[0089] S606, a mobile platform for acquiring optimized AI models;

[0090] In step S606, the mobile platform can obtain the optimized AI model. For example, after generating the optimized AI model, the one or more control terminals can directly send it to the mobile platform, or they can send the optimized AI model to the mobile platform after detecting the user's confirmation command. The mobile platform can store the optimized AI model.

[0091] S608, The mobile platform loads the optimized AI model and executes the corresponding functions.

[0092] In step S608, the mobile platform can load the optimized AI model and execute the corresponding functions during the execution of the task.

[0093] In this embodiment, users can build and deploy AI models on mobile platforms based on their own needs, thereby meeting the needs of mobile platforms in different application scenarios and improving the scalability of mobile platforms.

[0094] In some embodiments, the optimized AI model can be trained with incremental samples before being sent to the mobile platform. For example, after the optimized AI model is generated on the control end based on calibration samples and instruction information, if the user feels that the performance of the optimized AI model still needs improvement, new sample data can be used to further train the optimized AI model to further optimize it and improve its performance.

[0095] In some embodiments, when generating an optimized AI model based on calibration samples and user instructions, a preliminary model can first be generated based on the calibration samples and user instructions. This preliminary model can then be transformed to obtain a model that the mobile platform can recognize, which serves as the optimized AI model. For example, considering that the preliminary model might have a complex network structure, many model parameters, and high parameter accuracy, such models require significant storage space and computing resources. Given the limited memory and computing resources of the mobile platform, such complex and large models may not be deployable or usable on the platform. Therefore, after generating the preliminary model, it can be transformed to obtain a model with a simpler network structure, fewer model parameters, and lower parameter accuracy, which serves as the optimized AI model. This transformation can involve converting the network structure and model parameters of the preliminary model. For instance, if the preliminary model is a neural network model, some network layers can be removed to reduce the number of layers and make the network structure more lightweight. Alternatively, the model parameters can be quantized, converting high-precision model parameters into low-precision model parameters.

[0096] In some embodiments, the conversion of the preliminary model can be a process of quantization. For example, when generating an optimized AI model based on calibration samples and user instructions, a preliminary model can be generated first, and then quantized to obtain the optimized AI model. Alternatively, the model architecture of the AI ​​model can be determined based on user instructions. After determining the architecture, the AI ​​model can be pre-trained using calibration samples to obtain the preliminary model. Typically, to ensure model accuracy, the precision of the model parameters during training is often high; for example, 32-bit floating-point numbers are commonly used, resulting in high precision model parameters in the trained preliminary model. However, considering the limited processing resources of mobile platforms and the need to minimize the impact of the AI ​​model on the platform's battery life, the AI ​​model deployed on mobile platforms needs to be as lightweight as possible. That is, the precision of the model parameters should not be too high to avoid excessive computation and excessive consumption of the platform's processing resources. Therefore, after obtaining the preliminary model, it can be quantized to convert the high-precision model parameters into low-precision model parameters. For example, 32-bit model parameters can be converted into 16-bit or 8-bit model parameters to obtain an optimized AI model.

[0097] In some embodiments, the optimized AI model is a PyTorch model. PyTorch is an open-source deep learning framework that provides a flexible toolset for building, training, and deploying neural network models, particularly suitable for deep learning tasks such as computer vision, natural language processing, and reinforcement learning. Therefore, in some embodiments, the PyTorch deep learning framework can be used to build and train the AI ​​model when generating the optimized AI model; that is, the optimized AI model is a PyTorch model.

[0098] In some embodiments, after quantizing the preliminary model, the performance of the quantized preliminary model can be evaluated and optimized using the aforementioned calibration samples. For example, the user-defined calibration samples may include two parts: training samples and validation samples. The training samples can be used to train the constructed model to obtain the aforementioned preliminary model. After quantizing the preliminary model, the performance of the quantized preliminary model can be verified using validation samples. If the performance of the quantized preliminary model is found to be poor, the performance of the quantized preliminary model can be further optimized using the training samples to obtain the optimized AI model.

[0099] In some embodiments, the preliminary model can be trained on the user's local device. For example, the user can input instructions to determine the model architecture through an interactive interface provided by the developer's front end. The model training and / or quantization platform can determine the model architecture based on the instructions to obtain the original model. The user can download the source code of the original model through the developer's front end and train the original model on their own device (i.e., the user's local device) using calibration samples to obtain the preliminary model of the AI ​​model.

[0100] In some embodiments, the preliminary model can also be trained directly in the aforementioned model training and / or quantization platform. For example, users can input instruction information for determining the model architecture and upload calibration samples (or select calibration samples from the sample library provided by the developer front-end) through the interactive interface provided by the developer front-end. The model training and / or quantization platform can determine the model architecture based on the instruction information, obtain the original model, and then use the calibration samples to train the original model to obtain the preliminary model.

[0101] In some embodiments, if the preliminary model is trained on the user's local device, when quantizing the preliminary model, the model parameters of the preliminary model uploaded by the user can be obtained, the model parameters of the preliminary model can be quantized based on the calibration sample, and the quantized model parameters can be used as the model parameters of the AI ​​model.

[0102] In some embodiments, the indication information may be one or more of the following: indication information related to the selection of AI model type, indication information related to the selection of AI model input information type, indication information related to the selection of AI model output information type, and indication information related to the selection of the type, number, and / or number of runs of the AI ​​model's sub-models.

[0103] For example, the control panel can provide an AI model framework configuration interface, offering options for model types so users can choose the appropriate type based on their needs. Model types could include convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, generative adversarial networks (GANs), Transformers, and so on. Different model types excel at handling different tasks. For instance, CNNs are adept at image processing and recognition; therefore, users who wish to use this AI model for image recognition can choose a CNN. RNNs excel at processing sequential data; therefore, users who wish to use this AI model for sequential data processing can choose a RNN. GANs are suitable for image and text generation; therefore, users who wish to use this AI model for text and image generation can choose a GAN.

[0104] The configuration interface can also provide options for the type of input information for the model. For example, a mobile platform can be equipped with one or more sensors such as visible light cameras, infrared cameras, LiDAR, and ultrasonic devices. Therefore, the model's input information can be data collected by these different types of sensors. Thus, the configuration interface can provide data type options so that users can select one or more data types as the type of input information for the model.

[0105] Similarly, the tasks that AI models need to perform vary depending on the scenario. For example, AI models can be used to perform tasks such as object recognition, classification, and obstacle avoidance. Therefore, the type of output information of the model can be selected based on the task type. Thus, this configuration interface can also provide options for the type of output information, allowing users to select the appropriate type based on their actual needs. For instance, if the model's task is to recognize target objects in an image, the user can select the type of target object through this configuration interface, such as a person, vehicle, or ship.

[0106] In some scenarios, the AI ​​model can include multiple sub-models, and users can select the type of sub-model, the number of sub-models, and the number of times each sub-model is run through the configuration interface. In some embodiments, the type of input information for the AI ​​model is related to the type of onboard sensors of the mobile platform. For example, the mobile platform can be equipped with one or more sensors such as visible light cameras, infrared cameras, LiDAR, and ultrasonic devices, and the AI ​​model can be used to analyze the data collected by these sensors. For instance, it can analyze whether a bridge has cracks or whether power equipment has safety hazards based on visible light images collected by a visible light camera, or analyze whether there are obstacles based on point cloud data collected by LiDAR. Therefore, the input information of the model can be data collected by the above-mentioned different types of sensors. Therefore, multiple types of options can be provided in the configuration interface so that users can select one or more data types as the type of input information for the model.

[0107] In some embodiments, the type of output information of the AI ​​model is related to the type of onboard payload of the mobile platform. The onboard payload can be a gimbal, camera, robotic arm, etc., mounted on the mobile platform. Different onboard payloads typically perform different types of tasks, thus affecting the output information of the AI ​​model. Therefore, users can use the type of onboard payload selected by the mobile platform as the output information of the AI ​​model. For example, if the onboard payload of the mobile platform includes a camera or a nozzle, the AI ​​model can be used to decide whether to use the camera or the nozzle based on the working environment.

[0108] In some embodiments, the calibration sample can be provided by the user. For example, the user can collect sample data based on their own needs, calibrate the sample data to obtain the calibration sample, and then upload the calibration sample to the control terminal so that the control terminal can generate an optimized AI model based on the calibration sample uploaded by the user and the instruction information of the model architecture input by the user.

[0109] In some embodiments, the calibration sample can be a pre-configured calibration sample in the control terminal, and the user only needs to select a suitable calibration sample from the control terminal based on their own needs. For example, the control terminal can be pre-set with a calibration sample library, which includes pre-calibrated calibration sample data of different types, so that the user can select a calibration sample that suits their needs for training and / or optimizing the model.

[0110] In some embodiments, the calibration sample is related to the execution result of the function that the AI ​​model can achieve. For example, the calibration sample can be used to improve the accuracy and precision of the execution result.

[0111] In some embodiments, the calibration samples are related to the quality of the performance results of the functions that the AI ​​model can achieve. That is, the calibration samples can be used to train or optimize the AI ​​model to improve the accuracy of the AI ​​model in performing certain functions. For example, taking the AI ​​model as a target recognition model, the calibration samples can be used to improve the accuracy of the AI ​​model in recognizing targets.

[0112] In some embodiments, the calibration sample is related to the efficiency of the execution results of the functions that the AI ​​model can achieve. That is, the calibration sample can be used to train or optimize the AI ​​model to improve the efficiency of the AI ​​model in performing certain functions. For example, taking the AI ​​model as a target recognition model, the calibration sample can be used to improve the speed of the AI ​​model in recognizing targets, thereby improving recognition efficiency.

[0113] In some embodiments, when determining a calibration sample, the user can select one or more of the data type of the calibration sample and the type of the target object in the calibration sample.

[0114] In some embodiments, the data type of the calibration sample is related to the type of airborne sensors on the mobile platform. For example, AI models deployed in the mobile platform are typically used to analyze and process data collected by sensors mounted on the mobile platform; therefore, the data type of the calibration sample is typically the same as the type of data collected by the airborne sensors.

[0115] In some embodiments, the data type of the calibration sample includes at least one of the following: visible light image, infrared image, ultrasonic data, and point cloud data from lidar.

[0116] In some embodiments, the type of the target object in the calibration sample is the type of the target object to be identified or tracked in the calibration sample. For example, the target object to be identified or tracked may be a person, a vehicle, or a cow or sheep, etc., and the user can select the appropriate type of target object based on their own needs.

[0117] In some embodiments, when a mobile platform loads an optimized AI model to implement a certain function, the execution of that function requires running one or more functional modules, wherein the AI ​​model needs to be loaded during the operation of at least one functional module.

[0118] In some embodiments, each functional module is used to perform one or more sub-functions. For example, taking a drone as a mobile platform, when the drone is used to perform power line inspection tasks, the implementation of this power line inspection function often relies on multiple functional modules, such as a route generation module, an automatic inspection module, a target recognition and tracking module, and a defect detection module. These functional modules work together to complete the aforementioned power line inspection task. Each functional module implements one or more sub-functions. For example, the route generation module plans the route for the drone, the automatic inspection module controls the drone's movement within the route to perform the inspection, the target recognition and tracking module identifies and tracks power equipment, and the defect detection module detects defects in the power equipment, such as broken wires. During operation, one or more of these functional modules can load the AI ​​model to implement the corresponding sub-function. For example, the target recognition and tracking module can load the AI ​​model to identify and track power equipment.

[0119] In some embodiments, the sub-function includes at least one of the following: path planning function, path execution function, obstacle perception function, obstacle avoidance function, target recognition function, target tracking function, and target classification function.

[0120] In some embodiments, the route planning function includes at least one of the following: automatic route generation function, automatic route update function, and automatic route suggestion function.

[0121] In some embodiments, the path execution function includes at least one of the following: automatic path execution function and path switching prompt function.

[0122] In some embodiments, the obstacle perception function includes at least one of the following: obstacle recognition function, obstacle location perception function.

[0123] In some embodiments, the obstacle avoidance function includes at least one of the following: obstacle avoidance path planning function, obstacle warning function.

[0124] In some embodiments, the target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

[0125] In some embodiments, the target tracking function includes at least one of the following: automatic target following function, and automatic target preset distance maintenance function.

[0126] In some embodiments, the target classification function includes at least one of the following: a target identification function for a specific type, and a target filtering function for a specific type.

[0127] Considering that the implementation of a function in a mobile platform may depend on the operation of multiple functional modules, and some of these modules are closely related to the security of the mobile platform, granting configuration permissions to users for these modules—for example, allowing users to define their own AI models for implementing the corresponding sub-functions—could pose significant security risks. Therefore, functional modules related to the security of the mobile platform can be permanently embedded within the platform, and users should not have permission to update or modify these modules. For functional modules less related to the security of the mobile platform, configuration permissions can be granted to users; for example, users can define the AI ​​models used by these modules to implement their respective sub-functions. This allows the mobile platform to meet the needs of different users, improve its scalability, and ensure that its security remains unaffected.

[0128] Therefore, in some embodiments, when a mobile platform loads an optimized AI model to implement a certain function, the execution of that function requires the running of multiple functional modules. Some of these functional modules can load the AI ​​model during their operation. That is, certain functional modules that are not closely related to the security of the mobile platform can load user-built AI models to achieve the user's expected functions.

[0129] In some embodiments, another portion of the functional modules are airborne functional modules embedded in the mobile platform. These airborne functional modules embedded in the mobile platform do not grant users configuration permissions; that is, users cannot modify or update these types of functional modules, nor can they define the AI ​​models required by these functional modules.

[0130] In some embodiments, the airborne functional modules embedded in the mobile platform can be functional modules related to the safety control of the mobile platform. For example, path planning and motion trajectory control of the mobile platform are closely related to the motion safety of the mobile platform. Therefore, these functional modules can be embedded in the mobile platform as airborne functional modules at the time of manufacture, and their configuration permissions cannot be disclosed to the outside world.

[0131] In some embodiments, the airborne functional modules embedded in the mobile platform include at least one of the following: a pose control module for the mobile platform, a trajectory control module for the mobile platform, and a path planning module for the mobile platform. The path planning, pose control, and trajectory control of the mobile platform determine the mobile platform's...

[0132] The movement trajectory and posture of the mobile platform determine whether the mobile platform will collide with obstacles during movement. For this type of functional module, which is closely related to the safety of the mobile platform, configuration permissions are not open to the public.

[0133] In some embodiments, the functional modules that load the AI ​​model are related to the load operation mode of the mobile platform. For example, the load in the mobile platform can be various sensors, robotic arms, etc., mounted on the mobile platform. These loads have different operation modes, which require different functions from the mobile platform. Therefore, a corresponding AI model can be built based on the functional requirements corresponding to the load's operation mode to achieve the corresponding function.

[0134] In some embodiments, the optimized AI model may include a first AI model and a second AI model different from the first AI model. Correspondingly, the functions implemented by the mobile platform using the optimized AI model may include a first function and a second function different from the first function. The AI ​​model may include both the first and second AI models. The functional module implementing the first function can load the first AI model during operation, and the functional module implementing the second function can load the second AI model during operation. That is, different AI models can be constructed to implement different types of functions required for the mobile platform. For example, for the piping detection function of the mobile platform, a piping detection model can be constructed to identify piping areas; for the power inspection function of the mobile platform, a power equipment defect identification model can be constructed to identify defects in the power equipment.

[0135] In some embodiments, the execution of the first function and the execution of the second function include the operation of the same functional modules. That is, when executing the first function and the second function, some of the functional modules to be run are the same functional modules.

[0136] In some embodiments, the same functional module is an airborne functional module embedded in a mobile platform.

[0137] For example, taking a drone as a mobile platform, the drone can perform two functions: power grid inspection and piping detection. The power grid inspection function relies on a flight path planning module to plan the flight path, an automatic inspection module to control the drone's inspection, and a defect identification module to identify defects in power equipment (such as broken power lines). The piping detection function relies on the flight path planning module to plan the flight path, the automatic inspection module to control the drone's inspection, and the piping detection module to identify piping areas on the ground. The flight path planning module and the automatic inspection module can be fixed onboard modules within the drone, running simultaneously when performing both power grid inspection and piping detection. The defect identification module can load a first AI model to identify defects in power equipment, while the piping detection module can load a second AI model to identify piping areas.

[0138] In some embodiments, in a scenario where the optimized AI model includes a first AI model and a second AI model, the calibration sample may also include a first calibration sample and a second calibration sample, wherein the first calibration sample is different from the second calibration sample. The first calibration sample is used to train and / or optimize the first AI model, and the second calibration sample is used to train and / or optimize the second AI model. The first calibration sample and the second calibration sample may be based on the functional settings required to be implemented by the two AI models.

[0139] Considering that mobile platforms typically carry different types of sensors, these sensors can collect data to perform specific functions during task execution. Given that sensor data can be used for different functions in different application scenarios, and to leverage the existing airborne sensors within the mobile platform to meet the needs of various applications, improve the platform's scalability, or enhance the accuracy of data analysis, thereby improving operational accuracy, in some embodiments, AI models can be selectively associated with different types or numbers of airborne sensors within the mobile platform. This allows the AI ​​model to process and analyze the data collected by these sensors to achieve various functionalities.

[0140] For example, in some embodiments, the mobile platform includes multiple airborne sensors, and the AI ​​model can be selectively associated with the sensing information of at least one airborne sensor to be configured into different functional modules, wherein the airborne sensors associated with the AI ​​models in the different functional modules are at least partially different. For instance, the AI ​​model includes a first AI model and a second AI model. The first AI model is associated with the sensing information of one or more airborne sensors and configured as a first functional module, and the second AI model is associated with the sensing information of one or more airborne sensors and configured as a second functional module. The airborne sensors associated with the first AI model are at least partially different from those associated with the second AI model.

[0141] In some embodiments, the number and / or type of airborne sensors associated with the AI ​​model in the first functional module differs at least partially from the number and / or type of airborne sensors associated with the AI ​​model in the second functional module. For example, the types of airborne sensors associated with the AI ​​models in the two functional modules may be different. For instance, the first functional module is an obstacle avoidance module, and the airborne sensors associated with the AI ​​model in the first functional module are LiDAR; the second functional module is a surge detection module, and the airborne sensors associated with the AI ​​model in the second functional module are visible light cameras and infrared cameras.

[0142] Alternatively, the types of airborne sensors associated with the AI ​​models in two functional modules can be the same, but the number can be different. For example, the airborne sensors associated with the AI ​​models in two functional modules are both visible light cameras. In one functional module, the AI ​​model is associated with two visible light cameras, while in the other functional module, the AI ​​model is associated with only one visible light camera.

[0143] In some embodiments, the perception information of the airborne sensor associated with the AI ​​model in each functional module can be used as the input information of the AI ​​model. For example, assuming that the airborne sensor associated with the AI ​​model of the first functional module is a LiDAR, the point cloud data collected by the LiDAR can be used as the input information of the AI ​​model.

[0144] In some embodiments, the number and / or frequency of AI models used in the first functional module differs from the number and / or frequency of AI models used in the second functional module. The difference in the number of AI models used refers to the different number of AI models that the functional modules need to load during operation. For example, the first functional module can load two AI models during operation, while the second functional module can load one AI model.

[0145] The different number of times the AI ​​model is used refers to the different number of times each AI model is loaded during the operation of the functional module. For example, for a certain AI model, the first functional module needs to be loaded twice during the operation, while the second functional module needs to be loaded once during the operation.

[0146] In some embodiments, the number of times the AI ​​model is used and / or the number of times it is used in the first functional module is less than the number of times the AI ​​model is used and / or the number of times it is used in the second functional module.

[0147] In some embodiments, the number and / or type of airborne sensors associated with the AI ​​model in the first functional module are less than the number and / or type of airborne sensors associated with the AI ​​model in the second functional module.

[0148] In some embodiments, the type of airborne sensor associated with at least one AI model in the second functional module is different from the type of airborne sensor associated with at least one AI model in the first functional module. For example, in some scenarios, the operation of both the first and second functional modules may depend on multiple AI models, wherein at least one AI model in the two functional modules is associated with a different type of airborne sensor.

[0149] Taking drone-based piping inspection as an example, piping refers to the phenomenon where groundwater suddenly flows out in large quantities and gushs upwards through cracks and fissures in the soil during engineering construction. It commonly occurs in water conservancy, hydropower, railway, and highway projects. Piping can cause soil instability and even damage to infrastructure. With technological advancements, drones can be used for piping inspection to identify piping. Considering that drones typically carry sensors such as wide-angle cameras, telephoto cameras, and infrared cameras, these sensors can be correlated with AI models to obtain a solution that can accurately identify piping.

[0150] For example, considering that wide-angle cameras can capture a larger area with a wider field of view, but the objects in the image are smaller, while telephoto cameras can capture a smaller area with a narrower field of view, but the objects appear larger, we can first use images captured by the wide-angle camera to perform preliminary identification of piping surges, that is, to initially screen out areas suspected of piping surges from the images captured by the wide-angle camera, and then use images captured by the telephoto camera to perform fine identification of the piping surges.

[0151] Furthermore, considering that groundwater may seep to the surface during piping, potentially causing abnormal surface temperatures, infrared images captured by infrared cameras can also be used to identify piping. For example, infrared images can be used to quickly locate areas with abnormal heat, thereby inferring the presence of piping.

[0152] Based on the characteristics of the aforementioned sensors, different piping identification schemes can be constructed. For example, in some scenarios, images acquired by infrared cameras and telephoto cameras can be used to identify piping areas. Therefore, as shown in Figure 7(a), an AI model 1 can be constructed and associated with the infrared camera and the telephoto camera, so that the AI ​​model 1 can identify piping in infrared images and visible light images and obtain piping identification results.

[0153] In some scenarios, images captured by infrared and wide-angle cameras can be used to initially identify piping areas. Then, a telephoto camera can be used to capture an image of the area, i.e., a magnified image of the area. Further identification of whether the area is a piping area can be based on the magnified image captured by the telephoto camera. Therefore, as shown in Figure 7(b), an AI model 2 can be constructed. This AI model 2 can be associated with the infrared and wide-angle cameras to perform initial identification of piping areas in these two types of images, thus initially screening suspected piping areas. Then, an AI model 3 can be constructed, which can be associated with the telephoto camera to perform secondary identification of suspected piping areas in the images captured by the telephoto camera, to determine whether the suspected piping areas are indeed piping areas.

[0154] In some embodiments, the AI ​​model or a sub-model of the AI ​​model includes at least one of the following: a deep learning model, a natural language processing model, and a traditional machine learning model.

[0155] In some embodiments, the airborne sensors in the mobile platform include at least one of the following: a light sensor, a temperature sensor, a pressure sensor, a magnetic sensor, and a gas sensor.

[0156] In some embodiments, the above-mentioned optical sensor includes at least one of the following: an infrared sensor, a laser sensor, a vision sensor, a spectral sensor, and an ultraviolet light sensor.

[0157] In some embodiments, the above-mentioned visual sensor includes at least one of the following: a wide-angle visible light camera, a zoom visible light camera, and an infrared camera.

[0158] In some embodiments, the above functions include at least one of the following: automatic mapping and automatic inspection.

[0159] In some embodiments, automatic mapping includes at least one of the following: land surveying and urban planning.

[0160] In some embodiments, automatic inspection includes at least one of the following: power grid inspection, forest inspection, river monitoring, patrol monitoring, and pipeline inspection.

[0161] In some embodiments, the one or more control terminals include at least one of the following: a developer front-end, a model training and / or quantization platform, and a ground control device for a mobile platform.

[0162] In some embodiments, the model training and / or quantization platform is used to generate a preliminary model of the AI ​​model based on calibration samples and indication information. For example, a user can input indication information to determine the model architecture and upload calibration samples (or select calibration samples from a sample library provided by the developer front-end) through an interactive interface provided by the developer front-end. The model training and / or quantization platform determines the model architecture based on the indication information to obtain the original model, and then uses the calibration samples to train the original model to obtain the preliminary model.

[0163] In some embodiments, the preliminary model can be trained on the user's local device. For example, the user can input instructions to determine the model architecture through an interactive interface provided by the developer's front end. The model training and / or quantization platform can determine the model architecture based on the instructions to obtain the original model. The user can download the source code of the original model through the developer's front end and train the original model on their own device (i.e., the user's local device) using calibration samples to obtain the preliminary model of the AI ​​model.

[0164] In some embodiments, the preliminary model can also be trained directly in the aforementioned model training and / or quantization platform. For example, users can input instruction information for determining the model architecture and upload calibration samples (or select calibration samples from the sample library provided by the developer front-end) through the interactive interface provided by the developer front-end. The model training and / or quantization platform can determine the model architecture based on the instruction information, obtain the original model, and then use the calibration samples to train the original model to obtain the preliminary model.

[0165] In some embodiments, if the preliminary model is trained on the user's local device, when quantizing the preliminary model, the model parameters of the preliminary model uploaded by the user can be obtained, the model parameters of the preliminary model can be quantized based on the calibration sample, and the quantized model parameters can be used as the model parameters of the AI ​​model.

[0166] In some embodiments, the generation of the preliminary model of the AI ​​model includes at least one of the following: selection of model architecture, collection and calibration of model training data, and training of the preliminary model.

[0167] In some embodiments, the developer front-end is used to allow users to identify calibration samples and input instruction information for determining the architecture of the AI ​​model.

[0168] In some embodiments, the developer front-end can also be used to allow users to input information related to model quantization and to display the results of model quantization. The information related to model quantization is used to quantize the preliminary model to obtain an AI model, which is generated based on the aforementioned calibration samples and indication information.

[0169] In some embodiments, the information related to model quantization includes at least one of the following: calibration samples and model parameters.

[0170] In some embodiments, a model training and / or quantization platform is used to optimize a preliminary model to obtain the AI ​​model, wherein the preliminary model is generated based on the aforementioned calibration samples and indication information.

[0171] In some embodiments, the above optimization process includes at least one of the following operations: verifying model performance, model quantization, model conversion, cryptographic signing, and training the Model Zoo resource library.

[0172] In addition to training and quantizing models, model training and / or quantization platforms can also perform other processes, such as model structure conversion, model code language conversion, model signing, encryption, and model performance verification.

[0173] For example, in some embodiments, the preliminary model can be trained on the user's local device. For instance, as shown in Figure 7(c), the user can input instructions to determine the model architecture through an interactive interface provided by the developer's front-end. The model training and / or quantization platform can determine the model architecture based on this instructions to obtain the original model. The user can download the source code of the original model through the developer's front-end and train the original model on their own device (i.e., the user's local device) using calibration samples to obtain the preliminary model of the AI ​​model. When quantizing the preliminary model, the model training and / or quantization platform can obtain the model parameters and calibration samples uploaded by the user, quantize the model parameters based on the calibration samples, and use the quantized model parameters as the model parameters of the optimized AI model to obtain the optimized AI model. Then, the model training and / or quantization platform can send the optimized AI model to the ground control equipment. After detecting the user's confirmation command, the ground control equipment will send the AI ​​model to the mobile platform.

[0174] Mobile platforms typically carry various types of sensors, which collect data during task execution to achieve specific functions. Considering different application scenarios, the sensor data can be analyzed and processed to realize different functions. To improve the accuracy and efficiency of data processing, AI models can be used. However, in related technologies, the functions of AI models deployed in mobile platforms are often fixed, making it impossible to perform different types of data processing for different application scenarios, thus limiting the application scenarios of mobile platforms.

[0175] To leverage the existing airborne sensors within a mobile platform to meet the needs of different application scenarios, improve the scalability of the mobile platform, or enhance the accuracy of the mobile platform in analyzing and processing data collected by these sensors, thereby improving the operational accuracy of the mobile platform, another embodiment of this application provides a control method for a mobile platform. This method allows for the selective association of an AI model with different types or numbers of airborne sensors within the mobile platform. This enables the AI ​​model to process and analyze the data collected by these airborne sensors to achieve different functional requirements.

[0176] As shown in Figure 8, the method may include the following steps:

[0177] S802, One or more control terminals acquire user instruction information, wherein the instruction information is related to the association between the AI ​​model and the airborne sensors of the mobile platform;

[0178] In step S802, one or more control terminals can acquire user instructions, which are used to indicate the association between the AI ​​model and the airborne sensors of the mobile platform. For example, the control terminal can provide a configuration interface through which the user can configure the association between the AI ​​model and the airborne sensors of the mobile platform.

[0179] S804. One or more control terminals, based on the indication information, associate the AI ​​model with one or more airborne sensors of the mobile platform to generate corresponding functional modules. The AI ​​model can be selectively associated with the perception information of at least one of the airborne sensors to be configured into different functional modules. The different functional modules include a first functional module and a second functional module. The airborne sensor associated with the first AI model in the first functional module is at least partially different from the airborne sensor associated with the second AI model in the second functional module.

[0180] In step S804, the control terminal can associate the AI ​​model with one or more airborne sensors of the mobile platform based on the indication information to generate corresponding functional modules. The AI ​​model can be selectively associated with at least one airborne sensor to be configured into different functional modules, and the airborne sensors associated with the AI ​​models in different functional modules are at least partially different.

[0181] For example, the AI ​​model includes a first AI model and a second AI model. The first AI model is associated with the perception information of one or more airborne sensors and configured as a first functional module. The second AI model is associated with the perception information of one or more airborne sensors and configured as a second functional module. The airborne sensors associated with the first AI model are at least partially different from those associated with the second AI model.

[0182] S806. One or more control terminals send the functional module to the mobile platform;

[0183] In step S806, after generating the aforementioned functional module, the one or more control terminals can send the functional module to the mobile platform. For example, the control terminal can send the functional module directly to the mobile platform after generating it, or it can send the functional module to the mobile platform after receiving a confirmation instruction from the user. The specific settings can be flexibly configured based on actual needs.

[0184] S808, The mobile platform acquires the functional module;

[0185] In step S808, the mobile platform can obtain the functional module from the control terminal. For example, the mobile platform can obtain the functional module from the control terminal and store it.

[0186] S810. The mobile platform loads the functional module and executes the corresponding functions of the functional module.

[0187] In step S810, when the mobile platform needs to execute the function corresponding to the function module during the execution of the task, it can load the function module to execute the corresponding function.

[0188] In some embodiments, the number and / or type of airborne sensors associated with the AI ​​model in the first functional module are at least partially different from the number and / or type of airborne sensors associated with the AI ​​model in the second functional module.

[0189] In some embodiments, the perception information from the airborne sensors is used as input information for the AI ​​model.

[0190] In some embodiments, the number of times the AI ​​model is used in the first functional module is different from the number of times the AI ​​model is used in the second functional module.

[0191] In some embodiments, the number of times the AI ​​model is used and / or the number of times it is used in the first functional module is less than the number of times the AI ​​model is used and / or the number of times it is used in the second functional module.

[0192] In some embodiments, the number and / or type of airborne sensors associated with the AI ​​model in the first functional module is less than the number and / or type of airborne sensors associated with the AI ​​model in the second functional module.

[0193] In some embodiments, the type of the airborne sensor associated with at least one AI model in the second functional module is different from the type of airborne sensor associated with at least one AI model in the first functional module.

[0194] In some embodiments, the AI ​​model or a sub-model of the AI ​​model includes at least one of the following: a deep learning model, a natural language processing model, and a traditional machine learning model.

[0195] In some embodiments, the airborne sensor includes at least one of the following: a light sensor, a temperature sensor, a pressure sensor, a magnetic sensor, and a gas sensor.

[0196] In some embodiments, the optical sensor includes at least one of the following: an infrared sensor, a laser sensor, a vision sensor, a spectral sensor, and an ultraviolet light sensor.

[0197] In some embodiments, the visual sensor includes at least one of the following: a wide-angle visible light camera, a zoom visible light camera, and an infrared camera.

[0198] In some embodiments, the function includes at least one of the following: automatic mapping and automatic inspection.

[0199] In some embodiments, the automatic mapping includes at least one of the following: land surveying and urban planning.

[0200] In some embodiments, the automatic inspection includes at least one of the following: power grid inspection, forest inspection, river monitoring, patrol monitoring, and pipeline inspection.

[0201] In some embodiments, the functional module performs one or more sub-functions.

[0202] In some embodiments, the sub-functions include at least one of the following: obstacle perception function, obstacle avoidance function, target recognition function, target tracking function, and target classification function.

[0203] In some embodiments, the obstacle perception function includes at least one of the following: obstacle recognition function, obstacle location perception function.

[0204] In some embodiments, the obstacle avoidance function includes at least one of the following: obstacle avoidance path planning function, obstacle warning function.

[0205] In some embodiments, the target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

[0206] In some embodiments, the target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

[0207] In some embodiments, the target tracking function includes at least one of the following: automatic target following function, and automatic target preset distance maintenance function.

[0208] In some embodiments, the target classification function includes at least one of the following: a target identification function for a specific type, and a target filtering function for a specific type.

[0209] The specific implementation details of the control method for the mobile platform can be found in the descriptions in the above embodiments, and will not be repeated here.

[0210] Considering the different needs of different application scenarios, the functions that mobile platforms need to achieve also vary. However, in related technologies, the functional modules on mobile platforms are often fixed, meaning that the functions these modules can achieve are fixed and cannot meet the needs of different application scenarios.

[0211] To enable mobile platforms to meet the needs of different application scenarios and improve their scalability, this application provides a control method for mobile platforms. For certain security-related functional modules within the mobile platform, configuration permissions are not publicly available, meaning users are not allowed to modify, update, or customize these modules. However, for security-related functional modules, configuration permissions can be publicly granted, allowing users to define the AI ​​models within these modules. This allows the mobile platform to meet the needs of different application scenarios or improve the accuracy of operations within the same work environment. Simultaneously, it ensures that the security of the mobile platform remains unaffected.

[0212] As shown in Figure 9, the control method for this mobile platform may include the following steps:

[0213] S902, One or more control terminals acquire user instruction information;

[0214] In step S902, one or more control terminals may acquire user instruction information, which can be used to determine the AI ​​model to be deployed to the mobile platform. For example, the instruction information may include instruction information for determining the architecture of the AI ​​model, or the instruction information may be used to determine calibration samples for training and / or optimizing the AI ​​model.

[0215] S904. One or more control terminals determine an AI model based on the instruction information; wherein, the execution of functions related to the AI ​​model in the mobile platform includes the operation of multiple functional modules, the multiple functional modules including a first functional module and a second functional module, the first functional module loading the AI ​​model, the second functional module being an airborne functional module fixed to the mobile platform, the second functional module including the path planning module of the mobile platform;

[0216] In step S904, the control terminal can determine the AI ​​model based on the indication information. The execution of functions related to the AI ​​model in the mobile platform includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module needs to load the AI ​​model during operation. The second functional module is an airborne functional module fixed in the mobile platform, including the path planning module of the mobile platform.

[0217] For functional modules on a mobile platform that are less related to security, users can define, update, or modify the AI ​​models within those modules. However, for functional modules on a mobile platform that are more closely related to security, they can be set as fixed onboard functional modules, meaning users cannot modify them. For example, considering the path planning module used to plan the movement path of the mobile platform, since it is closely related to the platform's security, this functional module can be set as a fixed onboard functional module.

[0218] S906. The control terminal sends the AI ​​model to the mobile platform;

[0219] In step S906, the control terminal can send the AI ​​model to the mobile platform. For example, the control terminal can send the AI ​​model directly to the mobile platform after generating it, or it can send the AI ​​model to the mobile platform after receiving a confirmation instruction from the user. The specific settings can be flexibly configured based on actual needs.

[0220] S908, The mobile platform acquires the AI ​​model;

[0221] In step S908, the mobile platform can acquire the AI ​​model. For example, after acquiring the AI ​​model from the control terminal, the mobile platform can store the AI ​​model.

[0222] S910, the mobile platform executes functions related to the AI ​​model.

[0223] In step S910, during the execution of a task on the mobile platform, the AI ​​model can be invoked to perform functions related to the AI ​​model.

[0224] In some embodiments, the mobile platform acquiring the AI ​​model may be the mobile platform acquiring a first functional module containing the AI ​​model.

[0225] In some embodiments, the second functional module further includes at least one of the following: a pose control module for a movable platform and a movement trajectory control module for a movable platform.

[0226] In some embodiments, the airborne functional module is associated with the safety control of the mobile platform.

[0227] In some embodiments, the route planning module is used to perform at least one of the following functions: automatic route generation, automatic route update, and automatic route suggestion.

[0228] In some embodiments, the path execution function includes at least one of the following: automatic path execution function and path switching prompt function.

[0229] In some embodiments, each of the first functional modules performs one or more sub-functions.

[0230] In some embodiments, the sub-functions include at least one of the following: obstacle perception function, obstacle avoidance function, target recognition function, target tracking function, and target classification function.

[0231] In some embodiments, the obstacle perception function includes at least one of the following: obstacle recognition function, obstacle location perception function.

[0232] In some embodiments, the obstacle avoidance function includes at least one of the following: obstacle avoidance path planning function, obstacle warning function.

[0233] In some embodiments, the target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

[0234] In some embodiments, the target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

[0235] In some embodiments, the target tracking function includes at least one of the following: automatic target following function, and automatic target preset distance maintenance function.

[0236] In some embodiments, the target classification function includes at least one of the following: a target identification function for a specific type, and a target filtering function for a specific type.

[0237] In some embodiments, the first functional module is associated with the load operation mode of the mobile platform.

[0238] In some embodiments, the function includes a first function and a second function different from the first function, and the AI ​​model includes a first AI model and a second AI model, wherein the first AI model is loaded during the operation of the first function module that implements the first function, and the second AI model is loaded during the operation of the first function module that implements the second function.

[0239] In some embodiments, the execution of the first function and the execution of the second function include the operation of the same functional module.

[0240] In some embodiments, the same functional module is an airborne functional module embedded in the mobile platform.

[0241] The specific details of the control method for the mobile platform can be found in the description of the above embodiments, and will not be repeated here.

[0242] Furthermore, embodiments of this application also provide a control system for a mobile platform, including:

[0243] One or more control terminals are configured to acquire (a) calibration samples determined by a user and (b) user instruction information, and generate an optimized AI model based on the calibration samples and (b) the instruction information, wherein the instruction information is used to determine the model architecture of the AI ​​model, and the optimized AI model can be recognized by the mobile platform; and

[0244] A mobile platform, communicatively connected to at least one of the control terminals.

[0245] The mobile platform is used to load the AI ​​model and execute functions related to the AI ​​model.

[0246] In some embodiments, the one or more control terminals include at least one of the following: a developer front-end, a model training and / or quantization platform, and a ground control device for a mobile platform.

[0247] In some embodiments, the model training and / or quantization platform is used to generate a preliminary model of the AI ​​model based on the calibration samples and the indication information.

[0248] In some embodiments, the generation of the preliminary model of the AI ​​model includes at least one of the following: selection of model architecture, data collection and calibration, and training of the preliminary model.

[0249] In some embodiments, the developer front-end is used to allow users to determine the calibration sample and input the indication information; and / or

[0250] The developer front-end is used for users to input information related to model quantization and to display the results of model quantization. The information related to model quantization is used to quantize the preliminary model to obtain the AI ​​model. The preliminary model is generated based on the calibration samples and the indication information.

[0251] In some embodiments, the information related to model quantization includes at least one of the following: calibration samples and model parameters.

[0252] In some embodiments, the model training and / or quantization platform is used to optimize a preliminary model to obtain the AI ​​model, wherein the preliminary model is generated based on the calibration samples and the indication information.

[0253] In some embodiments, the optimization process includes at least one of the following operations: verifying model performance, model quantization, model conversion, cryptographic signing, and training the Model Zoo resource library.

[0254] In some embodiments, the ground control device of the mobile platform is used to control the mobile platform and upload the AI ​​model to the mobile platform.

[0255] In some embodiments, the ground control terminal includes at least one of the following: a remote controller and a ground base station.

[0256] The specific details of the interaction between one or more control terminals and the mobile platform can be found in the descriptions in the above embodiments, and will not be repeated here.

[0257] Furthermore, embodiments of this application also provide a control system for a mobile platform, including:

[0258] One or more control terminals are used to acquire user instruction information, wherein the instruction information is related to the association between the AI ​​model and the airborne sensors of the mobile platform;

[0259] Based on the indicated information, the AI ​​model is associated with one or more airborne sensors of the mobile platform to generate corresponding functional modules. The AI ​​model can be selectively associated with at least one of the airborne sensors to be configured into different functional modules, including a first functional module and a second functional module. The airborne sensor associated with the AI ​​model in the first functional module is at least partially different from the airborne sensor associated with the AI ​​model in the second functional module.

[0260] A mobile platform, communicatively connected to at least one of the control terminals.

[0261] The mobile platform is used to acquire the functional module, load the functional module, and execute the corresponding functions of the functional module.

[0262] The specific details of the interaction between one or more control terminals and the mobile platform can be found in the descriptions in the above embodiments, and will not be repeated here.

[0263] Furthermore, embodiments of this application also provide a control system for a mobile platform, including:

[0264] One or more control terminals are used to acquire user instruction information; and to determine an AI model based on the instruction information.

[0265] A mobile platform, communicatively connected to at least one of the control terminals.

[0266] The mobile platform acquires the AI ​​model and performs functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model, and the second functional module is an onboard functional module fixed to the mobile platform. The second functional module includes the path planning module of the mobile platform.

[0267] The specific details of the interaction between one or more control terminals and the mobile platform can be found in the descriptions in the above embodiments, and will not be repeated here.

[0268] Furthermore, this application embodiment also provides a mobile platform, as shown in FIG10, the mobile platform including:

[0269] At least one processor 101; and

[0270] At least one memory 102 including computer program code, wherein the at least one memory 102 and the computer program code, together with at least one processor 101, are configured to enable the portable platform to perform at least the following operations:

[0271] Obtain an optimized AI model, wherein the AI ​​model is generated based on (a) calibration samples determined by the user and (b) user instruction information, wherein the instruction information is used to determine the architecture of the AI ​​model; and load the AI ​​model and execute the corresponding functions of the AI ​​model;

[0272] and / or

[0273] The system acquires a functional module corresponding to user instruction information, wherein the functional module is generated by associating the AI ​​model with one or more airborne sensors of the mobile platform based on the instruction information, wherein the AI ​​model can be selectively associated with at least one of the airborne sensors to configure different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the first AI model in the first functional module is at least partially different from the airborne sensor associated with the second AI model in the second functional module; and loads the functional module and executes the corresponding function of the functional module.

[0274] and / or

[0275] The system acquires an AI model and executes functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model. The second functional module is an airborne functional module embedded in the mobile platform and includes a path planning module of the mobile platform.

[0276] It is easy to understand that the solutions described in the above embodiments can be combined when there is no conflict, and not all of them are listed in the embodiments of this application.

[0277] Accordingly, this application also provides a computer storage medium storing a program that, when executed by a processor, implements the method in any of the above embodiments.

[0278] The embodiments of this application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0279] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0280] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0281] The methods and apparatus provided in the embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A control method for a mobile platform, characterized in that, include: Obtain (a) the calibration sample determined by the user and (b) the user's instruction information; Based on the calibration samples and the indication information (b), an optimized AI model is generated, wherein the indication information is used to determine the architecture of the AI ​​model, and the AI ​​model can be recognized by the mobile platform; and The mobile platform is controlled to load the AI ​​model and execute the corresponding functions of the AI ​​model.

2. The method according to claim 1, characterized in that, Before controlling the mobile platform to load the AI ​​model, the following is also included: Incremental sample training is performed on the AI ​​model.

3. The method according to claim 1, characterized in that, The step of generating an optimized AI model based on the calibrated samples and the indication information includes: A preliminary model is generated based on the calibration samples and the indication information. The preliminary model is then transformed to obtain a model that the mobile platform can recognize, which serves as the optimized AI model.

4. The method according to claim 3, characterized in that, The preliminary model is transformed, including: The preliminary model is then quantified.

5. The method according to claim 4, characterized in that, After quantizing the preliminary model, the method further includes: The preliminary model after quantization is evaluated and optimized based on the calibration samples to obtain the AI ​​model.

6. The method according to claim 4, characterized in that, The initial model was trained on the user's local device.

7. The method according to claim 6, characterized in that, The preliminary model is quantized, including: Obtain the model parameters of the preliminary model uploaded by the user, and quantize the model parameters of the preliminary model based on the calibration samples.

8. The method according to claim 1, characterized in that, The AI ​​model is a PyTorch model.

9. The method according to claim 1, characterized in that, The indication information includes at least one of the following: indication information related to the selection of the type of the AI ​​model, indication information related to the selection of the type of the input information of the AI ​​model, indication information related to the selection of the type of the output information of the AI ​​model, and indication information related to the selection of the type, quantity, and / or frequency of the sub-models of the AI ​​model.

10. The method according to claim 9, characterized in that, The type of input information selected for the AI ​​model is related to the type of airborne sensors on the mobile platform.

11. The method according to claim 9, characterized in that, The type of output information of the AI ​​model is related to the type of onboard payload of the mobile platform.

12. The method according to claim 1, characterized in that, The calibration sample is determined by the user in at least one of the following ways: The calibration sample is selected by the user; The calibration sample is calibrated by the user.

13. The method according to claim 12, characterized in that, The calibration sample is related to the result of the function execution.

14. The method according to claim 13, characterized in that, The calibration sample is related to the efficiency of the result of the function execution.

15. The method according to claim 13, characterized in that, The calibration sample is related to the quality of the result of the function execution.

16. The method according to claim 1, characterized in that, The calibration sample determined by the user includes: the data type of the calibration sample selected by the user and / or the type of the target object in the calibration sample.

17. The method according to claim 16, characterized in that, The data type of the calibration sample is related to the type of airborne sensor of the mobile platform.

18. The method according to claim 17, characterized in that, The data types of the calibration samples include at least one of the following: visible light images, infrared images, ultrasonic data, and point cloud data.

19. The method according to claim 1, characterized in that, The target objects in the calibration sample include at least one of the following: the target object to be identified in the calibration sample, and the target object to be tracked in the calibration sample.

20. The method according to claim 1, characterized in that, The execution of the function includes running one or more functional modules, wherein the AI ​​model is loaded during the operation of at least one of the functional modules.

21. The method according to claim 20, characterized in that, Each of the aforementioned functional modules executes one or more sub-functions.

22. The method according to claim 21, characterized in that, The sub-functions include at least one of the following: path planning function, path execution function, obstacle perception function, obstacle avoidance function, target recognition function, target tracking function, and target classification function.

23. The method according to claim 22, characterized in that, The route planning function includes at least one of the following: automatic route generation, automatic route update, and automatic route suggestion.

24. The method according to claim 22, characterized in that, The path execution function includes at least one of the following: automatic path execution function and path switching prompt function.

25. The method according to claim 22, characterized in that, The obstacle perception function includes at least one of the following: obstacle recognition function, obstacle location perception function.

26. The method according to claim 25, characterized in that, The obstacle avoidance function includes at least one of the following: obstacle avoidance path planning function, obstacle warning function.

27. The method according to claim 22, characterized in that, The target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

28. The method according to claim 27, characterized in that, The target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

29. The method according to claim 22, characterized in that, The target tracking function includes at least one of the following: automatic target following function, and automatic target preset distance maintenance function.

30. The method according to claim 22, characterized in that, The target classification function includes at least one of the following: a target identification function for a specific type, and a target filtering function for a specific type.

31. The method according to claim 1, characterized in that, The execution of the function includes running multiple functional modules, wherein the AI ​​model is loaded during the operation of some of the functional modules.

32. The method according to claim 31, characterized in that, The other part of the functional modules are the airborne functional modules fixed to the mobile platform.

33. The method according to claim 32, characterized in that, The airborne functional module includes at least one of the following: a pose control module for a mobile platform, a movement trajectory control module for a mobile platform, and a path planning module for a mobile platform.

34. The method according to claim 32, characterized in that, The airborne functional module is related to the safety control of the mobile platform.

35. The method according to claim 31, characterized in that, The functional module that loads the AI ​​model is related to the load operation mode of the mobile platform.

36. The method according to claim 1, characterized in that, The functions include a first function and a second function different from the first function. The AI ​​model includes a first AI model and a second AI model. The first AI model is loaded during the operation of the functional module that implements the first function, and the second AI model is loaded during the operation of the functional module that implements the second function.

37. The method according to claim 36, characterized in that, The execution of the first function and the execution of the second function involve the operation of the same functional module.

38. The method according to claim 37, characterized in that, The same functional module is the airborne functional module fixed to the mobile platform.

39. The method according to claim 36, characterized in that, The calibration samples include a first calibration sample and a second calibration sample. The first calibration sample is different from the second calibration sample. The first calibration sample is used to train and / or optimize the first AI model, and the second calibration sample is used to train and / or optimize the second AI model.

40. The method according to claim 1, characterized in that, The mobile platform includes multiple airborne sensors, and the AI ​​model can be selectively associated with the sensing information of at least one of the airborne sensors to be configured into different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the AI ​​model in the first functional module is at least partially different from the airborne sensor associated with the AI ​​model in the second functional module.

41. The method according to claim 40, characterized in that, The number and / or type of airborne sensors associated with the AI ​​model in the first functional module are at least partially different from the number and / or type of airborne sensors associated with the AI ​​model in the second functional module.

42. The method according to claim 40, characterized in that, The sensing information from the airborne sensors serves as the input information for the AI ​​model.

43. The method according to claim 40, characterized in that, The number of times the AI ​​model is used in the first functional module is different from the number of times the AI ​​model is used in the second functional module.

44. The method according to claim 43, characterized in that, The number of times the AI ​​model is used in the first functional module is less than the number of times the AI ​​model is used in the second functional module.

45. The method according to claim 41, characterized in that, The number and / or type of airborne sensors associated with the AI ​​model in the first functional module is less than the number and / or type of airborne sensors associated with the AI ​​model in the second functional module.

46. ​​The method according to claim 40, characterized in that, The type of airborne sensor associated with at least one of the AI ​​models in the second functional module is different from the type of airborne sensor associated with at least one of the AI ​​models in the first functional module.

47. The method according to claim 1, characterized in that, The AI ​​model or its sub-models include at least one of the following: a deep learning model, a natural language processing model, or a traditional machine learning model.

48. The method according to claim 40, characterized in that, The airborne sensors include at least one of the following: a light sensor, a temperature sensor, a pressure sensor, a magnetic sensor, and a gas sensor.

49. The method according to claim 48, characterized in that, The optical sensor includes at least one of the following: an infrared sensor, a laser sensor, a vision sensor, a spectral sensor, and an ultraviolet light sensor.

50. The method according to claim 49, characterized in that, The visual sensor includes at least one of the following: a wide-angle visible light camera, a zoom visible light camera, and an infrared camera.

51. The method according to claim 1, characterized in that, The functions include at least one of the following: automatic mapping and automatic inspection.

52. The method according to claim 51, characterized in that, The automatic surveying includes at least one of the following: land surveying and urban planning.

53. The method according to claim 51, characterized in that, The automatic inspection includes at least one of the following: power grid inspection, forest inspection, river monitoring, patrol monitoring, and pipeline inspection.

54. A control method for a mobile platform, characterized in that, include: Obtain an optimized AI model, wherein the AI ​​model is generated based on (a) user-defined calibration samples and (b) user-instructed information, wherein the instruction information is used to determine the architecture of the AI ​​model; and Load the AI ​​model and execute the corresponding functions of the AI ​​model.

55. A control method for a mobile platform, characterized in that, include: Obtain user instruction information, wherein the instruction information is related to the correlation between the AI ​​model and the airborne sensors of the mobile platform; Based on the indicated information, the AI ​​model is associated with one or more airborne sensors of the mobile platform to generate corresponding functional modules, wherein the AI ​​model can be selectively associated with at least one of the airborne sensors to be configured into different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the AI ​​model in the first functional module is at least partially different from the airborne sensor associated with the AI ​​model in the second functional module; and; The mobile platform is controlled to load the functional module and execute the corresponding functions of the functional module.

56. The method according to claim 55, characterized in that, The number and / or type of airborne sensors associated with the AI ​​model in the first functional module are at least partially different from the number and / or type of airborne sensors associated with the AI ​​model in the second functional module.

57. The method according to claim 56, characterized in that, The sensing information from the airborne sensors serves as the input information for the AI ​​model.

58. The method according to claim 56, characterized in that, The number of times the AI ​​model is used in the first functional module is different from the number of times the AI ​​model is used in the second functional module.

59. The method according to claim 58, characterized in that, The number of times the AI ​​model is used in the first functional module is less than the number of times the AI ​​model is used in the second functional module.

60. The method according to claim 56, characterized in that, The number and / or type of airborne sensors associated with the AI ​​model in the first functional module is less than the number and / or type of airborne sensors associated with the AI ​​model in the second functional module.

61. The method according to claim 55, characterized in that, The type of airborne sensor associated with at least one of the AI ​​models in the second functional module is different from the type of airborne sensor associated with at least one of the AI ​​models in the first functional module.

62. The method according to claim 55, characterized in that, The AI ​​model or its sub-models include at least one of the following: a deep learning model, a natural language processing model, or a traditional machine learning model.

63. The method according to claim 62, characterized in that, The airborne sensors include at least one of the following: a light sensor, a temperature sensor, a pressure sensor, a magnetic sensor, and a gas sensor.

64. The method according to claim 63, characterized in that, The optical sensor includes at least one of the following: an infrared sensor, a laser sensor, a vision sensor, a spectral sensor, and an ultraviolet light sensor.

65. The method according to claim 64, characterized in that, The visual sensor includes at least one of the following: a wide-angle visible light camera, a zoom visible light camera, and an infrared camera.

66. The method according to claim 55, characterized in that, The functions include at least one of the following: automatic mapping and automatic inspection.

67. The method according to claim 66, characterized in that, The automatic surveying includes at least one of the following: land surveying and urban planning.

68. The method according to claim 66, characterized in that, The automatic inspection includes at least one of the following: power grid inspection, forest inspection, river monitoring, patrol monitoring, and pipeline inspection.

69. The method according to claim 55, characterized in that, The functional module executes one or more sub-functions.

70. The method according to claim 69, characterized in that, The sub-functions include at least one of the following: obstacle perception function, obstacle avoidance function, target recognition function, target tracking function, and target classification function.

71. The method according to claim 70, characterized in that, The obstacle perception function includes at least one of the following: obstacle recognition function, obstacle location perception function.

72. The method according to claim 71, characterized in that, The obstacle avoidance function includes at least one of the following: obstacle avoidance path planning function, obstacle warning function.

73. The method according to claim 70, characterized in that, The target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

74. The method according to claim 73, characterized in that, The target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

75. The method according to claim 70, characterized in that, The target tracking function includes at least one of the following: automatic target following function, and automatic target preset distance maintenance function.

76. The method according to claim 70, characterized in that, The target classification function includes at least one of the following: a target identification function for a specific type, and a target filtering function for a specific type.

77. A control method for a mobile platform, characterized in that, include: A functional module is acquired to obtain user instruction information, wherein the functional module is generated by associating an AI model with one or more airborne sensors of the mobile platform based on the instruction information, wherein the AI ​​model can be selectively associated with at least one of the airborne sensors to configure different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the first AI model in the first functional module is at least partially different from the airborne sensor associated with the second AI model in the second functional module; and Load the functional module and execute the corresponding functions of the functional module.

78. A control method for a mobile platform, characterized in that, include: Obtain user instructions; Based on the indicated information, determine the AI ​​model; as well as The mobile platform is controlled to execute functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model, and the second functional module is an onboard functional module fixed to the mobile platform. The second functional module includes the path planning module of the mobile platform.

79. The method according to claim 78, characterized in that, The second functional module further includes at least one of the following: a pose control module for the movable platform and a movement trajectory control module for the movable platform.

80. The method according to claim 78, characterized in that, The airborne functional module is related to the safety control of the mobile platform.

81. The method according to claim 78, characterized in that, The route planning module is used to perform at least one of the following functions: automatic route generation, automatic route update, and automatic route suggestion.

82. The method according to claim 81, characterized in that, The path execution function includes at least one of the following: automatic path execution function and path switching prompt function.

83. The method according to claim 81, characterized in that, Each of the first functional modules executes one or more sub-functions.

84. The method according to claim 83, characterized in that, The sub-functions include at least one of the following: obstacle perception function, obstacle avoidance function, target recognition function, target tracking function, and target classification function.

85. The method according to claim 84, characterized in that, The obstacle perception function includes at least one of the following: obstacle recognition function, obstacle location perception function.

86. The method according to claim 84, characterized in that, The obstacle avoidance function includes at least one of the following: obstacle avoidance path planning function, obstacle warning function.

87. The method according to claim 84, characterized in that, The target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

88. The method according to claim 87, characterized in that, The target recognition function includes at least one of the following: automatic target category recognition function and automatic target shape recognition function.

89. The method according to claim 84, characterized in that, The target tracking function includes at least one of the following: automatic target following function, and automatic target preset distance maintenance function.

90. The method according to claim 84, characterized in that, The target classification function includes at least one of the following: a target identification function for a specific type, and a target filtering function for a specific type.

91. The method according to claim 78, characterized in that, The first functional module is related to the load operation mode of the mobile platform.

92. The method according to claim 1, characterized in that, The functions include a first function and a second function different from the first function. The AI ​​model includes a first AI model and a second AI model. The first function module that implements the first function loads the first AI model during operation, and the first function module that implements the second function loads the second AI model during operation.

93. The method according to claim 92, characterized in that, The execution of the first function and the execution of the second function involve the operation of the same functional module.

94. The method according to claim 93, characterized in that, The same functional module is the airborne functional module fixed to the mobile platform.

95. A control method for a mobile platform, characterized in that, include: Acquiring AI models; and The AI ​​model-related functions are executed, wherein the execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model; the second functional module is an airborne functional module fixed to the mobile platform, and the second functional module includes the path planning module of the mobile platform.

96. The method according to claim 95, characterized in that, The acquisition of the AI ​​model includes: Obtain the first functional module containing the AI ​​model.

97. A mobile platform, characterized in that, include: At least one processor; as well as At least one memory including computer program code, wherein the at least one memory and the computer program code, together with at least one processor, are configured to enable the mobile platform to perform at least the following operations: Obtain an optimized AI model, wherein the AI ​​model is generated based on (a) calibration samples determined by the user and (b) user instruction information, wherein the instruction information is used to determine the architecture of the AI ​​model; and load the AI ​​model and execute the corresponding functions of the AI ​​model; and / or The system acquires a functional module corresponding to user instruction information, wherein the functional module is generated by associating the AI ​​model with one or more airborne sensors of the mobile platform based on the instruction information, wherein the AI ​​model can be selectively associated with at least one of the airborne sensors to configure different functional modules, the different functional modules including a first functional module and a second functional module; the airborne sensor associated with the first AI model in the first functional module is at least partially different from the airborne sensor associated with the second AI model in the second functional module; and loads the functional module and executes the corresponding function of the functional module. and / or The system acquires an AI model and executes functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model. The second functional module is an airborne functional module embedded in the mobile platform and includes a path planning module of the mobile platform.

98. A control system for a mobile platform, characterized in that, include: One or more control terminals are configured to acquire (a) calibration samples determined by a user and (b) user instruction information, and generate an optimized AI model based on the calibration samples and (b) the instruction information, wherein the instruction information is used to determine the model architecture of the AI ​​model, and the optimized AI model can be recognized by the mobile platform; and A mobile platform, communicatively connected to at least one of the control terminals. The mobile platform is used to load the AI ​​model and execute functions related to the AI ​​model.

99. The control system according to claim 98, characterized in that, The one or more control terminals include at least one of the following: a developer front-end, a model training and / or quantization platform, or a ground control device for a mobile platform.

100. The control system according to claim 99, characterized in that, The model training and / or quantization platform is used to generate a preliminary model of the AI ​​model based on the calibration samples and the indication information.

101. The control system according to claim 100, characterized in that, The generation of the preliminary model of the AI ​​model includes at least one of the following: selection of model architecture, collection and calibration of model training data, and training of the preliminary model.

102. The control system according to claim 99, characterized in that, The developer front-end is used by the user to determine the calibration sample and input the instruction information; and / or The developer front-end is used for users to input information related to model quantization and to display the results of model quantization. The information related to model quantization is used to quantize the preliminary model to obtain the AI ​​model. The preliminary model is generated based on the calibration samples and the indication information.

103. The control system according to claim 102, characterized in that, The information related to model quantization includes at least one of the following: calibration samples and model parameters.

104. The control system according to claim 99, characterized in that, The model training and / or quantization platform is used to optimize the preliminary model to obtain the AI ​​model, wherein the preliminary model is generated based on the calibration samples and the indication information.

105. The control system according to claim 104, characterized in that, The optimization process includes at least one of the following operations: verifying model performance, model quantization, model conversion, encrypted signature, and ModelZoo resource library training.

106. The control system according to claim 99, characterized in that, The ground control equipment of the mobile platform is used to control the mobile platform and upload the AI ​​model to the mobile platform.

107. The control system according to claim 106, characterized in that, The ground control terminal includes at least one of the following: a remote controller and a ground base station.

108. A control system for a mobile platform, characterized in that, include: One or more control terminals are used to acquire user instruction information, wherein the instruction information is related to the association between the AI ​​model and the airborne sensors of the mobile platform; Based on the indicated information, the AI ​​model is associated with one or more airborne sensors of the mobile platform to generate corresponding functional modules. The AI ​​model can be selectively associated with at least one of the airborne sensors to be configured into different functional modules, including a first functional module and a second functional module. The airborne sensor associated with the AI ​​model in the first functional module is at least partially different from the airborne sensor associated with the AI ​​model in the second functional module. A mobile platform, communicatively connected to at least one of the control terminals. The mobile platform is used to acquire the functional module, load the functional module, and execute the corresponding functions of the functional module.

109. A control system for a mobile platform, characterized in that, include: One or more control terminals are used to obtain user instructions. Based on the indicated information, determine the AI ​​model; A mobile platform, communicatively connected to at least one of the control terminals. The mobile platform acquires the AI ​​model and performs functions related to the AI ​​model. The execution of the functions includes the operation of multiple functional modules, including a first functional module and a second functional module. The first functional module loads the AI ​​model, and the second functional module is an onboard functional module fixed to the mobile platform. The second functional module includes the path planning module of the mobile platform.

110. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1-96.

Images