Unmanned system efficient intelligent obstacle avoidance method based on heterogeneous brain model

Abstract

The invention discloses an unmanned system efficient intelligent obstacle avoidance method based on a heterogeneous brain model, a heterogeneous brain model training method and device, a storage medium, equipment and a computer program product. The method comprises the steps that input feature data used for inputting the heterogeneous brain model is generated according to driving sampling data collected from a sensor of an unmanned system; inputting the input feature data into a heterogeneous brain model so as to initialize each neural unit in the heterogeneous brain model; operating the heterogeneous brain model, and determining the target output state data of the target neural unit in the operated heterogeneous brain model as the model output data of the heterogeneous brain model; and controlling the navigation state of the unmanned system according to the model output data. According to the method, the operation process of the data is interpretable by utilizing the corresponding relationship among the neural units in the heterogeneous brain model, so that visual decision process interpretation is provided while real-time obstacle avoidance on low-power-consumption small embedded equipment is realized, and the safety and reliability of an unmanned system are improved.

Description

Technical Field

[0001] The present application belongs to the field of unmanned driving, and specifically relates to an efficient and intelligent obstacle avoidance method for unmanned systems based on a heterogeneous brain model, and a training method, device, storage medium, equipment and computer program product of the heterogeneous brain model. Background Art

[0002] The rapid development of unmanned driving technology has led to the transformation of intelligent transportation systems, and unmanned systems are increasingly used in various environments. In scenarios such as intelligent transportation, automated production, and drone cruising, the efficient obstacle avoidance capability of unmanned systems is crucial to improving their autonomy and safety. Therefore, how to achieve efficient and intelligent obstacle avoidance in complex and changing environments has become an important research topic in the field of unmanned systems.

[0003] At present, the obstacle avoidance methods of unmanned systems include those based on deep convolutional neural networks, which use the powerful feature extraction capabilities of deep convolutional neural networks to automatically learn obstacle features from perceived images and generate obstacle avoidance decisions.

[0004] However, methods based on deep convolutional neural networks require a large amount of data and computing resources during training and inference, making it difficult to achieve real-time obstacle avoidance on small embedded devices. At the same time, due to the "black box" properties of the deep convolutional neural network model, the decision-making process is difficult to explain intuitively and has poor interpretability, which will affect the safety and reliability of unmanned systems. Summary of the invention

[0005] The present application aims to provide an efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models, as well as a training method, device, storage medium, equipment and computer program product for heterogeneous brain models, which at least solves the problem of insufficient safety and reliability in the obstacle avoidance process of unmanned systems.

[0006] In a first aspect, the present application discloses an efficient and intelligent obstacle avoidance method for an unmanned system based on a heterogeneous brain model, which is applied to a controller of the unmanned system, including: Acquire driving sampling data collected from sensors of an unmanned system, and generate input feature data for inputting a heterogeneous brain model according to the driving sampling data; the heterogeneous brain model comprises a plurality of neural units; each input neural unit in the heterogeneous brain model has a corresponding output neural unit; each input neural unit is used to acquire a state value of an output neural unit corresponding to the input neural unit, and solve the state value of the input neural unit according to the acquired state value; Inputting the input feature data into the heterogeneous brain model to initialize each neural unit in the heterogeneous brain model; Running the heterogeneous brain model, and determining target output state data of a target neural unit in the running heterogeneous brain model as model output data of the heterogeneous brain model; The navigation state of the unmanned system is controlled according to the model output data.

[0007] In a second aspect, the embodiment of the present application further discloses a method for training a heterogeneous brain model for efficient and intelligent obstacle avoidance, which is used to train the heterogeneous brain model in the efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models as described in the first aspect, comprising: Dividing the heterogeneous brain model into a plurality of data processing areas so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and establishing a connection relationship between the neural units of each two data processing areas according to an initialized connection strength; the connection strength is used to characterize the number of connections and connection weights between the plurality of neural units contained in the two data processing areas; The connection strength is trained to obtain a trained heterogeneous brain model.

[0008] In a third aspect, the present application also discloses an efficient and intelligent obstacle avoidance device for an unmanned system based on a heterogeneous brain model, which is applied to a controller of an unmanned system, including: A collection module, used to obtain driving sample data collected from sensors of an unmanned system, and generate input feature data for inputting a heterogeneous brain model according to the driving sample data; the heterogeneous brain model includes a plurality of neural units; each input neural unit in the heterogeneous brain model has a corresponding output neural unit; each input neural unit is used to obtain a state value of an output neural unit corresponding to the input neural unit, and solve the state value of the input neural unit according to the obtained state value; An input module, used for inputting the input feature data into the heterogeneous brain model to initialize each neural unit in the heterogeneous brain model; an output module, used for running the heterogeneous brain model, and determining target output state data of target neural units in the running heterogeneous brain model as model output data of the heterogeneous brain model; A control module is used to control the navigation state of the unmanned system according to the model output data.

[0009] In a fourth aspect, the embodiment of the present application further discloses a training device for a heterogeneous brain model for efficient and intelligent obstacle avoidance, which is used to train the heterogeneous brain model in the efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models as described in the first aspect, comprising: An initialization module, used to divide the heterogeneous brain model into a plurality of data processing areas, so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and establish a connection relationship between the neural units of every two data processing areas according to the initialized connection strength; the connection strength is used to characterize the number of connections and connection weights between the plurality of neural units contained in the two data processing areas; The training module is used to train the connection strength to obtain a trained heterogeneous brain model.

[0010] In a fifth aspect, an embodiment of the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps described in the first aspect or the second aspect are implemented.

[0011] In a sixth aspect, an embodiment of the present application further discloses an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps described in the first aspect or the second aspect.

[0012] In a seventh aspect, an embodiment of the present application further discloses a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the steps described in the first aspect or the second aspect are implemented.

[0013] In summary, in the embodiment of the present application, by using the input feature data generated by acquiring driving sampling data and generating input feature data as the initial input of the heterogeneous brain model, the model can make full use of the environmental perception data for efficient and intelligent obstacle avoidance; and then by inputting the input feature data into each neural unit in the heterogeneous brain model, the model is initialized, so that each neural unit of the model is in a ready state, so that the initialized neural unit can accurately reflect the characteristics of the input data; then by running the heterogeneous brain model, the corresponding relationship between the neural units in the heterogeneous brain model is used to make the data operation process interpretable, while achieving real-time obstacle avoidance on low-power small embedded devices, it also overcomes the high computational complexity and "black box" properties of the deep convolutional neural network, and provides an intuitive explanation of the decision process while achieving efficient calculation under low power conditions; and then by determining the output state data of the target neural unit, the navigation state of the unmanned system is controlled, and the intelligent obstacle avoidance decision of the unmanned system is realized, ensuring efficient obstacle avoidance in a complex and changeable environment. Therefore, based on the method of the embodiment of the present application, efficient and intelligent obstacle avoidance is achieved, and the safety and reliability of the unmanned system are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 It is a flowchart of the steps of an efficient and intelligent obstacle avoidance method for an unmanned system based on a heterogeneous brain model provided in an embodiment of the present application; Figure 2 is a flowchart of another method for efficient and intelligent obstacle avoidance of an unmanned system based on a heterogeneous brain model provided in an embodiment of the present application; Figure 3 It is a process of determining input feature data under an embodiment of the present application; Figure 4 It is the architecture of a heterogeneous brain model under the embodiment of the present application; Figure 5 is a schematic diagram of a control method of an unmanned system under an embodiment of the present application; Figure 6 It is a flowchart of the steps of a training method of a heterogeneous brain model for efficient and intelligent obstacle avoidance provided in an embodiment of the present application; Figure 7 It is a training configuration of a heterogeneous brain model provided according to an embodiment of the present application; Figure 8 is a flowchart of the steps of another method for training a heterogeneous brain model for efficient and intelligent obstacle avoidance provided in an embodiment of the present application; Fig. 9 It is a structural schematic diagram of an efficient and intelligent obstacle avoidance device for an unmanned system based on a heterogeneous brain model provided in an embodiment of the present application; Fig.10 It is a structural schematic diagram of a training device for a heterogeneous brain model for efficient and intelligent obstacle avoidance provided in an embodiment of the present application; Fig.11 is a block diagram of an electronic device provided in an embodiment of the present application; Fig.12 It is a block diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0015] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0016] Figure 1This embodiment provides an efficient and intelligent obstacle avoidance method for an unmanned system based on a heterogeneous brain model, which specifically includes the following steps: Step 101 : Acquire driving sample data collected from sensors of an unmanned system, and generate input feature data for inputting a heterogeneous brain model according to the driving sample data.

[0017] Among them, the heterogeneous brain model includes multiple neural units; each input neural unit in the heterogeneous brain model has a corresponding output neural unit; each input neural unit is used to obtain the state value of the output neural unit corresponding to the input neural unit, and solve the state value of the input neural unit according to the acquired state value.

[0018] In some embodiments of the present application, driving sampling data collected from sensors of unmanned systems is obtained, and input feature data for inputting into heterogeneous brain models is generated based on the driving sampling data. This process includes: collecting environmental data acquired by sensors of unmanned systems, and converting these data into input feature data that can be used by heterogeneous brain models. Input feature data refers to specific data converted from driving sampling data and used to initialize and drive heterogeneous brain models. After executing this step, the input feature data provides a basis for the initialization and operation of subsequent models, enabling the model to make full use of environmental perception data for efficient and intelligent obstacle avoidance.

[0019] In a specific example, the LiDAR (Light Detection and Ranging, LiDAR) and camera sensors equipped in the driverless car obtain real-time environmental data. The system collects this data and converts it into an input feature data matrix through a data processing algorithm. These input feature data contain the location information and image features of obstacles in the environment. After executing the above execution process, the input feature data obtained can be directly used to initialize the heterogeneous brain model, enabling the model to accurately perceive the environment and make obstacle avoidance decisions.

[0020] Step 102 , inputting the input feature data into the heterogeneous brain model to initialize each neural unit in the heterogeneous brain model.

[0021] In some embodiments of the present application, each neural unit in the heterogeneous brain model is initialized by inputting input feature data into the heterogeneous brain model. This process includes: passing the generated input feature data to each neural unit in the heterogeneous brain model. After executing this step, each neural unit in the heterogeneous brain model is in a ready state, so that the initialized neural unit can accurately reflect the characteristics of the input data, providing a basis for subsequent model operation and decision-making.

[0022] In a specific example, the system converts the environmental data obtained by the driverless car sensor into an input feature data matrix and inputs this data into each neural unit of the heterogeneous brain model. Each neural unit in the heterogeneous brain model is initialized according to the input data and is ready to receive and process subsequent environmental perception data. After executing the above execution process, the initialized neural unit accurately reflects the characteristics of the input data, enabling the heterogeneous brain model to efficiently make subsequent intelligent obstacle avoidance decisions.

[0023] Step 103 , running the heterogeneous brain model, and determining the target output state data of the target neural unit in the running heterogeneous brain model as the model output data of the heterogeneous brain model.

[0024] In some embodiments of the present application, a heterogeneous brain model is run, and the target output state data of the target neural unit in the heterogeneous brain model after running is determined as the model output data of the heterogeneous brain model. This process includes: starting the heterogeneous brain model, simulating the calculation and interaction between neurons, until the target neural unit generates output state data. The target neural unit refers to the neuron used to generate the final decision output in the heterogeneous brain model. After executing this step, the model output data of the heterogeneous brain model will be used as the basis for the intelligent obstacle avoidance decision of the unmanned system, ensuring its efficient obstacle avoidance in complex and changing environments.

[0025] In a specific example, the system has started the heterogeneous brain model and input the initialization data generated in the previous step. The neural units in the model generate the output state data of the target neural units through a series of operations and interactions. The output state data of the target neural units are determined as the model output data of the heterogeneous brain model and are used to guide the obstacle avoidance operation of the unmanned vehicle. After executing according to the above execution process, the obtained model output data provides the information basis for the unmanned system to make intelligent obstacle avoidance decisions in the actual environment.

[0026] Step 104, controlling the navigation state of the unmanned system according to the model output data.

[0027] In some embodiments of the present application, the navigation state of the unmanned system is controlled according to the model output data. This process includes: receiving the model output data generated by the heterogeneous brain model, and using the data to adjust and control the navigation state of the unmanned system. The model output data refers to the state data output by the target neural unit in the heterogeneous brain model. After executing this step, the unmanned system can perform intelligent obstacle avoidance according to the real-time environment and model decision, thereby achieving efficient and safe navigation.

[0028] In a specific example, the driverless car receives the model output data generated by the heterogeneous brain model, which contains the obstacle avoidance decision information. The system then controls the direction and speed of the driverless car based on these output data so that it can safely avoid obstacles on the road. After executing the above execution process, the adjustment and control information obtained ensures that the driverless car can achieve intelligent obstacle avoidance in a complex traffic environment.

[0029] In summary, in the embodiment of the present application, by using the input feature data generated by acquiring driving sampling data and generating input feature data as the initial input of the heterogeneous brain model, the model can make full use of the environmental perception data for efficient and intelligent obstacle avoidance; and then by inputting the input feature data into each neural unit in the heterogeneous brain model, the model is initialized, so that each neural unit of the model is in a ready state, so that the initialized neural unit can accurately reflect the characteristics of the input data; then by running the heterogeneous brain model, the corresponding relationship between the neural units in the heterogeneous brain model is used to make the data operation process interpretable, while achieving real-time obstacle avoidance on low-power small embedded devices, it also overcomes the high computational complexity and "black box" properties of the deep convolutional neural network, and provides an intuitive explanation of the decision process while achieving efficient calculation under low power conditions; and then by determining the output state data of the target neural unit, the navigation state of the unmanned system is controlled, and the intelligent obstacle avoidance decision of the unmanned system is realized, ensuring efficient obstacle avoidance in a complex and changeable environment. Therefore, based on the method of the embodiment of the present application, efficient and intelligent obstacle avoidance is achieved, and the safety and reliability of the unmanned system are improved.

[0030] Figure 2 Another efficient and intelligent obstacle avoidance method for unmanned systems based on a heterogeneous brain model provided by this embodiment includes the following steps: Step 201 : Acquire driving sample data collected from sensors of an unmanned system, and generate input feature data for inputting a heterogeneous brain model based on the driving sample data.

[0031] Among them, the heterogeneous brain model includes multiple neural units; each input neural unit in the heterogeneous brain model has a corresponding output neural unit; each input neural unit is used to obtain the state value of the output neural unit corresponding to the input neural unit, and solve the state value of the input neural unit according to the acquired state value.

[0032] The method shown in this step has been explained in step 101 and will not be repeated here.

[0033] Optional, such as Figure 3 As shown, the sensor is an image sensor, the driving sampling data is image driving sampling data A, and step 201 includes the following sub-steps: Sub-step 2011, down-sampling the image driving sample data A to obtain a plurality of image block data B of the image driving sample data.

[0034] In some embodiments of the present application, the image driving sampling data A is downsampled to obtain multiple image block data B of the image driving sampling data. This process includes: reducing the resolution of the image or reducing the number of pixels of the image from the original image data obtained by the sensor, and dividing it into multiple small image block data. Downsampling refers to the process of simplifying image data to improve the efficiency of subsequent processing by reducing unnecessary redundant information. After executing this step, the generated multiple image block data B significantly reduces the amount of data, thereby reducing the complexity of subsequent calculations and storage requirements.

[0035] In a specific example, an image sensor of an unmanned vehicle is used to collect images from a video of a city road scene that changes over time t as image driving sampling data A. This video data is input into the system, and the system downsamples each frame of the image and divides the high-resolution image into several small image block data B. The resolution of each image block data B is 256×144. According to this execution process, multiple image block data B can be obtained in the end, and these data blocks can be used for subsequent feature extraction and analysis.

[0036] Sub-step 2012, generating a data vector C for each image block data B, and determining a data matrix P obtained by splicing the generated multiple data vectors C as input feature data.

[0037] In some embodiments of the present application, a data matrix P is obtained by generating a data vector C for each image block data B and splicing the generated multiple data vectors C as input feature data. This process includes: converting each image block data B into a corresponding data vector C, which is a linear representation of the image block data. A data vector refers to a multidimensional vector converted from the image block data. Then, all the generated data vectors C are spliced ​​in sequence to form a data matrix P as input feature data. After executing this step, the data matrix P contains the data features of all image blocks, providing complete environmental perception information for subsequent model input.

[0038] In a specific example, the image captured by the image sensor of the driverless car is downsampled into multiple image block data B of 256×144. Each image block data B is expanded into a data vector C of length 64. Then, the data vectors C of 32×18 image blocks are sequentially spliced ​​together to form a 576×64 data matrix P. After executing the above execution process, the obtained data matrix P is used as input feature data, which contains the data features of all image blocks and can be used for the initialization of the heterogeneous brain model and intelligent obstacle avoidance decision-making.

[0039] Step 202 : inputting the input feature data into the heterogeneous brain model to initialize each neural unit in the heterogeneous brain model.

[0040] The method shown in this step has been explained in step 102 and will not be repeated here.

[0041] Optionally, a corresponding second-order nonlinear differential calculation model is preset in each neural unit of the present application, and the second-order nonlinear differential calculation model is characterized by the following formula group: ; ; in, Represents the state value of the i-th neural unit; Represents the state value of the jth neural unit; represents the recovery variable of the ith neural unit; Represents input feature data; Parameter representing the linear growth rate of neural units; Parameters representing nonlinear effects of heterogeneous brain models; represents the set of neural units connected to the i-th neural unit; represents the weight of the information transmitted between the jth neural unit and the ith neural unit; Represents the perceptual weight of the input feature data to the i-th neural unit; is the coupling parameter, indicating right the extent of the impact; Represents the attenuation parameter, controlling The recovery rate; represents the bias term of the heterogeneous brain model.

[0042] The model can effectively simulate the response behavior of neural units in complex environments through a preset second-order nonlinear differential calculation model, enhancing the adaptability and accuracy of heterogeneous brain models. The reason for this design is to overcome the problems of large computational complexity and poor interpretability of traditional deep convolutional neural networks, so that the model can achieve real-time obstacle avoidance under low power consumption conditions and provide intuitive explanations of the decision-making process.

[0043] like Figure 4 As shown, each black dot in the figure (Ni, i=1,2…,20) represents a neural unit. Further, optionally, multiple neural units can be divided into six data processing areas: acquisition input data processing area M1, primary input data processing area M2, advanced input data processing area M3, integration data processing area M4, decision data processing area M5, and action data processing area M6.

[0044] The above architecture can effectively organize and manage data transmission and processing between neural units by rationally dividing multiple neural units into different data processing areas, thereby improving the efficiency and accuracy of data processing. It can better realize hierarchical processing and integration of data, optimize the overall performance of the neural network, and improve the intelligent obstacle avoidance capability of the system. Through this architecture, the system can achieve real-time obstacle avoidance under low power consumption conditions and provide intuitive explanations of the decision-making process, overcoming the problems of large computational complexity and poor interpretability of traditional deep convolutional neural networks.

[0045] As a preferred embodiment of the present application, Figure 4 As shown, in a further embodiment of the present application, a heterogeneous brain model architecture is determined through a training process, which consists of an acquisition input data processing area M1 including 20 neural units (N1-N8), a primary input data processing area M2 including 4 neural units (N17-N20), an advanced input data processing area M3 including 3 neural units (N14-N16), an integrated data processing area M4 including 2 neural units (N9, N10), a decision data processing area (M5) including 2 neural units (N11, N12), and an action data processing area (M6) including 1 neural unit (N13).

[0046] The above architecture can more efficiently manage and transmit data between neural units by reasonably dividing and training each data processing area, thereby dividing a limited number of neural units and improving the processing efficiency and accuracy of heterogeneous brain models. By functionally dividing and regionalizing neural units, it can better realize hierarchical processing and integration of data, optimize the overall performance of the neural network, and improve the intelligent obstacle avoidance capability of the unmanned system.

[0047] Optional, such as Figure 4 As shown, based on the above second-order nonlinear differential calculation model, it can be set that if and only if the i-th neural unit belongs to the acquisition input data processing area, , to ensure that the collected input data processing area is used as the only external data (ie input feature data P) input module.

[0048] In the preferred embodiment of the present application, in order to clearly distinguish the functions of the data processing area, avoid confusion and redundancy in data input, thereby optimizing the performance of the neural network and enhancing the intelligent obstacle avoidance capability of the heterogeneous brain model, by ensuring that only the neural units in the data processing area that collects input data can receive external data, the source and processing flow of data input can be effectively controlled, and the data processing efficiency and accuracy of the model can be improved. In this way, it is ensured that the data processing area that collects input data is the only external data input module, so that only the neural units in the data processing area that collects input data have non-zero perception weights.

[0049] Optionally, based on the above second-order nonlinear differential calculation model, when solving the state value of the input neural unit according to the acquired state value, the neural unit completes the calculation of the state value through the following iterative steps: Sub-step 2020, iterate the following formula group according to a preset number of iterations to solve the state value of the input neural unit: ; ; ; ; Among them, r and s represent the intermediate variables of the algorithm. represents the step size of iterative solution, Indicates the time corresponding to each state value; Indicated in Input the state value of the neural unit at all times, Indicated in The recovery variable of the neural unit at each moment.

[0050] In some embodiments of the present application, iterations are performed according to a preset number of iterations to solve the state value of the input neural unit. This process includes: repeatedly executing the calculations in the formula group within a preset number of iterations to gradually update the state value of the input neural unit and the recovery variable. Iteration refers to a method of gradually approaching the final result by repeatedly executing calculation steps. After executing this step, the state value and recovery variable of the input neural unit are effectively updated to ensure that the model can accurately reflect the characteristics of the input data. Through the calculation of the preset number of iterations and the formula group, the state value of the input neural unit can be effectively updated and solved, thereby improving the computational efficiency and accuracy of the model. The reason for this design is that the second-order nonlinear differential model is converted into an iterative model that can be quickly calculated through the formula, which can simulate the dynamic behavior of the neural unit more quickly and improve the real-time performance of the heterogeneous brain model in a complex environment.

[0051] Step 203 , running the heterogeneous brain model, and determining the target output state data of the target neural unit in the running heterogeneous brain model as the model output data of the heterogeneous brain model.

[0052] The method shown in this step has been explained in step 103 and will not be repeated here.

[0053] Step 204, obtaining the value of the target data component in the model output data.

[0054] In some embodiments of the present application, the value of the target data component in the model output data is obtained. This process includes: extracting the output component value of a specific target neural unit from the model output data generated by the heterogeneous brain model. The target data component refers to the specific value in the model output data used to control the navigation of the unmanned system. After executing this step, the obtained target data component will be used for the navigation control of the unmanned system in the subsequent steps to ensure that the system can make accurate obstacle avoidance decisions based on real-time environmental data.

[0055] In a specific example, after running the heterogeneous brain model, the system extracts the output component value of the target neural unit in the model output data. This value represents the specific decision information of the driverless car during the obstacle avoidance process, such as steering angle or speed adjustment. After executing the above execution process, the obtained target data component value will be used in the control system of the driverless car to ensure that it can perform safe and effective obstacle avoidance operations according to the real-time environment.

[0056] Step 205, controlling the navigation direction of the unmanned system according to the positive or negative value of the target data component, and controlling the navigation speed of the unmanned system according to the absolute value of the target data component.

[0057] In some embodiments of the present application, the navigation direction of the unmanned system is controlled according to the positive or negative value of the target data component, and the navigation speed of the unmanned system is controlled according to the absolute value of the target data component. This process includes: analyzing the value of the target data component, determining its positive or negative condition to determine in which direction the unmanned system should move. The target data component refers to a specific numerical value extracted from the model output data. Then, the navigation speed of the unmanned system is adjusted according to the absolute value of the target data component. After executing this step, the unmanned system can quickly adjust its navigation direction and speed according to environmental changes, achieving more flexible and efficient obstacle avoidance.

[0058] In a specific example, Figure 5As shown, the control system of the drone receives the output data of the model that changes with time t. Assume that at a certain moment the target data component is -0.8. The system determines that the drone needs to move to the left based on the positive or negative value of this value, and adjusts the steering speed of the drone based on its absolute value (0.8). For example, a value of -0.8 may mean that the drone needs to move to the left at a speed of 0.8 meters per second. After executing according to the above execution process, the adjustment information obtained ensures that the drone can quickly avoid obstacles on the road and achieve safe and reliable automatic flight.

[0059] In summary, in the embodiment of the present application, by using the input feature data generated by acquiring driving sampling data and generating input feature data as the initial input of the heterogeneous brain model, the model can make full use of the environmental perception data for efficient and intelligent obstacle avoidance; and then by inputting the input feature data into each neural unit in the heterogeneous brain model, the model is initialized, so that each neural unit of the model is in a ready state, so that the initialized neural unit can accurately reflect the characteristics of the input data; then by running the heterogeneous brain model, the corresponding relationship between the neural units in the heterogeneous brain model is used to make the data operation process interpretable, while achieving real-time obstacle avoidance on low-power small embedded devices, it also overcomes the high computational complexity and "black box" properties of the deep convolutional neural network, and provides an intuitive explanation of the decision process while achieving efficient calculation under low power conditions; and then by determining the output state data of the target neural unit, the navigation state of the unmanned system is controlled, and the intelligent obstacle avoidance decision of the unmanned system is realized, ensuring efficient obstacle avoidance in a complex and changeable environment. Therefore, based on the method of the embodiment of the present application, efficient and intelligent obstacle avoidance is achieved, and the safety and reliability of the unmanned system are improved.

[0060] Figure 6 A training method for a heterogeneous brain model for efficient and intelligent obstacle avoidance provided in this embodiment is used to train the heterogeneous brain model in the above embodiment of the efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models, and specifically includes the following steps: Step 301, divide the heterogeneous brain model into multiple data processing areas, so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and establish a connection relationship between the neural units of every two data processing areas according to the initialized connection strength.

[0061] The connection strength is used to characterize the number and weight of connections between multiple neural units included in two data processing areas.

[0062] In some embodiments of the present application, a heterogeneous brain model is divided into multiple data processing areas so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and a connection relationship between the neural units in every two data processing areas is established according to the initialized connection strength. This process includes: first, the entire heterogeneous brain model is divided into multiple data processing areas so that each neural unit in the model can correspond to a data processing area. Then, according to the pre-set connection strength, a connection relationship is established between the neural units in every two data processing areas. Connection strength refers to a parameter used to characterize the number of connections and connection weights between multiple neural units contained in two data processing areas. After executing this step, through reasonable connection strength settings, the connection between the neural units of the heterogeneous brain model is made more reasonable and effective, laying the foundation for subsequent model training and operation.

[0063] In a specific example, Figure 4 As shown, the heterogeneous brain model of the unmanned system can be divided into six data processing areas: acquisition input data processing area M1, primary input data processing area M2, advanced input data processing area M3, integrated data processing area M4, decision data processing area M5, and action data processing area M6. Each data processing area contains a specific number of neural units. Then, according to the initialized connection strength, a connection relationship can be established between the neural units in each data processing area. The strength of these connection relationships characterizes the number and weight of connections between each data processing area. After executing according to the above execution process, the reasonable connection strength and connection relationship obtained enable the heterogeneous brain model to have a good infrastructure to support the subsequent model training and operation of efficient and intelligent obstacle avoidance.

[0064] Step 302: Train the connection strength to obtain a trained heterogeneous brain model.

[0065] In some embodiments of the present application, the connection strength is trained to obtain a trained heterogeneous brain model. This process includes: adjusting and optimizing the connection strength between each data processing area in the heterogeneous brain model according to a predetermined training algorithm. Connection strength refers to the number of connections and connection weights between multiple neural units contained in two data processing areas. By continuously adjusting these connection parameters, the model can learn and optimize the information transmission path between neural units. After executing this step, the obtained trained heterogeneous brain model can process input data more efficiently and achieve intelligent obstacle avoidance.

[0066] In a specific example, the connection strength of the heterogeneous brain model can be trained using the Gradient Descent Algorithm (GDA). During the training process, the model continuously adjusts the connection weights to minimize the prediction error. The performance of the model is monitored and the connection strength is optimized based on the feedback information. After multiple iterations of training, the trained heterogeneous brain model can accurately and quickly process the input data and achieve intelligent obstacle avoidance. After executing according to the above execution process, the trained heterogeneous brain model shows excellent obstacle avoidance performance in complex and changing environments.

[0067] like Figure 7 As shown, the number of connections and connection weights between various neural units in a heterogeneous brain model (including 20 neural units) obtained by training a heterogeneous brain model provided in an embodiment of the present application. In the figure, each square in the square table on the left represents the data transmission relationship (that is, the number of connections) between any two neural units; and the depth of the square determines the connection weight between the two neural units; the upper half of the table is used to represent the activation effect (positive weight) of the transmitted data on the neural unit, and the lower half of the table is used to represent the inhibition effect (negative weight) of the transmitted data on the neural unit.

[0068] In summary, in the embodiment of the present application, by obtaining the untranslated original text of the streaming text and inputting it into the semantic evaluation model to obtain the semantic integrity score of the target single sentence, it is possible to determine whether the target single sentence is a complete semantic unit, thereby deciding whether to update the untranslated original text; when the target single sentence meets the preset conditions, the untranslated original text is input into the machine translation model to obtain the translation result of the streaming text. Therefore, based on the method of the embodiment of the present application, when performing machine translation of streaming text, real-time translation of the streaming text can be achieved without waiting for the entire text to be input, which solves the problem in the related art that it is difficult to perform effective segmentation and translation due to the uncertain input time of the streaming text, resulting in low translation efficiency and low translation quality. In addition, in the embodiment of the present application, by using the semantic evaluation model to evaluate the semantic integrity of the target single sentence, translation errors or ambiguities caused by single sentence segmentation errors can be effectively avoided.

[0069] Figure 8 Another method for training a heterogeneous brain model for efficient and intelligent obstacle avoidance provided by this embodiment is used to train the heterogeneous brain model in the above embodiment of the efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models, and specifically includes the following steps: Step 401 , dividing the heterogeneous brain model into multiple data processing areas according to the number of neural units corresponding to each data processing area.

[0070] Therein, a corresponding number of connections and a probability model are preset between each data processing area and the target data processing area.

[0071] In some embodiments of the present application, the heterogeneous brain model is divided into multiple data processing areas according to the number of neural units corresponding to each data processing area. This process includes: dividing the heterogeneous brain model into several data processing areas according to the number of neural units in the heterogeneous brain model, so that each data processing area contains a specific number of neural units. A corresponding number of connections and probability models are preset between each data processing area and the target data processing area. These probability models characterize the connection probability and number of connections between data processing areas. After executing this step, the multiple data processing areas obtained can manage and connect neural units more effectively, which is helpful for subsequent training and model optimization.

[0072] In a specific example, the heterogeneous brain model of the unmanned system is divided into multiple data processing areas. And according to the number of neural units and functional requirements of each data processing area, the number of connections and probability models between the areas are set. For example, the number of connections and probability models from the visual data processing area to the primary visual data processing area may be set as each visual neuron connects to 2 primary visual neurons, with a connection probability of 50%. After executing according to the above execution process, the connection relationship between the multiple data processing areas is reasonable, which improves the processing efficiency and accuracy of the model.

[0073] Step 402, taking each neural unit in the data processing area as an output neural unit, determining multiple target neural units in the target data processing area according to the probability model and the number of connections to the target data processing area to serve as input neural units of the output neural units, and establishing a data connection between the output neural units and the input neural units.

[0074] In some embodiments of the present application, by treating each neural unit in the data processing area as an output neural unit, multiple target neural units are determined in the target data processing area according to the probability model and the number of connections to the target data processing area as input neural units of the output neural units, and a data connection is established between the output neural unit and the input neural unit. This process includes: first, marking each neural unit in the data processing area as an output neural unit. Then, according to the set probability model and the number of connections, multiple target neural units are selected in the target data processing area as corresponding input neural units. Data connection refers to the information transmission path established between the output neural unit and the input neural unit. After executing this step, the connection relationship between the neural units in the data processing area is clear, supporting the model to effectively transmit and process data in subsequent steps.

[0075] In a specific example, some neural units in the heterogeneous brain model are marked as output neural units. According to the preset probability model and the number of connections, multiple target neural units are selected as input neural units of these output neural units. For example, each visual neural unit is connected to 2 primary visual neural units with a connection probability of 50%. Then, by establishing data connections between these output neural units and the input neural units, it is ensured that information can be effectively transmitted between the neural units.

[0076] Step 403, establishing activation weights between output data of the corresponding output neural unit and input data of the input neural unit according to a probability model corresponding to the data connection between the corresponding output neural unit and the input neural unit.

[0077] In some embodiments of the present application, an activation weight between the output data of the corresponding output neural unit and the input data of the input neural unit is established according to a probability model corresponding to the data connection between the corresponding output neural unit and the input neural unit. This process includes: analyzing a predetermined probability model and the number of connections, and allocating activation weights between the output neural unit and the input neural unit based on this information. Activation weight refers to a parameter used to characterize the connection strength and information transmission efficiency in the neural unit connection. After executing this step, the activation weight between the output data and the input data is established, so that the model can efficiently transmit and process information.

[0078] In a specific example, the connection between the visual area and the primary visual area in the heterogeneous brain model is analyzed by using a set probability model. For example, the corresponding activation weights can be assigned between the output neural units of the visual area and the input neural units of the primary visual area: the connection weight between each visual neural unit and the primary visual neural unit may be set to 0.5. In this setting, the activation weight characterizes the connection strength and information transmission efficiency between the two data processing areas. After executing according to the above execution process, the activation weight obtained ensures that the model can efficiently transmit and process information during operation, thereby improving the overall performance of the heterogeneous brain model.

[0079] Step 404: train the connection strength to obtain a trained heterogeneous brain model.

[0080] The method shown in this step has been explained in step 302 and will not be repeated here.

[0081] In summary, in the embodiment of the present application, by using the input feature data generated by acquiring driving sampling data and generating input feature data as the initial input of the heterogeneous brain model, the model can make full use of the environmental perception data for efficient and intelligent obstacle avoidance; and then by inputting the input feature data into each neural unit in the heterogeneous brain model, the model is initialized, so that each neural unit of the model is in a ready state, so that the initialized neural unit can accurately reflect the characteristics of the input data; then by running the heterogeneous brain model, the corresponding relationship between the neural units in the heterogeneous brain model is used to make the data operation process interpretable, while achieving real-time obstacle avoidance on low-power small embedded devices, it also overcomes the high computational complexity and "black box" properties of the deep convolutional neural network, and provides an intuitive explanation of the decision process while achieving efficient calculation under low power conditions; and then by determining the output state data of the target neural unit, the navigation state of the unmanned system is controlled, and the intelligent obstacle avoidance decision of the unmanned system is realized, ensuring efficient obstacle avoidance in a complex and changeable environment. Therefore, based on the method of the embodiment of the present application, efficient and intelligent obstacle avoidance is achieved, and the safety and reliability of the unmanned system are improved.

[0082] like Fig. 9 As shown, the embodiment of the present application also discloses an unmanned system efficient intelligent obstacle avoidance device 50 based on a heterogeneous brain model, which is applied to a controller of an unmanned system, including: The acquisition module 501 is used to acquire driving sampling data acquired from the sensor of the unmanned system, and generate input feature data for inputting into the heterogeneous brain model according to the driving sampling data; the heterogeneous brain model includes a plurality of neural units; each input neural unit in the heterogeneous brain model has a corresponding output neural unit; each input neural unit is used to acquire the state value of the output neural unit corresponding to the input neural unit, and solve the state value of the input neural unit according to the acquired state value; An input module 502, for inputting input feature data into the heterogeneous brain model to initialize each neural unit in the heterogeneous brain model; An output module 503 is used to run the heterogeneous brain model and determine the target output state data of the target neural unit in the heterogeneous brain model after running as the model output data of the heterogeneous brain model; The control module 504 is used to control the navigation state of the unmanned system according to the model output data.

[0083] Optionally, the sensor is an image sensor, the driving sampling data is image driving sampling data, and the acquisition module 501 includes: A downsampling submodule, used for downsampling the image driving sampling data to obtain a plurality of image block data of the image driving sampling data; The feature submodule is used to generate a data vector for each image block data, and determine the data matrix obtained by splicing the generated multiple data vectors as input feature data.

[0084] Optionally, the control module 504 includes: The sub-module is used to obtain the value of the target data component in the model output data; The control submodule is used to control the navigation direction of the unmanned system according to the positive or negative value of the target data component, and to control the navigation speed of the unmanned system according to the absolute value of the target data component.

[0085] like Fig.10 As shown, the embodiment of the present application further discloses a training device 60 for a heterogeneous brain model for efficient and intelligent obstacle avoidance, which is used to train the heterogeneous brain model in the above embodiment of the efficient and intelligent obstacle avoidance method for an unmanned system based on a heterogeneous brain model, including: Initialization module 601, used to divide the heterogeneous brain model into multiple data processing areas, so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and establish a connection relationship between the neural units of every two data processing areas according to the initialized connection strength; the connection strength is used to characterize the number and connection weight of the connections between the multiple neural units contained in the two data processing areas; The training module 602 is used to train the connection strength to obtain a trained heterogeneous brain model.

[0086] Optionally, the initialization module 601 includes: A division submodule is used to divide the heterogeneous brain model into multiple data processing areas according to the number of neural units corresponding to each data processing area; a corresponding number of connections and a probability model are preset between each data processing area and the target data processing area; A connection submodule, used to use each neural unit in the data processing area as an output neural unit, determine multiple target neural units in the target data processing area according to the probability model and the number of connections between the target data processing area and the target data processing area to serve as input neural units of the output neural units, and establish data connections between the output neural units and the input neural units; The activation submodule is used to establish the activation weight between the output data of the corresponding output neural unit and the input data of the input neural unit according to the probability model corresponding to the data connection between the corresponding output neural unit and the input neural unit.

[0087] In summary, in the embodiment of the present application, by using the input feature data generated by acquiring driving sampling data and generating input feature data as the initial input of the heterogeneous brain model, the model can make full use of the environmental perception data for efficient and intelligent obstacle avoidance; and then by inputting the input feature data into each neural unit in the heterogeneous brain model, the model is initialized, so that each neural unit of the model is in a ready state, so that the initialized neural unit can accurately reflect the characteristics of the input data; then by running the heterogeneous brain model, the corresponding relationship between the neural units in the heterogeneous brain model is used to make the data operation process interpretable, while achieving real-time obstacle avoidance on low-power small embedded devices, it also overcomes the high computational complexity and "black box" properties of the deep convolutional neural network, and provides an intuitive explanation of the decision process while achieving efficient calculation under low power conditions; and then by determining the output state data of the target neural unit, the navigation state of the unmanned system is controlled, and the intelligent obstacle avoidance decision of the unmanned system is realized, ensuring efficient obstacle avoidance in a complex and changeable environment. Therefore, based on the method of the embodiment of the present application, efficient and intelligent obstacle avoidance is achieved, and the safety and reliability of the unmanned system are improved.

[0088] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned unmanned system efficient intelligent obstacle avoidance method based on heterogeneous brain model or the training method of heterogeneous brain model for efficient intelligent obstacle avoidance is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0089] Fig.11 700 is a block diagram of an electronic device 700 provided in an embodiment of the present application. For example, the electronic device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0090] Reference Fig.11 , the electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0091] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned unmanned system efficient intelligent obstacle avoidance method based on heterogeneous brain models or the training method of heterogeneous brain models for efficient intelligent obstacle avoidance. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0092] The memory 704 is used to store various types of data to support the operation of the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, multimedia, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0093] The power supply component 706 provides power to the various components of the electronic device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 700.

[0094] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the demarcation of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a multimedia mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0095] The audio component 710 is used to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the electronic device 700 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is used to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0096] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0097] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the position change of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and the temperature change of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0098] The communication component 716 is used to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 7G), or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0099] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to implement the efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models or the training method for heterogeneous brain models for efficient and intelligent obstacle avoidance provided in the embodiments of the present application.

[0100] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the above instructions can be executed by the processor 720 of the electronic device 700 to complete the above-mentioned unmanned system efficient intelligent obstacle avoidance method based on heterogeneous brain model or the training method of heterogeneous brain model for efficient intelligent obstacle avoidance. For example, the non-transitory storage medium can be ROM, random access memory (RAM), CD-ROM, tape, floppy disk, optical data storage device, etc.

[0101] Fig.12 8 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be provided as a server. Fig.12 , the electronic device 800 includes a processing component 822, which further includes one or more processors, and a memory resource represented by a memory 832 for storing instructions that can be executed by the processing component 822, such as an application. The application stored in the memory 832 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 822 is configured to execute instructions to execute the unmanned system efficient intelligent obstacle avoidance method based on a heterogeneous brain model or the training method of a heterogeneous brain model for efficient intelligent obstacle avoidance provided in an embodiment of the present application.

[0102] The electronic device 800 may also include a power supply component 826 configured to perform power management of the electronic device 800, a wired or wireless network interface 850 configured to connect the electronic device 800 to a network, and an input / output (I / O) interface 858. The electronic device 800 may operate based on an operating system stored in the memory 832, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0103] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements an efficient and intelligent obstacle avoidance method for an unmanned system based on a heterogeneous brain model or a training method for a heterogeneous brain model for efficient and intelligent obstacle avoidance.

[0104] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0105] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0106] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0107] It is easy for a person skilled in the art to think that any combination of the above embodiments is feasible, so any combination of the above embodiments is an implementation scheme of the present application. However, due to space limitations, this specification will not describe them in detail here.

[0108] The efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models or the training method for heterogeneous brain models for efficient and intelligent obstacle avoidance provided herein are not inherently related to any specific computer, virtual system or other device. Various general systems can also be used with the teaching based on this. According to the above description, it is obvious to construct the structure required for the system with the solution of the present application. In addition, the present application is not directed to any specific programming language. It should be understood that the content of the present application described herein can be implemented using various programming languages, and the above description of specific languages ​​is to disclose the best implementation mode of the present application.

[0109] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0110] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various application aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed application requires more features than the features explicitly recited in each claim. More specifically, as reflected in the claims, the application aspects lie in less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present application.

[0111] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device so disclosed may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0112] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0113] The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the unmanned system efficient intelligent obstacle avoidance method based on a heterogeneous brain model or a training method for a heterogeneous brain model for efficient intelligent obstacle avoidance according to an embodiment of the present application. The present application may also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present application may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0114] In another embodiment provided by the present invention, a computer program product containing instructions is also provided. When the computer is run on a computer, the computer executes the unmanned system efficient and intelligent obstacle avoidance method based on a heterogeneous brain model or the training method of a heterogeneous brain model for efficient and intelligent obstacle avoidance according to the embodiment of the present application.

[0115] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0116] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0117] It should be noted that, for the sake of simplicity, the method embodiments of the present application are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described action sequence, because according to the embodiments of the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0118] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the system or device, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. An efficient and intelligent obstacle avoidance method for unmanned systems based on a heterogeneous brain model, characterized in that: Controllers for unmanned systems include: Acquire driving sampling data collected from sensors of an unmanned system, and generate input feature data for inputting a heterogeneous brain model according to the driving sampling data; the heterogeneous brain model comprises a plurality of neural units; each input neural unit in the heterogeneous brain model has a corresponding output neural unit; each input neural unit is used to acquire a state value of an output neural unit corresponding to the input neural unit, and solve the state value of the input neural unit according to the acquired state value; Inputting the input feature data into the heterogeneous brain model to initialize each neural unit in the heterogeneous brain model; Running the heterogeneous brain model, and determining target output state data of a target neural unit in the running heterogeneous brain model as model output data of the heterogeneous brain model; The navigation state of the unmanned system is controlled according to the model output data.

2. The efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models according to claim 1, characterized in that: The sensor is an image sensor, the driving sample data is image driving sample data, and the driving sample data collected from the sensor of the unmanned system is acquired, and input feature data for inputting into the heterogeneous brain model is generated according to the driving sample data, including: Downsampling the image driving sampling data to obtain a plurality of image block data of the image driving sampling data; A data vector is generated for each of the image block data, and a data matrix obtained by splicing the generated multiple data vectors is determined as the input feature data.

3. The efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models as claimed in claim 1, characterized in that: Each of the neural units is preset with a corresponding second-order nonlinear differential calculation model, which is characterized by the following formula group: ； ； in, Represents the state value of the i-th neural unit; Represents the state value of the jth neural unit; represents the recovery variable of the ith neural unit; Represents input feature data; Parameter representing the linear growth rate of neural units; a parameter representing a nonlinear effect of the heterogeneous brain model; represents the set of neural units connected to the i-th neural unit; represents the weight of the information transmitted between the jth neural unit and the i-th neural unit; Represents the perceptual weight of the input feature data to the i-th neural unit; is the coupling parameter, indicating right the extent of the impact; Represents the attenuation parameter, controlling The recovery rate; represents the bias term of the heterogeneous brain model.

4. The efficient and intelligent obstacle avoidance method for unmanned systems based on heterogeneous brain models as claimed in claim 3, characterized in that: The step of solving the state value of the input neural unit according to the acquired state value comprises: The following formula group is iterated according to a preset number of iterations to solve the state value of the input neural unit: ； ； ； ； Among them, r and s represent the intermediate variables of the algorithm. represents the step size of iterative solution, Indicates the time corresponding to each state value; Indicated in The input neural unit state value at the moment, Indicated in The restored variable of the input neural unit at this moment.

5. A method for training a heterogeneous brain model for efficient and intelligent obstacle avoidance, characterized in that: The method for training a heterogeneous brain model in an efficient and intelligent obstacle avoidance method for an unmanned system based on a heterogeneous brain model as claimed in any one of claims 1 to 4 comprises: Dividing the heterogeneous brain model into a plurality of data processing areas so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and establishing a connection relationship between the neural units of each two data processing areas according to an initialized connection strength; the connection strength is used to characterize the number of connections and connection weights between the plurality of neural units contained in the two data processing areas; The connection strength is trained to obtain a trained heterogeneous brain model.

6. The method for training a heterogeneous brain model for efficient and intelligent obstacle avoidance according to claim 5, characterized in that: The step of dividing the heterogeneous brain model into a plurality of data processing areas so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and establishing a connection relationship between neural units in every two data processing areas according to an initialized connection strength, comprises: According to the number of neural units corresponding to each of the data processing areas, the heterogeneous brain model is divided into a plurality of data processing areas; a corresponding number of connections and a probability model are preset between each data processing area and a target data processing area; Taking each neural unit in the data processing area as an output neural unit, determining a plurality of target neural units in the target data processing area according to the probability model and the number of connections to the target data processing area to serve as input neural units of the output neural unit, and establishing a data connection between the output neural unit and the input neural unit; According to the probability model corresponding to the data connection between the corresponding output neural unit and the input neural unit, the activation weight between the output data of the corresponding output neural unit and the input data of the input neural unit is established.

7. An efficient and intelligent obstacle avoidance device for unmanned systems based on a heterogeneous brain model, characterized in that: Controllers for unmanned systems include: A collection module, used to obtain driving sample data collected from sensors of an unmanned system, and generate input feature data for inputting a heterogeneous brain model according to the driving sample data; the heterogeneous brain model includes a plurality of neural units; each input neural unit in the heterogeneous brain model has a corresponding output neural unit; each input neural unit is used to obtain a state value of an output neural unit corresponding to the input neural unit, and solve the state value of the input neural unit according to the obtained state value; An input module, used for inputting the input feature data into the heterogeneous brain model to initialize each neural unit in the heterogeneous brain model; an output module, used for running the heterogeneous brain model, and determining target output state data of target neural units in the running heterogeneous brain model as model output data of the heterogeneous brain model; A control module is used to control the navigation state of the unmanned system according to the model output data.

8. A training device for a heterogeneous brain model for efficient and intelligent obstacle avoidance, characterized in that: The method for training a heterogeneous brain model in an efficient and intelligent obstacle avoidance method for an unmanned system based on a heterogeneous brain model as claimed in any one of claims 1 to 4 comprises: An initialization module, used to divide the heterogeneous brain model into a plurality of data processing areas, so that each neural unit in the heterogeneous brain model has a corresponding data processing area, and establish a connection relationship between the neural units of every two data processing areas according to the initialized connection strength; the connection strength is used to characterize the number of connections and connection weights between the plurality of neural units contained in the two data processing areas; The training module is used to train the connection strength to obtain a trained heterogeneous brain model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 6 when executed by the processor.

Images