Perception model combination training method and device based on calculation foundation platform
By decomposing the intelligent driving perception model into common functional modules and conducting combined training, the low efficiency of perception system development and difficulty in model adaptation under different vehicle models and sensor position changes are solved, and efficient and flexible intelligent driving perception algorithm development is achieved. Improves system performance and maintainability.
Patent Information
- Application Number
- CN202411079784.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-06
AI Technical Summary
When facing changes in different vehicle models and sensor positions, existing intelligent driving perception systems need to re-acquisition, cleaning and training, resulting in low development efficiency, poor reusability, and difficult model adaptation, resulting in high development costs and low efficiency.
通过将感知模型分解为多个共性功能模块,进行排列组合和迭代训练,生成多组合感知模型及其权重,支持用户灵活构建自定义的智能驾驶感知算法,降低系统复杂性和开发成本。
It improves the development efficiency and performance of the perceived system, increases maintainability, portability and scalability, reduces R&D cycle and development costs, and improves the generalization and robustness of the model.
Smart Images

Figure CN119091279A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual perception technology, and in particular to a perception model combination training method and device based on a computing basic platform. Background Art
[0002] The Computing Brain DEvelopment System (CBDES) consists of two parts: the Computing Base Brain (CBB) and the Graphical ADAS-AD Software Developer (GAASD). Among them, CBB consists of computing platform hardware, real-time kernel, middleware and functional software. Functional software is the core of this product, which aims to provide basic algorithm components and frameworks for various intelligent driving systems. Based on the innovative solution of "layered decoupling and cross-domain sharing" and based on the functional software function library, CBDES transfers the application algorithm development capabilities to host manufacturers, supports host engineers to quickly build their own defined intelligent driving systems, and perform functional adaptation and parameter adjustment. Compared with existing products and development models, CBDES has significant advantages in improving development efficiency, ensuring software quality and adopting generative development.
[0003] The environmental perception system is the channel for smart driving cars to obtain external information. It is responsible for the comprehensive collection, processing and analysis of the vehicle's surrounding environment information. It is the basis and prerequisite for the autonomous driving of smart vehicles. The perception system of smart driving cars uses sensors such as cameras, lidar, and millimeter-wave radar to replace human eyes, collect, process and analyze environmental information and in-vehicle information, and understand the local driving environment around the car in real time, including static information such as lane lines, traffic lights, and traffic signs, as well as dynamic obstacle information such as motor vehicles, non-motor vehicles, and pedestrians. As a core component of intelligent driving technology, the perception system provides important input information for downstream module positioning, prediction, decision-making, planning, and control. The quality of the underlying algorithm must be very high. Improving the accuracy, precision, robustness, and real-time performance of the perception system is a key link to ensure the safety and comfort of smart driving cars, and it is also an important prerequisite for its formal implementation.
[0004] With the rapid development of autonomous driving and artificial intelligence technology, visual BEV perception technology has developed rapidly under the trend of focusing on perception and neglecting mapping. By mapping surround image features to BEV space through Transformer technology, deep neural networks are designed for training and reasoning for various functional requirements such as 3D targets and local maps, which can comprehensively improve the accuracy and recall of local driving environment perception. However, in the process of the development and popularization of visual perception, the perception system solutions widely used in the industry currently face the following difficulties and challenges: 1) In the face of different models, different functions, and even changes in sensor positions, data collection, cleaning, labeling and training are required again, and the data scale is huge and the utilization efficiency is low. 2) The development and application process of intelligent driving perception algorithms still has problems such as long development cycle, poor reusability, and complex and numerous platform and sensor configuration steps; mainstream solutions lack functional software, common algorithm modules are not extracted, and the degree of component reuse is low; 3) As the intelligent driving system platform gradually diversifies and the function update speed is accelerated, it is easy to cause engineers to repeat work, model adaptation is difficult, and development costs are high; in the reasoning deployment stage, the visual perception model lacks common functional module disassembly and hierarchical reuse, which makes efficiency optimization for perception mass production more difficult. That is, the existing perception algorithm development has problems such as difficulty in reusing black boxes and difficulty in generalizing to multiple vehicle models. Summary of the invention
[0005] The present invention provides a perception model combination training method and device based on a computing basic platform to solve the problems of difficulty in reusing black boxes and difficulty in generalizing multiple vehicle models.
[0006] The first aspect of the present invention provides a perception model combination training method based on a computing basic platform, comprising the following steps: based on the computing basic platform, decomposing the target perception model into multiple common functional modules; arranging and combining each common functional module to obtain multiple combined perception models; iteratively training the multiple combined perception models according to the target model until a preset maximum number of training rounds is reached, to obtain the final multi-combination perception model and the weight of each of its functional modules.
[0007] Optionally, the multiple common functional modules include an image feature extraction module, an image perspective vision to bird's-eye view vision perspective conversion module, a bird's-eye view vision image temporal feature fusion module and a detection head module.
[0008] Optionally, each common functional module is arranged and combined to obtain a plurality of combined perception models, including:
[0009] Each common functional module is decomposed into a plurality of functional sub-modules; and the plurality of functional sub-modules of each common functional module are arranged and combined to obtain the plurality of combined perception models.
[0010] Optionally, the iterative training of the multiple combined perception models according to the target model until a preset maximum number of training rounds is reached to obtain the final multiple combined perception models and the weight of each functional module thereof includes:
[0011] Perform short-cycle training on each combined perception model to obtain the weight of each combined perception model after training; select a target functional module from the multiple combined perception models according to the target model; average the weights of the target functional modules in the multiple combined perception models based on the weight of each combined perception model after training to obtain a weight mean; copy the weight mean to the target functional modules in the multiple combined perception models to form multiple new combined perception models; iterate the above-mentioned training process until the preset maximum training rounds are reached to obtain the weight of the final multiple combined perception model and each functional module thereof.
[0012] The second aspect of the present invention provides a perception model combination training device based on a computing basic platform, including: a decomposition unit, used to decompose the target perception model into multiple common functional modules based on the computing basic platform; a combination unit, used to arrange and combine each common functional module to obtain multiple combined perception models; an iterative training unit, used to iteratively train the multiple combined perception models according to the target model until a preset maximum training round is reached, so as to obtain the final multi-combination perception model and the weight of each functional module thereof.
[0013] Optionally, the multiple common functional modules include an image feature extraction module, an image perspective vision to bird's-eye view vision perspective conversion module, a bird's-eye view vision image temporal feature fusion module and a detection head module.
[0014] Optionally, the combination unit comprises:
[0015] A disassembly sub-unit, used for disassembling each common functional module into a plurality of functional sub-modules;
[0016] The combining sub-unit is used to arrange and combine the multiple functional sub-modules of each common functional module to obtain the multiple combined perception models.
[0017] Optionally, the iterative training unit includes:
[0018] A training subunit, used for performing short-cycle training on each combined perception model to obtain a weight of each combined perception model after training;
[0019] A selection subunit, used for selecting a target functional module from the multiple combined perception models according to the target model;
[0020] An averaging subunit, configured to average the weights of the target functional modules in the plurality of combined perception models based on the weight of each combined perception model after training to obtain a weight mean;
[0021] A copying subunit, used for copying the weight mean to the target function module in the multiple combined perception models to form a new multiple combined perception models;
[0022] The iterative subunit is used to iteratively execute the aforementioned training process until the preset maximum training rounds are reached, thereby obtaining the final multi-combination perception model and the weight of each functional module thereof.
[0023] The third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the perception model combination training method based on the computing base platform as described in the above embodiment.
[0024] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned perception model combination training method based on a computing base platform.
[0025] The embodiment of the present invention proposes a perception model combination training method and device based on a computing basic platform. Based on the design and development concept of hierarchical decoupling, the perception model is divided into different functional modules, each module is responsible for a specific function, and supports users to select and combine different functional modules according to target requirements, and flexibly construct a customized intelligent driving perception algorithm, thereby reducing the complexity of the perception system, improving development efficiency and system performance, increasing maintainability, portability and scalability, and reducing R&D cycle and development costs.
[0026] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0028] Figure 1 A flowchart of a perception model combination training method based on a computing basic platform provided by an embodiment of the present invention;
[0029] Figure 2 The disassembly and combined training architecture diagram of the autonomous driving perception model module for computing basic platform application provided by an embodiment of the present invention;
[0030] Figure 3 A schematic diagram of disassembling and assembling a perception algorithm (i.e., a target perception model) using an obstacle perception model as an example provided by an embodiment of the present invention;
[0031] Figure 4 A schematic diagram of module enumeration and combination using an obstacle perception model as an example provided in an embodiment of the present invention;
[0032] Figure 5 A schematic diagram of the structure of a perception model combination training device based on a computing basic platform provided in an embodiment of the present invention;
[0033] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0034] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0035] The following describes the perception model combination training method and device based on the computing basic platform of the embodiment of the present invention with reference to the accompanying drawings. In view of the problems mentioned in the background technology center above that the existing perception algorithm development has the problem of difficulty in reusing black boxes and difficulty in generalizing to multiple models, the present invention provides a perception model combination training method and device based on the computing basic platform, in which a standardized perception common functional module disassembly training and combination reconstruction framework that supports multiple models, multiple functions and multiple platforms is designed. In this way, the problems of difficulty in reusing black boxes and difficulty in generalizing to multiple models in the existing perception algorithm development are solved.
[0036] Specifically, Figure 1 A flowchart of a perception model combination training method based on a computing basic platform provided in an embodiment of the present invention.
[0037] like Figure 1 As shown, the perception model combination training method based on the computing basic platform includes the following steps:
[0038] In step S101, based on the computing basic platform, the target perception model is decomposed into multiple common functional modules.
[0039] In some embodiments, the plurality of common functional modules include an image feature extraction module, an image perspective vision to bird's-eye view vision perspective conversion module, a bird's-eye view image temporal feature fusion module and a detection head module.
[0040] like Figure 2As shown in the figure, in the actual implementation process, based on the idea of "layered decoupling and cross-domain sharing", for the single-vehicle intelligent system, the target perception model is decomposed into a series of reusable and reconfigurable common functional modules, mainly including the image feature extraction module (Backbone_Camera), the image feature conversion from perspective vision to bird's-eye vision module (PV2BEV), the bird's-eye vision image temporal feature fusion module (TFF_Camera), and the obstacle detection head module (OBH_Detection).
[0041] Among them, the main function of the image feature extraction module is to extract low-level and high-level visual features from the input image to form a general visual representation. This module is usually completed using a convolutional neural network (CNN), such as VoV-Net, ResNet, etc. These models can capture low-level features such as edges and textures in the image as well as more complex high-level semantic information through multi-layer convolution and pooling operations. In order to better handle features of different scales, the feature pyramid network (FPN) is also often used in this module to extract and fuse multi-scale features at different levels;
[0042] The image perspective to bird's-eye view conversion module converts the image features of perspective vision into a bird's-eye view representation to adapt to the changes in different sensor layouts and vehicle models. The purpose of this module is to convert the image of perspective vision into a bird's-eye view to facilitate subsequent spatial analysis and obstacle detection. This module is crucial for accurately understanding the relative position and distance in the scene;
[0043] The bird's-eye view image temporal feature fusion module fuses the temporal features of the bird's-eye view image to better understand the dynamic environment. Temporal sequence information can enhance obstacle perception. This module effectively extracts and fuses temporal sequence information, allowing the model to better understand the dynamic changes of moving objects.
[0044] The detection head module is the final part used for obstacle detection and classification. It analyzes the features extracted and fused by the previous modules and outputs specific detection results. This module includes a classifier and a regressor to determine the category and location of the perceived target.
[0045] In step S102, each common functional module is arranged and combined to obtain a plurality of combined perception models.
[0046] In some embodiments, each common functional module is arranged and combined to obtain a plurality of combined perception models, including:
[0047] Decompose each common functional module into multiple functional sub-modules;
[0048] Arrange and combine multiple functional sub-modules of each common functional module to obtain multiple combined perception models.
[0049] In the actual implementation process, based on the above functional module decomposition scheme, each functional module can develop multiple replaceable common functional sub-modules, such as Figure 3 As shown, taking the obstacle sensing module as an example, the details are as follows: the image feature extraction module includes the residual neural network ResNet50 and the dense neural network VoV-Net, the image perspective vision to bird's-eye view visual perspective conversion module includes the spatial cross attention module (Spatial Cross Attention, SCA) and the geometry-guided kernel attention module (Geometry-guided Kernel Transformer, GKT), the bird's-eye view image temporal feature fusion module includes the temporal self-attention module (Temporal Self-Attention, TSA) and the recurrent temporal fusion module (Recurrent Temporal Fusion, RTF), and the detection head module includes the obstacle detection head module (3D Object Detection Head, 3D OBH). Based on the above example module composition scheme, a total of 2×2×2×1=8 model structures can be combined, that is, a customized intelligent driving perception algorithm can be flexibly constructed.
[0050] In step S103, multiple combined perception models are iteratively trained according to the target model until a preset maximum number of training rounds is reached, so as to obtain the final multiple combined perception model and the weight of each functional module thereof.
[0051] In some embodiments, multiple combined perception models are iteratively trained according to the target model until a preset maximum number of training rounds is reached to obtain the final multiple combined perception models and the weight of each functional module thereof, including:
[0052] Perform short-cycle training on each combined perception model to obtain the weight of each combined perception model after training;
[0053] Selecting a target functional module from multiple combined perception models according to the target model;
[0054] Based on the weight of each combined perception model after training, the weights of the target functional modules in multiple combined perception models are averaged to obtain a weight mean;
[0055] Copying the weight mean to the target function modules in the multiple combined perception models to form new multiple combined perception models;
[0056] The aforementioned training process is iterated until the preset maximum number of training rounds is reached, and the final multi-combination perception model and the weight of each functional module thereof are obtained.
[0057] In the actual implementation process, the core idea of the multi-model combination iterative training framework of the embodiment of the present invention is to perform pre-training based on the public large-scale Nuscenes data set to exhaustively combine perception function modules to form a perception function module library, so that modules with different computing power and different video memory can support the same function. Figure 4 As shown, taking the obstacle perception model as an example, the combined training scheme designed in the embodiment of the present invention mainly adopts the enumeration combined training method to train multiple combined models one by one. After each round of training, the weights of the same module of each model are updated with parameter fusion, and then the training iterative optimization is continued until the preset maximum training round is reached. The specific steps are as follows:
[0058] (1) Independent training: Use the traditional end-to-end training mode to perform a short-term training of one mini-epoch (for example, the mini-epoch is set to 3) on each of the eight combined models to obtain the initial weights of the first round of eight models;
[0059] (2) Module matching: Select common target function modules from the eight model combinations based on the target model. The main calculation process is to traverse the weight path of each model combination, load the weight of each model, traverse the weight dictionary of each model combination, and select the target function module based on the key of the weight dictionary;
[0060] (3) Weight fusion update: Take the average of the weight values of the target function modules of different model combinations, copy the solved average to each target function module, and recombine the target function modules with new weights with the modules of the original combination to obtain new different model combinations.
[0061] The above three steps constitute a round of combined training, and the combined training is iteratively performed until the preset maximum training round is reached to obtain the final multi-combination perception model and the weight of each functional module thereof. Compared with the traditional end-to-end method of directly training N=24 epochs, the combined training framework adopted in the embodiment of the present invention only needs N / mini-epoch=24 / 3=8 times to iterate the above independent training, module selection, and weight fusion process to obtain the final functional module weight.
[0062] The following uses the large-scale public dataset Nuscenes as an example to compare the training method proposed in the embodiment of the present invention with the existing traditional training method to further illustrate the accuracy of the embodiment of the present invention.
[0063] Taking the 8 combination models composed of obstacle perception models as an example, it can be seen from Table 1 that the accuracy of each combination model trained by the training method proposed in the embodiment of the present invention is higher than the accuracy of each combination model trained by the traditional training method. Therefore, the model trained by the multi-module combination training and shared module parameter joint update proposed in the embodiment of the present invention has stronger generalization and robustness.
[0064]
[0065] In summary, the perception model combination training method based on the computing basic platform proposed in the embodiment of the present invention has the following beneficial effects:
[0066] (1) Based on the design and development concept of hierarchical decoupling, the perception model is divided into different functional modules, so that each module is responsible for a specific function. It supports users to select and combine different functional modules according to target requirements, and flexibly build customized intelligent driving perception algorithms, which reduces the complexity of the perception system, improves development efficiency and system performance, shortens the R&D cycle, and increases maintainability, portability and scalability. It can be oriented to the new vehicle-mounted intelligent computing platform architecture solution and the application needs in the next 3-5 years;
[0067] (2) The combined training method supports the separation and independent development of each functional module of the perception algorithm, and has highly flexible, reusable and reconfigurable characteristics;
[0068] (3) The combined training method subverts the existing end-to-end training model. The model trained by multi-module combined training and joint update of shared module parameters has stronger generalization and robustness.
[0069] Next, a perception model combination training device based on a computing basic platform proposed in accordance with an embodiment of the present invention will be described with reference to the accompanying drawings.
[0070] Figure 5 It is a block diagram of a perception model combination training device based on a computing basic platform according to an embodiment of the present invention.
[0071] like Figure 5 As shown, the perception model combination training device 50 based on the computing basic platform includes: a decomposition unit 501, a combination unit 502 and an iterative training unit 503.
[0072] Among them, the decomposition unit 501 is used to decompose the target perception model into multiple common functional modules based on the computing basic platform. The combination unit 502 is used to arrange and combine each common functional module to obtain multiple combined perception models. The iterative training unit 503 is used to iteratively train the multiple combined perception models according to the target model until the preset maximum training rounds are reached, and obtain the final multi-combination perception model and the weight of each functional module thereof.
[0073] In some embodiments, the plurality of common functional modules include an image feature extraction module, an image perspective vision to bird's-eye view vision perspective conversion module, a bird's-eye view image temporal feature fusion module and a detection head module.
[0074] In some embodiments, the combining unit 502 includes:
[0075] A disassembly sub-unit is used to disassemble each common functional module into multiple functional sub-modules;
[0076] The combination sub-unit is used to arrange and combine multiple functional sub-modules of each common functional module to obtain multiple combined perception models.
[0077] In some embodiments, the iterative training unit 503 includes:
[0078] A training subunit, used for performing short-cycle training on each combined perception model to obtain a weight of each combined perception model after training;
[0079] A selection subunit, used to select a target functional module from a plurality of combined perception models according to a target model;
[0080] An averaging subunit, used for averaging the weights of target functional modules in multiple combined perception models based on the trained weights of each combined perception model to obtain a weight mean;
[0081] A copy subunit, used for copying the weight mean to the target function modules in the multiple combined perception models to form multiple new combined perception models;
[0082] The iterative subunit is used to iteratively execute the aforementioned training process until a preset maximum number of training rounds is reached, thereby obtaining the final multi-combination perception model and the weight of each functional module thereof.
[0083] It should be noted that the aforementioned explanation of the embodiment of the perceptual model combination training method based on the computing base platform is also applicable to the perceptual model combination training device based on the computing base platform of this embodiment, and will not be repeated here.
[0084] The perception model combination training device based on the computing basic platform proposed in the embodiment of the present invention has the following beneficial effects:
[0085] (1) Based on the design and development concept of hierarchical decoupling, the perception model is divided into different functional modules, so that each module is responsible for a specific function. It supports users to select and combine different functional modules according to target requirements, and flexibly build customized intelligent driving perception algorithms, which reduces the complexity of the perception system, improves development efficiency and system performance, shortens the R&D cycle, and increases maintainability, portability and scalability. It can be oriented to the new vehicle-mounted intelligent computing platform architecture solution and the application needs in the next 3-5 years;
[0086] (2) The combined training method supports the separation and independent development of each functional module of the perception algorithm, and has highly flexible, reusable and reconfigurable characteristics;
[0087] (3) The combined training method subverts the existing end-to-end training model. The model trained by multi-module combined training and joint update of shared module parameters has stronger generalization and robustness.
[0088] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:
[0089] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .
[0090] When the processor 602 executes the program, the perception model combination training method based on the computing basic platform provided in the above embodiment is implemented.
[0091] Furthermore, the electronic device further comprises:
[0092] The communication interface 603 is used for communication between the memory 601 and the processor 602 .
[0093] The memory 601 is used to store computer programs that can be executed on the processor 602 .
[0094] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0095] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0096] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.
[0097] The processor 602 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.
[0098] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned perception model combination training method based on a computing basic platform.
[0099] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0100] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0101] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.
[0102] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.
[0103] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0104] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0105] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0106] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A perception model combination training method based on a computing basic platform, characterized in that: The following steps are involved: Based on the computing basic platform, the target perception model is decomposed into multiple common functional modules; Arrange and combine each common functional module to obtain multiple combined perception models; The multiple combined perception models are iteratively trained according to the target model until a preset maximum number of training rounds is reached, so as to obtain the final multiple combined perception model and the weight of each functional module thereof.
2. The perception model combination training method based on the computing basic platform according to claim 1 is characterized in that: The multiple common functional modules include an image feature extraction module, an image perspective vision to bird's-eye view vision perspective conversion module, a bird's-eye view vision image temporal feature fusion module and a detection head module.
3. The perception model combination training method based on the computing basic platform according to claim 1 is characterized in that: The common functional modules are arranged and combined to obtain multiple combined perception models, including: Decomposing each common functional module into a plurality of functional sub-modules; The multiple functional sub-modules of each common functional module are arranged and combined to obtain the multiple combined perception models.
4. The perception model combination training method based on the computing basic platform according to claim 1 is characterized in that: The iterative training of the multiple combined perception models according to the target model until a preset maximum number of training rounds is reached to obtain the final multiple combined perception model and the weight of each functional module thereof, including: Perform short-cycle training on each combined perception model to obtain the weight of each combined perception model after training; Selecting a target functional module from the multiple combined perception models according to the target model; Based on the weight of each combined perception model after the training, averaging the weights of the target functional modules in the multiple combined perception models to obtain a weight mean; Copying the weight mean to the target function modules in the multiple combined perception models to form multiple new combined perception models; The aforementioned training process is iteratively performed until the preset maximum number of training rounds is reached, thereby obtaining the final multi-combination perception model and the weight of each functional module thereof.
5. A perception model combination training device based on a computing basic platform, characterized in that: include: A decomposition unit, used to decompose the target perception model into multiple common functional modules based on the computing basic platform; A combination unit, used for arranging and combining each common functional module to obtain multiple combined perception models; The iterative training unit is used to iteratively train the multiple combined perception models according to the target model until a preset maximum number of training rounds is reached to obtain the final multiple combined perception model and the weight of each functional module thereof.
6. The perception model combination training device based on the computing basic platform according to claim 5 is characterized in that: The multiple common functional modules include an image feature extraction module, an image perspective vision to bird's-eye view vision perspective conversion module, a bird's-eye view vision image temporal feature fusion module and a detection head module.
7. The perception model combination training device based on the computing basic platform according to claim 5 is characterized in that: The combined unit comprises: A disassembly sub-unit, used for disassembling each common functional module into a plurality of functional sub-modules; The combining sub-unit is used to arrange and combine the multiple functional sub-modules of each common functional module to obtain the multiple combined perception models.
8. The perception model combination training device based on the computing basic platform according to claim 5 is characterized in that: The iterative training unit comprises: A training subunit, used for performing short-cycle training on each combined perception model to obtain a weight of each combined perception model after training; A selection subunit, used for selecting a target functional module from the multiple combined perception models according to the target model; An averaging subunit, configured to average the weights of the target functional modules in the plurality of combined perception models based on the weight of each combined perception model after training to obtain a weight mean; A copying subunit, used for copying the weight mean to the target function module in the multiple combined perception models to form a new multiple combined perception models; The iterative subunit is used to iteratively execute the aforementioned training process until the preset maximum training rounds are reached, thereby obtaining the final multi-combination perception model and the weight of each functional module thereof.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the perception model combination training method based on a computing basic platform as described in any one of claims 1 to 4.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the perception model combination training method based on a computing basic platform as described in any one of claims 1 to 4.
