Map processing method and device of vehicle and electronic equipment

Abstract

The invention provides a map processing method and device of a vehicle and electronic equipment. The method comprises the steps that a to-be-processed map is obtained, and the to-be-processed map is an original map collected by a vehicle; a to-be-processed map is analyzed through a lightweight generative adversarial neural network to obtain a high-quality map, the lightweight generative adversarial neural network is obtained through training of multiple groups of data, and each group of data in the multiple groups of data comprises a historical to-be-processed map and a high-quality map label corresponding to the historical to-be-processed map. The number of model parameters of the lightweight generative adversarial neural network is smaller than the number of model parameters of the standard generative adversarial neural network; and controlling vehicle driving according to the high-quality map. Through the application, the problems of large calculation amount and high complexity of a map processing method in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a vehicle map processing method, device, computer-readable storage medium and electronic device. Background Art

[0002] When a mobile robot performs SLAM (Simultaneous Localization and Mapping, SLAM for short), the mapping results obtained often have problems such as ghosting, map skew, and local drift, which can cause deviations when the robot performs downstream navigation tasks. To solve the above problems, traditional methods use algorithms such as loop detection, multi-sensor fusion, map matching and correction for optimization, but these methods often have problems such as high computational complexity, large computing resource requirements, and high real-time requirements.

[0003] Therefore, a computationally convenient map generation method is needed. Summary of the invention

[0004] The main purpose of the present application is to provide a vehicle map processing method, device, computer-readable storage medium and electronic device to at least solve the problem of large computational complexity and high complexity of map processing methods in the prior art.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a vehicle map processing method is provided, comprising: obtaining a map to be processed, wherein the map to be processed is an original map collected by the vehicle; analyzing the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple groups of data, and each group of the multiple groups of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, the number of model parameters of the lightweight generative adversarial neural network is less than the number of model parameters of the standard generative adversarial neural network, and the high-quality map is a map after at least clarity processing of the map to be processed; and controlling the driving of the vehicle according to the high-quality map.

[0006] Optionally, before analyzing the map to be processed through a lightweight generative adversarial neural network, the method also includes: establishing an initial lightweight generative adversarial neural network; training the initial lightweight generative adversarial neural network through the historical map to be processed and the high-quality map labels until the loss function of the initial lightweight generative adversarial neural network converges, thereby obtaining the lightweight generative adversarial neural network.

[0007] Optionally, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, and the map to be processed is analyzed by the lightweight generative adversarial neural network to obtain a high-quality map, including: generating a generated map of the map to be processed by the lightweight generator, wherein the generated map is a false map corresponding to the regenerated map to be processed; comparing the map to be processed and the generated map by the discriminator to obtain distinguishing features of the map to be processed and the generated map; calculating the loss function of the lightweight generative adversarial neural network, and optimizing parameters in the lightweight generator and parameters in the discriminator according to the loss function until the loss function converges to obtain the high-quality map.

[0008] Optionally, comparing the map to be processed and the generated map by the discriminator includes: marking targets in the map to be processed and the generated map, wherein the targets include at least obstacles; matching the map to be processed and the generated map containing the same targets into a group of map combinations, and comparing each group of the map combinations.

[0009] Optionally, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, the lightweight generator is an optimized U-Net structure and at least includes a separate convolution module, and the separate convolution module includes a convolution kernel layer, a regularization processing layer and an activation function layer.

[0010] Optionally, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, the discriminator is a PatchGAN structure, and the method further includes: outputting distinguishing features in multiple areas on the map to be processed, wherein the distinguishing features are the discrimination results of the discriminator.

[0011] Optionally, before analyzing the map to be processed by a lightweight generative adversarial neural network, the method further includes: dividing the map to be processed to obtain a gridded map to be processed; and normalizing the gridded map to be processed to obtain a normalized map to be processed.

[0012] According to another aspect of the present application, a map processing device for a vehicle is provided, comprising: an acquisition unit, used to acquire a map to be processed, wherein the map to be processed is an original map collected by the vehicle; an analysis unit, used to analyze the map to be processed by means of a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training with multiple sets of data, each set of the multiple sets of data comprises: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, wherein the high-quality map is a map after at least clarity processing has been performed on the map to be processed; and a control unit, used to control the driving of the vehicle according to the high-quality map.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the vehicle map processing methods.

[0014] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing a map processing method for any one of the vehicles described.

[0015] Apply the technical solution of the present application to obtain a map to be processed, which is the original map collected by the vehicle; analyze the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each of which includes: historical maps to be processed and high-quality map labels corresponding to the historical maps to be processed, and the number of model parameters of the lightweight generative adversarial neural network is less than the number of model parameters of the standard generative adversarial neural network; control the vehicle driving according to the high-quality map. Compared with the method in the prior art that uses the traditional generative adversarial neural network model with large size and many parameters, resulting in large amount of calculation and high complexity, the present application uses a lightweight generative adversarial neural network, i.e. a simplified network, to analyze and process the above-mentioned map to be processed, obtain a high-quality map, simplify the calculation process, and reduce the amount of calculation. Therefore, it can solve the problem of large amount of calculation and high complexity of the map processing method in the prior art, and achieve the effect of simplifying the model and reducing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A schematic flow chart of a vehicle map processing method provided by an embodiment of the present application is shown;

[0018] Figure 2 A schematic diagram of the structure of a Pix2Pix model provided in an embodiment of the present application is shown;

[0019] Figure 3 A schematic diagram of the overall structure of a lightweight generator provided in an embodiment of the present application is shown;

[0020] Figure 4 A schematic diagram of the structure of a traditional generator provided by an embodiment of the present application is shown;

[0021] Figure 5 A specific lightweight generative adversarial neural network training flow chart provided in an embodiment of the present application is shown;

[0022] Figure 6 A structural block diagram of a vehicle map processing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0027] SLAM: Simultaneous Localization and Mapping, abbreviated as SLAM, is a technology that allows robots or vehicles to navigate autonomously in unknown environments.

[0028] Generative adversarial neural network: a machine learning paradigm framework, the network is generally composed of a generator and a discriminator. The generator samples from the noise distribution and generates realistic data samples, while the discriminator is used to distinguish between real data and generated data. The two are trained together in an adversarial way to minimize the target loss function. However, the generator structure of the traditional generative adversarial neural network is complex, with many parameters and large amount of calculation.

[0029] As introduced in the background technology, the map processing method in the prior art has large computational complexity. To solve the problem of large computational complexity, the embodiments of the present application provide a vehicle map processing method, device, computer-readable storage medium and electronic device.

[0030] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0031] In this embodiment, a map processing method for a vehicle running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 FIG. 1 is a flow chart of a vehicle map processing method according to an embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0033] Step S201, obtaining a map to be processed, wherein the map to be processed is an original map collected by the vehicle;

[0034] Specifically, the map to be processed may be mapping data obtained by executing a SLAM (Simultaneous Localization and Mapping, SLAM for short) task, and the mapping data may be an occupancy grid map, or may be image data collected by other sensors.

[0035] Step S202 is used to analyze the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each of the multiple sets of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, and the high-quality map is a map after the map to be processed is at least processed for clarity;

[0036] Specifically, the model reduces the parameters of the convolution module in the generator to achieve lightweight overall model. A lightweight generative adversarial neural network model is proposed to optimize the occupancy grid map. Through the trained generator model, the problematic occupancy grid map is processed end-to-end. Compared with the traditional method, the number of model parameters is reduced, the computational complexity is reduced, and the consumption of computing resources is reduced. After the map to be processed is trained, a high-quality map is obtained. The newly obtained map to be processed often has problems such as ghosting, map skew, and local drift. After analysis and processing by the above model, the high-quality map obtained is a map after overcoming the above defects and can be directly used to guide driving.

[0037] Step S203: controlling the vehicle to travel according to the high-quality map.

[0038] Specifically, the obtained high-quality map is finally output to the vehicle's navigation system, and the navigation system controls the vehicle's driving according to the map.

[0039] Through this embodiment, a map to be processed is obtained, and the map to be processed is the original map collected by the vehicle; the map to be processed is analyzed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: historical maps to be processed and high-quality map labels corresponding to historical maps to be processed, and the number of model parameters of the lightweight generative adversarial neural network is less than the number of model parameters of the standard generative adversarial neural network; the vehicle is controlled to travel according to the high-quality map. Compared with the method in the prior art that uses a traditional generative adversarial neural network with a large model and many parameters, resulting in a large amount of calculation and high complexity, the present application uses a lightweight generative adversarial neural network, i.e., a simplified network, to analyze and process the above-mentioned map to be processed, obtain a high-quality map, simplify the calculation process, and reduce the amount of calculation. Therefore, it can solve the problem of large amount of calculation and high complexity of the map processing method in the prior art, and achieve the effect of simplifying the model and reducing the amount of calculation.

[0040] In the specific implementation process, before the above step S202 analyzes the map to be processed by the lightweight generative adversarial neural network, the method further includes the following steps: step S204: establishing an initial lightweight generative adversarial neural network; step S205: training the initial lightweight generative adversarial neural network by the historical map to be processed and the high-quality map label until the loss function of the initial lightweight generative adversarial neural network converges to obtain the lightweight generative adversarial neural network. The method trains the initial lightweight generative adversarial neural network through the above steps to obtain a lightweight generative adversarial neural network, so as to process the map to be processed by the lightweight generative adversarial neural network.

[0041] Specifically, the initial lightweight generative adversarial neural network can be a Pix2Pix model, including an encoder-decoder structure, which divides the historical maps to be processed and the high-quality map labels into a training set and a test set, and trains the above model. During the training process, the generator G and the discriminator D compete with each other, so that the generator is continuously improved, and the generated maps have fewer and fewer problems and higher quality. The generator and the discriminator are trained alternately. In each training iteration, the discriminator is updated first, and then the generator is updated to minimize the loss function, that is, the loss function of the initial lightweight generative adversarial neural network converges to obtain the lightweight generative adversarial neural network.

[0042] In some optional embodiments, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator. The above step S202 analyzes the map to be processed by the lightweight generative adversarial neural network to obtain a high-quality map, which can be achieved by the following steps: Step S2021: Generate a generated map of the map to be processed by the lightweight generator, wherein the generated map is a false map corresponding to the regenerated map to be processed; Step S2022: Compare the map to be processed and the generated map by the discriminator to obtain the distinguishing features of the map to be processed and the generated map; Step S2023: Calculate the loss function of the lightweight generative adversarial neural network, and optimize the parameters in the lightweight generator and the parameters in the discriminator according to the loss function until the loss function converges to obtain the high-quality map. The method trains the model and generates a high-quality map through the above steps, which can reduce the amount of calculation in the map processing process and generate a clear and accurate map.

[0043] In the specific implementation process, the traditional Pix2Pix model is a generator and discriminator structure. Figure 2The schematic diagram of the structure of a Pix2Pix model is shown. x represents the map to be processed. A fake generated map G(x) is generated by the generator G. G(x) is compared with x by the discriminator D. The discriminator distinguishes between fake images and real images. The real image is represented by y. The generator G and the discriminator D compete with each other and calculate their respective loss functions, so that the generator and the discriminator are continuously improved and optimized, and the generated map has fewer and fewer problems. The loss function gradually converges, and finally a high-quality map is obtained. Since the SLAM mapping results often have problems such as ghosting, map skew, and local drift, a high-quality map is clear map data that removes ghosting, skew, and drift.

[0044] In some optional implementations, step S2022 compares the map to be processed and the generated map through the discriminator, which can be achieved by the following steps: target annotation of the target in the map to be processed and the target in the generated map, wherein the target at least includes obstacles; matching the map to be processed and the generated map containing the same target into a group of map combinations, and comparing each group of the map combinations. The method annotates and matches the map to be processed and the generated map through the above steps, so that the maps corresponding to the same scene can be compared, which facilitates the processing of the map to be processed.

[0045] In the specific implementation process, the collected occupancy grid map data is annotated and paired with the corresponding generated map to facilitate comparison by the discriminator.

[0046] In other optional embodiments, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, the lightweight generator is an optimized U-Net structure and at least includes a separate convolution module, and the separate convolution module includes a convolution kernel layer, a regularization processing layer and an activation function layer.

[0047] Specifically, the overall structure of the lightweight generator is as follows: Figure 3 As shown in the figure, the method of separate convolution (S-Conv) is used. First, a 3×3 convolution kernel (D-Conv) is used to obtain the feature map under a channel, and then 64 1×1 convolution kernels are used to extract the features in the feature map. After each of the above convolution operations, the BN layer (BtachNormalization) is used for regularization, and finally the ReLU activation function is used for nonlinear processing. Through calculation, the number of parameters used in the convolution operation of the S-Conv module is: 1×3×3+1×64×1×1=73, which is the same as Figure 4Compared with the overall structure of the traditional generator, the number of parameters for the same convolution operation is 1×64×3×3=576. The number of parameters of the lightweight module proposed in this proposal is reduced by 7.89 times. When extended to the entire generator model, the number of model parameters is greatly reduced, the model is lightweight, and it is more suitable for deployment in sweeping robots. Figure 4 A schematic diagram of the structure of a traditional generator provided by an embodiment of the present application is shown. The generator used by the traditional Pix2Pix is ​​a U-Net architecture, which uses a jump connection method to directly pass the information of the encoder layer to the decoder layer. In the encoder, the input of the model is an occupancy grid map image with problems. Through 4 convolution and pooling modules of different scales, the size and number of channels of the image are gradually changed, and the image features at different scales are extracted. The encoder extracts the key feature information in the image by encoding the input image into a low-dimensional feature vector; in the decoder, corresponding to the encoder, there are also 4 deconvolution and upsampling modules of different scales. The decoder receives the output feature representations of the encoder at different scales, and corresponding to the deconvolution layers and upsampling layers of different scales, gradually restores the size and number of channels of the image, maps different feature representations to the pixel space of the image, and generates an output image corresponding to the input image.

[0048] In some optional implementations, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, the discriminator is a PatchGAN structure, and the method further includes the following steps: outputting distinguishing features in multiple areas on the map to be processed, wherein the distinguishing features are the discrimination results of the discriminator. The method outputs the distinguishing features through the above steps, so that the above distinguishing features can be displayed.

[0049] Specifically, the discriminator used in the above model is a PatchGAN structure, which can output discrimination results (distinguishing features) in multiple regions on the image respectively. Local discrimination can enable the discriminator to more finely distinguish the difference between the generated image (generated map) and the real image (map to be processed).

[0050] In some other optional implementations, before analyzing the map to be processed by a lightweight generative adversarial neural network, the method further includes the following steps: dividing the map to be processed to obtain a gridded map to be processed; and normalizing the gridded map to be processed to obtain a normalized map to be processed. The method performs preprocessing through the above steps, which can facilitate subsequent processing of the model.

[0051] In the specific implementation process, the map to be processed is preprocessed. If the map to be processed is occupied grid map data, the input occupied grid map data is unified to 572×572 size and then normalized. Then, the processed data is input into the above-mentioned lightweight Pix2Pix model for forward propagation calculation, and the loss functions of the generator and discriminator are defined, including adversarial loss and L1 loss; then, back propagation is performed to continuously optimize the parameters in the generator and discriminator until the training converges. In the process, the generator and discriminator are trained alternately. In each training iteration, the discriminator is updated first, and then the generator is updated to minimize the loss function.

[0052] In actual applications, other lightweight generative adversarial neural network models can be used to optimize occupancy grid maps. Compared with other models, the lightweight Pix2Pix model can be directly trained end-to-end, which is faster and makes the model simpler and more efficient to use. On the other hand, due to the stronger versatility of the Pix2Pix model, the lightweight model can adapt to different map optimization tasks.

[0053] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the vehicle map processing method of the present application will be described in detail below in conjunction with specific embodiments.

[0054] This embodiment relates to a specific lightweight generative adversarial neural network training flow chart, such as Figure 5 As shown, the following steps are included:

[0055] Step S1: slam map pairing dataset (problem map and high-quality map), i.e., historical maps to be processed and high-quality map labels corresponding to the historical maps to be processed;

[0056] Step S2: image preprocessing (size adjustment, normalization);

[0057] Step S3: lightweight Pix2Pix model forward propagation training;

[0058] Step S4: (calculate) loss function;

[0059] Step S5: determine whether the training has converged. If yes, execute step S6; if no, execute step S3;

[0060] Step S6: Obtain the generator model (lightweight generative adversarial neural network).

[0061] The embodiment of the present application also provides a vehicle map processing device. It should be noted that the vehicle map processing device of the embodiment of the present application can be used to execute the vehicle map processing method provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0062] The following is an introduction to the map processing device for a vehicle provided in an embodiment of the present application.

[0063] Figure 6 Schematic diagram of a vehicle map processing device according to an embodiment of the present application. Figure 6 As shown, the device comprises:

[0064] An acquisition unit 10 is used to acquire a map to be processed, wherein the map to be processed is an original map collected by the vehicle;

[0065] Specifically, the map to be processed may be mapping data obtained by executing a SLAM (Simultaneous Localization and Mapping, SLAM for short) task, and the mapping data may be an occupancy grid map, or may be image data collected by other sensors.

[0066] The analysis unit 20 is used to analyze the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each of the multiple sets of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, and the high-quality map is a map after the map to be processed is at least processed for clarity;

[0067] Specifically, a lightweight generative adversarial neural network model is proposed to optimize the occupancy grid map. Through the trained generator model, the problematic occupancy grid map is processed end-to-end. Compared with the traditional method, the number of model parameters is reduced, the computational complexity is reduced, and the consumption of computing resources is reduced. After the map to be processed is trained, a high-quality map is obtained.

[0068] The control unit 30 is used to control the vehicle to travel according to the high-quality map.

[0069] Specifically, the obtained high-quality map is finally output to the vehicle's navigation system, and the navigation system controls the vehicle's driving according to the map.

[0070] Through this embodiment, a map to be processed is obtained, and the map to be processed is the original map collected by the vehicle; the map to be processed is analyzed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: historical maps to be processed and high-quality map labels corresponding to historical maps to be processed, and the number of model parameters of the lightweight generative adversarial neural network is less than the number of model parameters of the standard generative adversarial neural network; the vehicle is controlled to travel according to the high-quality map. Compared with the method in the prior art that uses a traditional generative adversarial neural network with a large model and many parameters, resulting in a large amount of calculation and high complexity, the present application uses a lightweight generative adversarial neural network, i.e., a simplified network, to analyze and process the above-mentioned map to be processed, obtain a high-quality map, simplify the calculation process, and reduce the amount of calculation. Therefore, it can solve the problem of large amount of calculation and high complexity of the map processing method in the prior art, and achieve the effect of simplifying the model and reducing the amount of calculation.

[0071] In the specific implementation process, the device also includes an establishment unit and a training unit. The establishment unit is used to establish an initial lightweight generative adversarial neural network; the training unit is used to train the initial lightweight generative adversarial neural network through the historical map to be processed and the high-quality map label until the loss function of the initial lightweight generative adversarial neural network converges to obtain the lightweight generative adversarial neural network. The method trains the initial lightweight generative adversarial neural network through the above steps to obtain a lightweight generative adversarial neural network, so as to process the map to be processed through the lightweight generative adversarial neural network.

[0072] Specifically, the initial lightweight generative adversarial neural network can be a Pix2Pix model, including an encoder-decoder structure, which divides the historical maps to be processed and the high-quality map labels into a training set and a test set, and trains the above model. During the training process, the generator G and the discriminator D compete with each other, so that the generator is continuously improved, and the generated maps have fewer and fewer problems and higher quality. The generator and the discriminator are trained alternately. In each training iteration, the discriminator is updated first, and then the generator is updated to minimize the loss function, that is, the loss function of the initial lightweight generative adversarial neural network converges to obtain the lightweight generative adversarial neural network.

[0073] In some optional embodiments, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, and the above-mentioned analysis unit includes a generation module, a comparison module and an optimization module. The generation module is used to generate a generated map of the map to be processed through the lightweight generator, wherein the generated map is a false map corresponding to the regenerated map to be processed; the comparison module is used to compare the map to be processed and the generated map through the discriminator to obtain the distinguishing features of the map to be processed and the generated map; the optimization module is used to calculate the loss function of the lightweight generative adversarial neural network, and optimize the parameters in the lightweight generator and the parameters in the discriminator according to the loss function until the loss function converges to obtain the high-quality map. The method trains the model and generates a high-quality map through the above steps, which can reduce the amount of calculation in the map processing process and generate a clear and accurate map.

[0074] In the specific implementation process, the traditional Pix2Pix model is a generator and discriminator structure. Figure 2 The schematic diagram of the structure of a Pix2Pix model is shown. x represents the map to be processed. A fake generated map G(x) is generated by the generator G. G(x) is compared with x by the discriminator D. The discriminator distinguishes between fake images and real images. The real image is represented by y. The generator G and the discriminator D compete with each other and calculate their respective loss functions, so that the generator and the discriminator are continuously improved and optimized, and the generated map has fewer and fewer problems. The loss function gradually converges, and finally a high-quality map is obtained. Since the SLAM mapping results often have problems such as ghosting, map skew, and local drift, a high-quality map is clear map data that removes ghosting, skew, and drift.

[0075] In some optional implementations, the comparison module includes a labeling submodule and a comparison submodule, the labeling submodule is used to label the target in the map to be processed and the target in the generated map, wherein the target at least includes obstacles; the comparison submodule is used to match the map to be processed and the generated map containing the same target into a group of map combinations, and compare each group of the map combinations. The method labels and matches the map to be processed and the generated map through the above steps, so that the maps corresponding to the same scene can be compared, which is convenient for processing the map to be processed.

[0076] In the specific implementation process, the collected occupancy grid map data is annotated and paired with the corresponding generated map to facilitate comparison by the discriminator.

[0077] In other optional embodiments, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, the lightweight generator is an optimized U-Net structure and at least includes a separate convolution module, and the separate convolution module includes a convolution kernel layer, a regularization processing layer and an activation function layer.

[0078] Specifically, the overall structure of the lightweight generator is as follows: Figure 3 As shown in the figure, the method of separate convolution (S-Conv) is used. First, a 3×3 convolution kernel (D-Conv) is used to obtain the feature map under a channel, and then 64 1×1 convolution kernels are used to extract the features in the feature map. After each of the above convolution operations, the BN layer (BtachNormalization) is used for regularization, and finally the ReLU activation function is used for nonlinear processing. Through calculation, the number of parameters used in the convolution operation of the S-Conv module is: 1×3×3+1×64×1×1=73, which is the same as Figure 4 Compared with the overall structure of the traditional generator, the number of parameters for the same convolution operation is 1×64×3×3=576. The number of parameters of the lightweight module proposed in this proposal is reduced by 7.89 times. When extended to the entire generator model, the number of model parameters is greatly reduced, the model is lightweight, and it is more suitable for deployment in sweeping robots. Figure 4 The generator used in the traditional Pix2Pix is ​​the U-Net architecture, which uses a jump connection to pass the information of the encoder layer directly to the decoder layer. In the encoder, the input of the model is an occupancy grid map image with problems. Through 4 convolution and pooling modules of different scales, the size and number of channels of the image are gradually changed, and the image features at different scales are extracted. The encoder extracts the key feature information in the image by encoding the input image into a low-dimensional feature vector; in the decoder, corresponding to the encoder, there are also 4 deconvolution and upsampling modules of different scales. The decoder receives the output feature representations of the encoder at different scales, and corresponding to the deconvolution layers and upsampling layers of different scales, gradually restores the size and number of channels of the image, maps different feature representations to the pixel space of the image, and generates an output image corresponding to the input image.

[0079] In some optional embodiments, the lightweight generative adversarial neural network includes a lightweight generator and a discriminator, the discriminator is a PatchGAN structure, and the device further includes an output unit for outputting distinguishing features for multiple areas on the map to be processed, wherein the distinguishing features are the discrimination results of the discriminator. The method outputs the distinguishing features through the above steps, so that the above distinguishing features can be displayed.

[0080] Specifically, the discriminator used in the above model is a PatchGAN structure, which can output discrimination results (distinguishing features) in multiple regions on the image respectively. Local discrimination can enable the discriminator to more finely distinguish the difference between the generated image (generated map) and the real image (map to be processed).

[0081] In some other optional implementations, the device further includes a division unit and a normalization processing unit, the division unit is used to divide the map to be processed to obtain a gridded map to be processed; the normalization processing unit is used to normalize the gridded map to be processed to obtain a normalized map to be processed. The method performs preprocessing through the above steps, which can facilitate subsequent processing of the model.

[0082] In the specific implementation process, the map to be processed is preprocessed. If the map to be processed is occupied grid map data, the input occupied grid map data is unified to 572×572 size and then normalized. Then, the processed data is input into the above-mentioned lightweight Pix2Pix model for forward propagation calculation, and the loss functions of the generator and discriminator are defined, including adversarial loss and L1 loss; then, back propagation is performed to continuously optimize the parameters in the generator and discriminator until the training converges. In the process, the generator and discriminator are trained alternately. In each training iteration, the discriminator is updated first, and then the generator is updated to minimize the loss function.

[0083] In actual applications, other lightweight generative adversarial neural network models can be used to optimize occupancy grid maps. Compared with other models, the lightweight Pix2Pix model can be directly trained end-to-end, which is faster and makes the model simpler and more efficient to use. On the other hand, due to the stronger versatility of the Pix2Pix model, the lightweight model can adapt to different map optimization tasks.

[0084] The map processing device of the vehicle includes a processor and a memory, wherein the acquisition unit, analysis unit and control unit are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0085] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the map processing method can be simplified and the amount of calculation can be reduced by adjusting the kernel parameters.

[0086] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0087] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the vehicle map processing method.

[0088] Specifically, the map processing method of the vehicle includes:

[0089] Step S201, obtaining a map to be processed, wherein the map to be processed is an original map collected by the vehicle;

[0090] Specifically, the map to be processed may be mapping data obtained by executing a SLAM (Simultaneous Localization and Mapping, SLAM for short) task, and the mapping data may be an occupancy grid map, or may be image data collected by other sensors.

[0091] Step S202 is used to analyze the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each of the multiple sets of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, and the high-quality map is a map after the map to be processed is at least processed for clarity;

[0092] Specifically, the model reduces the parameters of the convolution module in the generator to achieve lightweight overall model. A lightweight generative adversarial neural network model is proposed to optimize the occupancy grid map. Through the trained generator model, the problematic occupancy grid map is processed end-to-end. Compared with the traditional method, the number of model parameters is reduced, the computational complexity is reduced, and the consumption of computing resources is reduced. After the map to be processed is trained, a high-quality map is obtained. The newly obtained map to be processed often has problems such as ghosting, map skew, and local drift. After analysis and processing by the above model, the high-quality map obtained is a map after overcoming the above defects and can be directly used to guide driving.

[0093] Step S203: controlling the vehicle to travel according to the high-quality map.

[0094] Specifically, the obtained high-quality map is finally output to the vehicle's navigation system, and the navigation system controls the vehicle's driving according to the map.

[0095] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, at least the following steps are implemented:

[0096] Step S201, obtaining a map to be processed, wherein the map to be processed is an original map collected by the vehicle;

[0097] Specifically, the map to be processed may be mapping data obtained by executing a SLAM (Simultaneous Localization and Mapping, SLAM for short) task, and the mapping data may be an occupancy grid map, or may be image data collected by other sensors.

[0098] Step S202 is used to analyze the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each of the multiple sets of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, and the high-quality map is a map after the map to be processed is at least processed for clarity;

[0099] Specifically, the model reduces the parameters of the convolution module in the generator to achieve lightweight overall model. A lightweight generative adversarial neural network model is proposed to optimize the occupancy grid map. Through the trained generator model, the problematic occupancy grid map is processed end-to-end. Compared with the traditional method, the number of model parameters is reduced, the computational complexity is reduced, and the consumption of computing resources is reduced. After the map to be processed is trained, a high-quality map is obtained. The newly obtained map to be processed often has problems such as ghosting, map skew, and local drift. After analysis and processing by the above model, the high-quality map obtained is a map after overcoming the above defects and can be directly used to guide driving.

[0100] Step S203: controlling the vehicle to travel according to the high-quality map.

[0101] Specifically, the obtained high-quality map is finally output to the vehicle's navigation system, and the navigation system controls the vehicle's driving according to the map.

[0102] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0103] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in each embodiment of the present application:

[0104] Step S201, obtaining a map to be processed, wherein the map to be processed is an original map collected by the vehicle;

[0105] Specifically, the map to be processed may be mapping data obtained by executing a SLAM (Simultaneous Localization and Mapping, SLAM for short) task, and the mapping data may be an occupancy grid map, or may be image data collected by other sensors.

[0106] Step S202 is used to analyze the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each of the multiple sets of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, and the high-quality map is a map after the map to be processed is at least processed for clarity;

[0107] Specifically, the model reduces the parameters of the convolution module in the generator to achieve lightweight overall model. A lightweight generative adversarial neural network model is proposed to optimize the occupancy grid map. Through the trained generator model, the problematic occupancy grid map is processed end-to-end. Compared with the traditional method, the number of model parameters is reduced, the computational complexity is reduced, and the consumption of computing resources is reduced. After the map to be processed is trained, a high-quality map is obtained. The newly obtained map to be processed often has problems such as ghosting, map skew, and local drift. After analysis and processing by the above model, the high-quality map obtained is a map after overcoming the above defects and can be directly used to guide driving.

[0108] Step S203: controlling the vehicle to travel according to the high-quality map.

[0109] Specifically, the obtained high-quality map is finally output to the vehicle's navigation system, and the navigation system controls the vehicle's driving according to the map.

[0110] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0111] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0116] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0117] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0119] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0120] 1) In the map processing method of the vehicle of the present application, a map to be processed is obtained, and the map to be processed is the original map collected by the vehicle; the map to be processed is analyzed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: historical maps to be processed and high-quality map labels corresponding to the historical maps to be processed, and the number of model parameters of the lightweight generative adversarial neural network is less than the number of model parameters of the standard generative adversarial neural network; the vehicle is controlled to travel according to the high-quality map. Compared with the method in the prior art that uses the traditional generative adversarial neural network model with large size and many parameters, resulting in large amount of calculation and high complexity, the present application uses a lightweight generative adversarial neural network, i.e. a simplified network, to analyze and process the above-mentioned map to be processed, obtain a high-quality map, simplify the calculation process, and reduce the amount of calculation. Therefore, it can solve the problem of large amount of calculation and high complexity of the map processing method in the prior art, and achieve the effect of simplifying the model and reducing the amount of calculation.

[0121] 2) In the map processing device of the vehicle of the present application, a map to be processed is obtained, and the map to be processed is the original map collected by the vehicle; the map to be processed is analyzed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: historical maps to be processed and high-quality map labels corresponding to the historical maps to be processed, and the number of model parameters of the lightweight generative adversarial neural network is less than the number of model parameters of the standard generative adversarial neural network; the vehicle is controlled to travel according to the high-quality map. Compared with the prior art, which uses a traditional generative adversarial neural network with a large model and many parameters, resulting in a large amount of calculation and high complexity, the present application uses a lightweight generative adversarial neural network, i.e., a simplified network, to analyze and process the above-mentioned map to be processed to obtain a high-quality map, simplify the calculation process, and reduce the amount of calculation. Therefore, it can solve the problem of large amount of calculation and high complexity of the map processing device in the prior art, and achieve the effect of simplifying the model and reducing the amount of calculation.

[0122] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle map processing method, characterized in that: include: Obtaining a map to be processed, wherein the map to be processed is an original map collected by the vehicle; The map to be processed is analyzed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each set of data in the multiple sets of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, the number of model parameters of the lightweight generative adversarial neural network is less than the number of model parameters of the standard generative adversarial neural network, and the high-quality map is a map after the map to be processed is at least processed for clarity; The vehicle is controlled to travel according to the high-quality map.

2. The map processing method according to claim 1, characterized in that: Before analyzing the map to be processed by using a lightweight generative adversarial neural network, the method further includes: Establish an initial lightweight generative adversarial neural network; The initial lightweight generative adversarial neural network is trained using the historical map to be processed and the high-quality map labels until the loss function of the initial lightweight generative adversarial neural network converges, thereby obtaining the lightweight generative adversarial neural network.

3. The map processing method according to claim 1, characterized in that: The lightweight generative adversarial neural network includes a lightweight generator and a discriminator. The map to be processed is analyzed by the lightweight generative adversarial neural network to obtain a high-quality map, including: Generating a generated map of the map to be processed by the lightweight generator, wherein the generated map is a false map corresponding to the regenerated map to be processed; By using the discriminator, the map to be processed and the generated map are compared to obtain distinguishing features between the map to be processed and the generated map; The loss function of the lightweight generative adversarial neural network is calculated, and the parameters in the lightweight generator and the parameters in the discriminator are optimized according to the loss function until the loss function converges to obtain the high-quality map.

4. The map processing method according to claim 3, characterized in that: Comparing the map to be processed and the generated map by the discriminator includes: Marking targets in the map to be processed and targets in the generated map, wherein the targets at least include obstacles; The to-be-processed map and the generated map containing the same target are matched into a group of map combinations, and each group of the map combinations is compared.

5. The map processing method according to claim 1, characterized in that: The lightweight generative adversarial neural network includes a lightweight generator and a discriminator. The lightweight generator is an optimized U-Net structure and at least includes a separation convolution module. The separation convolution module includes a convolution kernel layer, a regularization processing layer and an activation function layer.

6. The map processing method according to claim 1, characterized in that: The lightweight generative adversarial neural network includes a lightweight generator and a discriminator, the discriminator is a PatchGAN structure, and the method further includes: Outputting distinguishing features in a plurality of areas on the map to be processed, wherein the distinguishing features are the distinguishing results of the distinguisher.

7. The map processing method according to claim 1, characterized in that: Before analyzing the map to be processed by using a lightweight generative adversarial neural network, the method further includes: Dividing the map to be processed to obtain a gridded map to be processed; The gridded map to be processed is normalized to obtain a normalized map to be processed.

8. A vehicle map processing device, characterized in that: include: An acquisition unit, used for acquiring a map to be processed, wherein the map to be processed is an original map collected by the vehicle; an analysis unit, configured to analyze the map to be processed by a lightweight generative adversarial neural network to obtain a high-quality map, wherein the lightweight generative adversarial neural network is obtained by training multiple sets of data, each set of the multiple sets of data includes: a historical map to be processed and a high-quality map label corresponding to the historical map to be processed, and the high-quality map is a map obtained by at least performing clarity processing on the map to be processed; A control unit is used to control the driving of the vehicle according to the high-quality map.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the vehicle map processing method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a map processing method for executing the vehicle described in any one of claims 1 to 7.

Images