Vehicle-mounted imaging road information reconstruction method based on deep learning and application thereof

Abstract

The invention provides a vehicle-mounted imaging road information reconstruction method based on deep learning and application thereof, and the method comprises the steps: carrying out the preprocessing of historical millimeter wave vehicle-mounted imaging data, and obtaining multi-channel data; constructing an initial deep learning model, and performing feature extraction on the multichannel data by using the initial deep learning model to obtain a feature sequence; performing multi-scale fusion and GRU time fusion based on the feature sequence, and training to obtain a trained deep learning model; and identifying the real-time millimeter wave vehicle-mounted imaging data by using the trained deep learning model to generate a road information reconstruction result. A deep learning model is adopted, multi-scale fusion and GRU time fusion are carried out, multi-dimensional information in millimeter wave vehicle-mounted imaging data can be fully utilized, high-precision reconstruction of road information is achieved, and the real-time processing capacity in a vehicle-mounted environment is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a vehicle-mounted imaging road information reconstruction method based on deep learning and its application. Background Art

[0002] With the development of autonomous driving technology, accurate reconstruction of road information is crucial for safe driving of vehicles. Traditional visual perception methods, such as cameras and lidar, are limited by factors such as lighting conditions and weather conditions and may not be able to provide reliable information in complex environments. Millimeter-wave radar has become an important perception method due to its insensitivity to bad weather and lighting changes, and contains rich road information. However, how to efficiently process millimeter-wave high-resolution vehicle-mounted imaging data to accurately extract road information is still a problem that needs to be solved.

[0003] Related methods mainly focus on using traditional image processing and machine learning algorithms, which cannot fully utilize the rich information contained in millimeter wave data. In addition, the real-time performance of data processing is poor and cannot meet the needs of vehicle-mounted environments.

[0004] Therefore, there is an urgent need for a deep learning-based vehicle-mounted imaging road information reconstruction method and its application to improve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a vehicle-mounted imaging road information reconstruction method based on deep learning and its application, which can realize real-time and high-precision reconstruction of road information.

[0006] In a first aspect, the present invention provides a method for reconstructing vehicle-mounted imaging road information based on deep learning, comprising: Preprocess the historical millimeter-wave vehicle-mounted imaging data to obtain multi-channel data; Constructing an initial deep learning model, and using the initial deep learning model to perform feature extraction on the multi-channel data to obtain a feature sequence; Based on the feature sequence, multi-scale fusion and GRU time fusion are performed, and a trained deep learning model is obtained by training; The trained deep learning model is used to identify real-time millimeter-wave vehicle-mounted imaging data to generate road information reconstruction results.

[0007] The method of the present invention has the following beneficial effects: by preprocessing historical millimeter wave vehicle-mounted imaging data, multi-channel data is obtained; an initial deep learning model is constructed, and the initial deep learning model is used to extract features from the multi-channel data to obtain a feature sequence; multi-scale fusion and GRU time fusion are performed based on the feature sequence, and a trained deep learning model is obtained; the trained deep learning model is used to identify real-time millimeter wave vehicle-mounted imaging data to generate road information reconstruction results. The use of a deep learning model and multi-scale fusion and GRU time fusion can fully utilize the multi-dimensional information in millimeter wave vehicle-mounted imaging data, realize high-precision reconstruction of road information, and improve real-time processing capabilities in a vehicle-mounted environment.

[0008] Optionally, the deep learning model is DeepLabV3+ and GRU model; and / or the road information reconstruction result includes the recognition of lane lines, road edges and traffic signs. The beneficial effect is that by using DeepLabV3+ and GRU model, the multi-dimensional information of each pixel in the millimeter wave vehicle imaging data can be further fully utilized to achieve high-precision reconstruction of road information.

[0009] Optionally, the preprocessing of the historical millimeter wave vehicle-mounted imaging data also includes: The millimeter wave radar is used to collect historical millimeter wave vehicle imaging data to obtain the position, height, speed and intensity information of each pixel. The beneficial effect is that it is convenient to fully utilize the position, height, speed and intensity information of each pixel as multi-channel data input into the deep learning model in the subsequent preprocessing process.

[0010] Optionally, preprocess the historical millimeter wave vehicle-mounted imaging data to obtain multi-channel data including: The position, height, speed and intensity information of each pixel in the historical millimeter-wave vehicle-mounted imaging data are normalized and encoded to obtain a multi-channel data matrix. The beneficial effect is that the generalization ability of the model is improved by normalizing and encoding the position, height, speed and intensity information of each pixel.

[0011] Optionally, after building the initial deep learning model, the following steps are also included: The number of input channels of the initial deep learning model is adjusted according to the dimensional configuration of the multi-channel data matrix. The beneficial effect is that by adjusting the initial deep learning model, it can adapt to the input of multi-dimensional data.

[0012] Optionally, performing multi-scale fusion and GRU time fusion based on the feature sequence includes: Use dilated convolution to capture spatial features of different scales and fuse multi-dimensional information to obtain feature sequences; The feature sequence is used to capture dynamic changes and correlations in the time series. The beneficial effect is that by using the dilated convolution to capture spatial features of different scales and fuse multi-dimensional information, the recognition ability of road edges and structures can be improved, and the GRU time fusion is used to capture dynamic changes and correlations in the time series, which enhances the recognition of dynamic road information.

[0013] Optionally, building an initial deep learning model includes: Construct a deep learning network architecture including a preprocessing layer, a DeepLabV3+ feature extraction layer, a GRU time modeling layer, and an output layer, configure the loss function and optimizer, and set the learning rate and training rounds. The beneficial effect is that by constructing a deep learning network architecture including a preprocessing layer, a DeepLabV3+ feature extraction layer, a GRU time modeling layer, and an output layer, configuring the loss function and optimizer, and setting the learning rate and training rounds, the accuracy and real-time performance of the reconstruction of road information can be further improved.

[0014] Optionally, the learning rate adopts an adaptive learning rate strategy; and / or the optimizer is an Adam optimizer; and / or the loss function adopts a cross entropy loss function; and / or the training rounds are stopped early according to the performance of the validation set. The beneficial effect is that by setting an adaptive learning rate strategy, an Adam optimizer, a cross entropy loss function, and setting the training rounds to stop early according to the performance of the validation set, the accuracy and real-time performance of the reconstruction of the road information can be further improved.

[0015] In a second aspect, the present invention provides a deep learning-based on-vehicle imaging road information reconstruction device, which includes a module / unit that executes any possible design method of the first aspect. These modules / units can be implemented by hardware, or by hardware executing corresponding software implementations.

[0016] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a program that can be run on the processor, and when the program is executed by the processor, the electronic device implements a method for executing any possible design of any of the above aspects.

[0017] In a fourth aspect, the present invention provides a readable storage medium, wherein the readable storage medium stores a program, and when the program is executed, it implements any possible design method of any of the above aspects.

[0018] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0019] For the beneficial effects of the second to fifth aspects, reference may be made to the description of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic flow chart of a method for reconstructing vehicle-mounted imaging road information based on deep learning provided in an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a vehicle-mounted imaging road information reconstruction device based on deep learning provided by an embodiment of the present invention; Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0022] In view of the current inability to fully utilize the rich information contained in millimeter wave data and the inconvenience of real-time data processing, the vehicle-mounted imaging road information reconstruction method based on deep learning provided by the present invention can make full use of the information in millimeter wave data to realize real-time high-precision reconstruction of road information. The technical scheme in the embodiment of the present invention is described below in conjunction with the drawings in the embodiment of the present invention. Among them, in the description of the embodiment of the present invention, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present invention. The singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present invention, "at least one", "one or more" refers to one or more than two (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0023] References to "one embodiment" or "some embodiments" described in this specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", and "in some other embodiments" that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connected" includes direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0024] In the embodiments of the present invention, "exemplarily" or "for example" is used to indicate an example, illustration or description. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of "exemplarily" or "for example" is intended to present the relevant concepts in a specific way.

[0025] like Figure 1 As shown, the present invention provides a method for reconstructing vehicle-mounted imaging road information based on deep learning, comprising: S101, preprocessing historical millimeter-wave vehicle-mounted imaging data to obtain multi-channel data.

[0026] In some embodiments, in order to facilitate the full use of the position, height, speed and intensity information of each pixel as a multi-channel data input deep learning model in the subsequent preprocessing process, before preprocessing the historical millimeter-wave vehicle-mounted imaging data, it also includes: using a millimeter-wave radar to collect historical millimeter-wave vehicle-mounted imaging data to obtain the position, height, speed and intensity information of each pixel.

[0027] In some specific embodiments, in order to improve the generalization ability of the model, the historical millimeter-wave vehicle-mounted imaging data is preprocessed to obtain multi-channel data, including: normalizing and encoding the position, height, speed and intensity information of each pixel in the historical millimeter-wave vehicle-mounted imaging data to obtain a multi-channel data matrix.

[0028] S102, constructing an initial deep learning model, and using the initial deep learning model to extract features from the multi-channel data to obtain a feature sequence.

[0029] In some embodiments, in order to further make full use of the multi-dimensional information of each pixel in the millimeter wave vehicle-mounted imaging data to achieve high-precision reconstruction of road information, the deep learning model is DeepLabV3+ and GRU model; and / or the road information reconstruction result includes the recognition of lane lines, road edges and traffic signs. It is worth noting that GUR time fusion is used as a layer in the deep learning model, and the features are enhanced by inserting the GUR model in the feature extraction stage of DeepLab, thereby achieving high-precision reconstruction of road information.

[0030] In some specific embodiments, in order to adapt to the input of multi-dimensional data, after constructing the initial deep learning model, it also includes: adjusting the number of input channels of the initial deep learning model according to the dimensional configuration of the multi-channel data matrix.

[0031] S103, performing multi-scale fusion and GRU time fusion based on the feature sequence, and training to obtain a trained deep learning model.

[0032] In some embodiments, in order to improve the recognition ability of road edges and structures and enhance the recognition of dynamic road information, multi-scale fusion and GRU time fusion are performed based on the feature sequence, including: using atrous convolution to capture spatial features of different scales, fusing multi-dimensional information to obtain a feature sequence; using the feature sequence to capture dynamic changes and correlations in the time series.

[0033] In some embodiments, in order to further improve the accuracy and real-time performance of the reconstruction of road information, constructing an initial deep learning model includes: constructing a deep learning network architecture including a preprocessing layer, a DeepLabV3+ feature extraction layer, a GRU time modeling layer and an output layer, configuring a loss function and an optimizer, and setting a learning rate and training rounds.

[0034] In some specific embodiments, in order to further improve the accuracy and real-time performance of the reconstruction of road information, the learning rate adopts an adaptive learning rate strategy; and / or the optimizer is an Adam optimizer; and / or the loss function adopts a cross entropy loss function; and / or the training rounds are early stopped according to the performance of the validation set. Exemplarily, the learning rate is preferably set to 0.001 and the training rounds are 100 rounds.

[0035] S104: Using the trained deep learning model to identify real-time millimeter-wave vehicle-mounted imaging data, and generating a road information reconstruction result.

[0036] Exemplary: 1. Data preprocessing and feature fusion The millimeter-wave high-resolution vehicle-mounted imaging data is preprocessed to encode the position, height, speed, intensity and other information of each pixel into a multi-channel input acceptable to the model.

[0037] Use techniques such as normalization and data augmentation to improve the generalization ability of the model.

[0038] Step a, DeepLabV3+ feature extraction Adjust the number of input channels of the DeepLabV3+ model to adapt to the input of multi-dimensional data.

[0039] Atrous Convolution is used to capture spatial features of different scales, fuse multi-dimensional information, and improve the ability to recognize road edges and structures.

[0040] 2. GRU time series modeling The feature sequence extracted by DeepLabV3+ is input into the GRU network to capture the dynamic changes and correlations in the time series.

[0041] The advantages of GRU in processing sequence data and temporal dependencies are utilized to enhance the recognition of dynamic road information, such as moving obstacles and vehicles.

[0042] 3. Network structure and training Build a deep learning network architecture consisting of a preprocessing layer, a DeepLabV3+ feature extraction layer, a GRU temporal modeling layer, and an output layer.

[0043] Select appropriate loss function and optimizer, set parameters such as learning rate and training rounds, and perform model training.

[0044] 4. Road information reconstruction and output The output of the model is post-processed to obtain high-precision road information reconstruction results, including lane lines, road edges, and traffic signs.

[0045] The output results can be used in the decision-making and control modules of the autonomous driving system.

[0046] The advantages of the embodiments of the present invention are that the multi-dimensional information of millimeter wave data is used to improve the accuracy of road information reconstruction, especially in complex environments, and it is insensitive to severe weather and lighting conditions, thereby enhancing the reliability and applicability of the system. By optimizing the network structure and model parameters, real-time processing capabilities in a vehicle-mounted environment are achieved.

[0047] For ease of understanding, this embodiment further describes the specific implementation process of the above method in combination with a specific application scenario system, which specifically includes the following steps: Step a: Data preprocessing Data collection: Use millimeter-wave high-resolution radar to collect vehicle-mounted imaging data and obtain information such as the position (x, y, z), height, speed, and intensity of each pixel.

[0048] Data encoding: Encode the above information by channel to form a multi-channel input data matrix.

[0049] Data enhancement: Use methods such as rotation, translation, and scaling to enhance data and expand the data set.

[0050] Step b, model training Model configuration: The input layer accepts multi-channel mmWave data.

[0051] DeepLabV3+ partially adjusts the convolution kernel and number of channels to adapt to the input data.

[0052] The GRU layer sets the number of hidden units and processes the feature sequence.

[0053] Training parameters: Learning rate: set to 0.001, using an adaptive learning rate strategy.

[0054] Optimizer: Use Adam optimizer.

[0055] Loss function: cross entropy loss function is used.

[0056] Training rounds: set to 100 rounds, with early stopping based on the performance of the validation set.

[0057] Step c, result verification Evaluation indicators: Use indicators such as accuracy, recall, and F1 score to evaluate model performance.

[0058] Comparative experiment: Compare with the method using only traditional visual data to verify the advantages of the method of the present invention.

[0059] Practical application: Deploy the model in the vehicle system to conduct real-time road information reconstruction tests to verify its practicality.

[0060] The advantages of the embodiments of the present invention are high recognition accuracy, strong robustness and high real-time performance.

[0061] like Figure 2As shown, based on the above-mentioned on-board imaging road information reconstruction method based on deep learning, the present invention provides a on-board imaging road information reconstruction device based on deep learning, including: a preprocessing unit 201, used to preprocess historical millimeter-wave on-board imaging data to obtain multi-channel data; a construction unit 202, used to construct an initial deep learning model, and use the initial deep learning model to extract features from the multi-channel data to obtain a feature sequence; a fusion unit 203, used to perform multi-scale fusion and GRU time fusion based on the feature sequence, and train to obtain a trained deep learning model; an identification unit 204, used to use the trained deep learning model to identify real-time millimeter-wave on-board imaging data to generate a road information reconstruction result. Exemplarily, the preprocessing unit 201 is used to normalize and encode the position, height, speed and intensity information of each pixel in the historical millimeter-wave vehicle-mounted imaging data to obtain a multi-channel data matrix; the construction unit 202 is used to construct an initial DeepLabV3+ and GRU model of a deep learning network architecture including a preprocessing layer, a DeepLabV3+ feature extraction layer, a GRU time modeling layer and an output layer, and use the initial DeepLabV3+ and GRU models to use atrous convolution (Atrous convolution) Convolution) captures spatial features of different scales, fuses multi-dimensional information, and obtains a feature sequence; a fusion unit 203 is used to capture spatial features of different scales by using hole convolution, and fuses multi-dimensional information to obtain a feature sequence; the feature sequence is used to capture dynamic changes and correlations in the time series, and the loss function and optimizer are configured, the learning rate and training rounds are set, and the model is trained to obtain a trained deep learning model; the recognition unit 204 is used to use the trained deep learning model to recognize real-time millimeter-wave vehicle-mounted imaging data, and generate a road information reconstruction result, wherein the road information reconstruction result includes the recognition of lane lines, road edges and traffic signs, and the learning rate adopts an adaptive learning rate strategy; the optimizer is an Adam optimizer; the loss function adopts a cross entropy loss function; the training rounds are early stopped according to the performance of the validation set.

[0062] It should be understood that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here. In addition, the use of suffixes such as "module", "component" or "unit" used to represent elements is only for the purpose of facilitating the description of the present invention, and has no specific meaning in itself. Therefore, "module", "component" or "unit" can be used in a mixed manner. The terminal can be implemented in various forms. For example, the terminal described in the present invention may include mobile terminals such as mobile phones, tablet computers, laptops, PDAs, portable media players (PMP), navigation devices, wearable devices, smart bracelets, pedometers, and fixed terminals such as digital TVs and desktop computers. The subsequent description will be described by taking the mobile terminal as an example, and those skilled in the art will understand that, in addition to components specifically used for mobile purposes, the construction according to the embodiment of the present invention can also be applied to fixed types of terminals.

[0063] In some other embodiments of the present invention, an electronic device 300 is disclosed. Figure 3 As shown, it may include: one or more processors 301; memory 302; display 303; one or more applications (not shown); and one or more computer programs 304. The above-mentioned components may be connected via one or more communication buses 305. The one or more computer programs 304 are stored in the above-mentioned memory 302 and configured to be executed by the one or more processors 301. The one or more computer programs 304 include instructions, which may be used to execute the following instructions: Figure 1 The various steps in the corresponding embodiments.

[0064] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0065] The memory 302 may be an internal storage unit of the electronic device 300, such as a hard disk or memory of the electronic device 300. The memory 302 may also be an external storage device of the electronic device 300, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (FlashCard), etc. equipped on the electronic device 300. Further, the memory 302 may also include both an internal storage unit of the electronic device 300 and an external storage device. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0066] The computer program 304 may be divided into one or more modules / units. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions. The instruction segments are used to describe the execution process of the computer program 304 in the electronic device 300 .

[0067] In addition to the above structure, those skilled in the art can understand that Figure 3 It is only an example of the electronic device 300 and does not constitute a limitation of the electronic device 300. The electronic device 300 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0068] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0069] Based on the above embodiments, the present invention further discloses a computer-readable storage medium having at least one computer program stored thereon. When the computer program is executed by a processor, the vehicle-mounted imaging road information reconstruction method based on deep learning in the above embodiments is implemented.

[0070] A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing a processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state disk (SSD)), etc.

[0071] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0072] In summary, the deep learning-based on-vehicle imaging road information reconstruction method and its application disclosed in the present invention utilize the combination of the first deep learning model and the second deep learning model to fully utilize the multi-dimensional information in the millimeter-wave on-vehicle imaging data, realize high-precision reconstruction of road information, and improve the real-time processing capability in the on-vehicle environment.

[0073] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for reconstructing vehicle-mounted imaging road information based on deep learning, characterized in that: include: Preprocess the historical millimeter-wave vehicle-mounted imaging data to obtain multi-channel data; Constructing an initial deep learning model, and using the initial deep learning model to perform feature extraction on the multi-channel data to obtain a feature sequence; Based on the feature sequence, multi-scale fusion and GRU time fusion are performed, and a trained deep learning model is obtained by training; The trained deep learning model is used to identify real-time millimeter-wave vehicle-mounted imaging data to generate road information reconstruction results.

2. The method according to claim 1, characterized in that The initial deep learning model is DeepLabV3+ and GRU model; and / or the road information reconstruction result includes the recognition of lane lines, road edges and traffic signs.

3. The method according to claim 1, characterized in that The preprocessing of historical millimeter wave vehicle imaging data also includes: Use millimeter-wave radar to collect historical millimeter-wave vehicle-mounted imaging data to obtain the position, height, speed and intensity information of each pixel.

4. The method according to claim 3, characterized in that Preprocess the historical millimeter-wave vehicle-mounted imaging data to obtain multi-channel data including: The position, height, speed and intensity information of each pixel in the historical millimeter-wave vehicle-mounted imaging data are normalized and encoded to obtain a multi-channel data matrix.

5. The method according to claim 4, characterized in that After building the initial deep learning model, it also includes: The number of input channels of the initial deep learning model is adjusted according to the dimensional configuration of the multi-channel data matrix.

6. The method according to claim 1, characterized in that The multi-scale fusion and GRU time fusion based on the feature sequence include: Use dilated convolution to capture spatial features of different scales and fuse multi-dimensional information to obtain feature sequences; The feature sequence is used to capture dynamic changes and correlations in the time series.

7. The method according to claim 1, characterized in that Building the initial deep learning model involves: Build a deep learning network architecture including preprocessing layer, DeepLabV3+ feature extraction layer, GRU temporal modeling layer and output layer, configure loss function and optimizer, set learning rate and training rounds.

8. The method according to claim 7, characterized in that The learning rate adopts an adaptive learning rate strategy; and / or the optimizer is an Adam optimizer; and / or the loss function adopts a cross entropy loss function; and / or the training rounds are stopped early according to the performance of the validation set.

9. A vehicle-mounted imaging road information reconstruction device based on deep learning, used in the method described in any one of claims 1 to 8, characterized in that: include: A preprocessing unit, used for preprocessing historical millimeter-wave vehicle-mounted imaging data to obtain multi-channel data; A construction unit, used to construct an initial deep learning model, and use the initial deep learning model to perform feature extraction on the multi-channel data to obtain a feature sequence; A fusion unit, used for performing multi-scale fusion and GRU time fusion based on the feature sequence, and training to obtain a trained deep learning model; The recognition unit is used to use the trained deep learning model to recognize real-time millimeter wave vehicle-mounted imaging data and generate a road information reconstruction result.

10. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a program executable on the processor, and when the program is executed by the processor, the electronic device implements the method according to any one of claims 1 to 8.

11. A readable storage medium, wherein a program is stored in the readable storage medium, characterized in that: When the program is executed, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images