Night road multi-dimensional three-dimensional feature perception method based on visual-tactile fusion

Abstract

The present application relates to the technical field of automobile intelligent perception, and particularly relates to a night road surface multi-dimensional three-dimensional feature perception method based on vision and touch fusion. The method comprises the following steps: acquiring visual data collected by a vehicle body surround-view camera system and touch data collected by a tire touch perception system in real time; when the visual data is invalid, inputting the touch data into a pre-trained perception model to generate pseudo-visual data corresponding to the touch data by using a pseudo-visual reconstruction model and a vision and touch correlation model in the perception model; and obtaining a real-time road surface perception result by using a road surface reconstruction model in the perception model based on a vision and touch fusion data pair composed of the touch data and the pseudo-visual data. The present application can still rely on single touch data and a trained perception model to perform a road surface reconstruction task when the visual data is invalid (such as at night, in heavy fog, under reflection, etc.), and still maintains a high perception accuracy.

Description

Technical Field

[0001] This invention relates to the field of automotive intelligent perception technology, and in particular to a method for nighttime road surface multi-dimensional stereoscopic feature perception based on visual-touch fusion. Background Technology

[0002] With the development of intelligent vehicles, technologies such as autonomous driving and active control are maturing, leading to an ever-increasing demand for more comprehensive, accurate, and reliable perception systems. Currently, vehicle perception methods based on visual-tactile fusion are being widely researched.

[0003] However, when performing road perception tasks at night, the recognition accuracy and robustness of the visual-tactile fusion model will also decrease significantly due to the deterioration of visual modality data quality, which may have serious consequences in real-vehicle applications.

[0004] Therefore, there is an urgent need for a method for perceiving multi-dimensional stereoscopic features of road surfaces at night based on visual-touch fusion. Summary of the Invention

[0005] To address the problem of failing to achieve high-precision road surface perception when nighttime visual data is unavailable, this invention provides a method for nighttime road surface multi-dimensional stereo feature perception based on visual-touch fusion.

[0006] In a first aspect, embodiments of the present invention provide a nighttime road surface multi-dimensional stereo feature perception method based on visual-touch fusion, applied to an in-vehicle computer, the method comprising:

[0007] Real-time acquisition of visual data collected by the vehicle surround view camera system and tactile data collected by the tire tactile perception system;

[0008] When the visual data is invalid, the tactile data is input into a pre-trained perception model to generate pseudo-visual data corresponding to the tactile data using the pseudo-visual reconstruction model and the visual-tactile association model in the perception model; wherein, the perception model includes a visual-tactile association model, a pseudo-visual reconstruction model and a road surface reconstruction model, and the visual-tactile association model, the pseudo-visual reconstruction model and the road surface reconstruction model are connected in series.

[0009] Using the road surface reconstruction model in the perception model, and based on the visual-touch fusion data pair composed of the tactile data and the pseudo-visual data, real-time road surface perception results are obtained.

[0010] Secondly, embodiments of the present invention also provide a nighttime road surface multi-dimensional stereoscopic feature perception device based on visual-touch fusion, which is installed in an on-board computer and includes:

[0011] The acquisition unit is used to acquire visual data collected by the vehicle surround view camera system and tactile data collected by the tire tactile perception system in real time.

[0012] A reconstruction unit is used to input the tactile data into a pre-trained perception model when the visual data is invalid, so as to generate pseudo-visual data corresponding to the tactile data using the pseudo-visual reconstruction model and the visual-tactile association model in the perception model; wherein, the perception model includes a visual-tactile association model, a pseudo-visual reconstruction model and a road surface reconstruction model, the visual-tactile association model is connected to the pseudo-visual reconstruction model, and the pseudo-visual reconstruction model is connected to the road surface reconstruction model;

[0013] The perception unit is used to obtain real-time road perception results based on the visual-touch fusion data pair composed of the tactile data and the pseudo-visual data, using the road reconstruction model in the perception model.

[0014] Thirdly, embodiments of the present invention also provide a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the method described in any embodiment of this specification.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods described in any embodiment of this specification.

[0016] This invention provides a method for multi-dimensional stereoscopic feature perception of road surfaces at night based on visual-tactile fusion. When only tactile data is available in dark conditions, the pseudo-visual reconstruction model can generate corresponding pseudo-visual data based on a pre-trained visual-tactile association model. This pseudo-visual feature includes visual modal features matching the input tactile information, and can replace invalid real visual data input into the road surface reconstruction model. Therefore, this solution can still perform road surface reconstruction tasks using single tactile data and a trained perception model even when visual data is invalid (such as in darkness, heavy fog, or glare), while maintaining a high perception accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a nighttime road surface multi-dimensional stereo feature perception method based on visual-touch fusion provided by an embodiment of the present invention;

[0019] Figure 2This is a hardware architecture diagram of a computing device provided in an embodiment of the present invention;

[0020] Figure 3 This is a structural diagram of a nighttime road surface multi-dimensional stereoscopic feature perception device based on visual-touch fusion, provided by an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] Please refer to Figure 1 This invention provides a method for nighttime road surface multi-dimensional stereo feature perception based on visual-touch fusion, applied to an in-vehicle computer. The method includes:

[0023] Step 100: Real-time acquisition of visual data collected by the vehicle surround view camera system and tactile data collected by the tire tactile perception system;

[0024] Step 102: When the visual data is invalid, the tactile data is input into the pre-trained perception model to generate the pseudo-visual data corresponding to the tactile data by using the pseudo-visual reconstruction model and the visual-tactile association model in the perception model; wherein, the perception model includes the visual-tactile association model, the pseudo-visual reconstruction model and the road reconstruction model, and the visual-tactile association model, the pseudo-visual reconstruction model and the road reconstruction model are connected in series.

[0025] Step 104: Using the road reconstruction model in the perception model, based on the visual-touch fusion data pair composed of the tactile data and pseudo-visual data, the real-time road perception result is obtained.

[0026] In this embodiment of the invention, when only tactile data is available under dark conditions, the pseudo-visual reconstruction model can generate corresponding pseudo-visual data based on a pre-trained visual-tactile association model. This pseudo-visual feature contains visual modal features that match the input tactile information, and can replace the invalid real visual data input to the road reconstruction model. Therefore, this solution can still perform road reconstruction tasks using only tactile data and a trained perception model even when visual data is invalid (such as in darkness, fog, or glare), while maintaining a high perception accuracy.

[0027] For step 100:

[0028] In this embodiment of the invention, the hardware consists of an onboard computer, an onboard data acquisition unit, and additional modules. The onboard data acquisition unit includes a vehicle surround-view camera system and a tire tactile sensing system, used for real-time acquisition of visual and tactile data, respectively.

[0029] Regarding step 102:

[0030] In this embodiment of the invention, a pre-trained perception model is required. The perception model in this embodiment includes a visual-tactile association model, a pseudo-visual reconstruction model, and a road surface reconstruction model, which are connected in series. The visual-tactile association model can match visual data features with tactile data features, providing self-supervised information for the road surface reconstruction task when visual data is invalid, and helping to train the pseudo-visual reconstruction model under conditions such as nighttime.

[0031] Therefore, when visual data is invalid, such as in the face of darkness, backlighting, heavy fog, or camera contamination, tactile data is input into a pre-trained pseudo-visual reconstruction model of the perception model. This pseudo-visual reconstruction model can then generate pseudo-visual data corresponding to the tactile data based on the visual-tactile association model. Compared to traditional visual-tactile fusion perception schemes, which mostly rely on vision with touch assisting vision for road surface perception, and whose recognition accuracy tends to decrease significantly when vision fails, this embodiment can generate pseudo-visual data encompassing road surface image features when visual data is invalid, thus supplementing tactile data. Therefore, the road surface reconstruction effect of this invention is significantly better than traditional schemes when visual data is invalid.

[0032] It is understandable that when visual data is available during the day, combining visual and tactile data into a data pair and inputting it into a pre-trained perception model for road reconstruction allows for real-time road perception.

[0033] Next, the training process of the perception model will be explained.

[0034] In some implementations, the perception model is trained and generated in the following manner:

[0035] Acquire a first training set when the visual data is good and a second training set when the visual data is bad; wherein, the first training set contains several labeled visual-touch sample pairs, and the second training set contains several labeled tactile samples, and the labels are the road surface type, road surface anomaly box, road surface friction coefficient, road surface depth, and road surface height corresponding to the current sample.

[0036] The visual-touch sample pairs from the first training set are input into the visual-touch association network in the pre-constructed perception network, and the output of the visual-touch association network is used to initially train the pseudo-visual reconstruction network and the road reconstruction network to obtain the visual-touch association model, the initial pseudo-visual reconstruction model, and the initial road reconstruction model.

[0037] Using the second training set and the visual-tactile association model, the initial pseudo-visual reconstruction model and the initial road surface reconstruction model are optimized and trained to obtain the final pseudo-visual reconstruction model and road surface reconstruction model.

[0038] In this embodiment, the visual-touch sample pairs from the first training set, taken during daylight when visual data is abundant and visual data is good, are used to pre-train the visual-touch association network, the pseudo-visual reconstruction network, and the road surface reconstruction network. The visual-touch association network can help optimize the pseudo-visual reconstruction model and the road surface reconstruction model when visual data is invalid, so as to achieve three-dimensional reconstruction of the road surface using only single tactile data.

[0039] In this embodiment, existing visual-tactile fusion road surface perception methods only perceive one-dimensional features of the road surface, such as determining the degree of hydroplaning or road anomalies, and cannot achieve accurate anomaly detection and multi-dimensional stereoscopic feature perception. Therefore, in this embodiment, the labels of the first and second training sets include road surface type (cement road, asphalt road, etc.), road anomaly boxes (manhole covers, speed bump detection boxes, etc.), road surface friction coefficient, road surface depth, and road surface height (road unevenness). This allows for accurate identification of road surface features based on the mature and powerful recognition capabilities of visual data, and for the perception of multi-dimensional stereoscopic features of the road surface based on the excellent light robustness of tactile data.

[0040] In some implementations, the visual samples in the first and second training sets are jointly collected by fisheye cameras at the four corners of the vehicle body in the vehicle surround view camera system, and the tactile samples in the first and second training sets are simultaneously collected by a first accelerometer located at the center of the four wheels, a PVDF piezoelectric sensor embedded in the inner wall of the tire, a second accelerometer embedded in the inner wall of the tire, and a tire pressure sensor embedded in the inner wall of the tire.

[0041] In this embodiment, the vehicle surround-view camera system consists of fisheye cameras at the four corners of the vehicle body, used to generate a bird's-eye view image of the vehicle's surround view, capturing road information as accurately as possible. The tire tactile sensing system includes a first accelerometer located at the center of each of the four wheels, a PVDF piezoelectric sensor embedded in the inner wall of the tire, a second accelerometer embedded in the inner wall of the tire, and a tire pressure sensor embedded in the inner wall of the tire. The first accelerometer at the center of the wheel is used to collect the vertical acceleration of the four wheel centers. When the tire rotates, the PVDF piezoelectric sensor generates an electrical signal due to the piezoelectric effect, which is used to reflect the strain information of the inner wall of the tire. This sensor does not require a power supply and is suitable for measuring tire condition. The second accelerometer embedded in the inner wall of the tire is used to collect the vertical acceleration of the inner wall of the tire when it is in contact with the ground. The tire pressure sensor embedded in the inner wall of the tire is used to collect the tire pressure change when the tire rotates. Therefore, the bird's-eye view image formed by the vehicle's surround-view cameras can cover more road surface details and correspond with tactile data in real time; the tire-embedded sensors that provide tactile data can accurately capture tire status, and tactile data based on multiple sensors can cover a large number of road surface features, which is beneficial for the perception of multi-dimensional and three-dimensional features of the road surface.

[0042] In some implementations, the vehicle is also equipped with a data acquisition module, a data transmission module, a data storage module, and a power supply module;

[0043] The visual and tactile samples in the first and second training sets were preprocessed in the following manner:

[0044] The onboard computer receives image data collected by fisheye cameras at the four corners of the vehicle body, and generates a bird's-eye view image of the road surface after stitching and correction, thus obtaining a visual sample;

[0045] The data acquisition module samples the first accelerometer, PVDF piezoelectric sensor, second accelerometer and tire pressure sensor to generate a one-dimensional tactile information group in real time;

[0046] The data transmission module transmits the one-dimensional tactile information group to the data storage module.

[0047] The onboard computer obtains a one-dimensional tactile information set from the data storage module, and performs outlier removal and wavelet thresholding on the one-dimensional tactile information set to obtain the corresponding tactile sample.

[0048] In this embodiment, the additional hardware modules include a data acquisition module, a data transmission module, a data storage module, and a power supply module. The frame rate of the surround-view camera after stitching is no less than 30p, the accelerometer sampling frequency is no less than 1000Hz, and the PVDF piezoelectric sensor sampling frequency is no less than 2000Hz.

[0049] In some implementations, the onboard computer removes outliers in the following manner:

[0050] p(|x-μ|>3σ)≤0.003

[0051] In the formula, x is the data value of the one-dimensional tactile information group, μ is the mean of the one-dimensional tactile information group, and σ is the standard deviation of the one-dimensional tactile information group.

[0052] In this embodiment, data points exceeding this range are considered discrete points and discarded.

[0053] In some implementations, the initial pseudo-visual reconstruction model and the initial road surface reconstruction model are optimized and trained using a second training set and a visual-tactile association model to obtain the final pseudo-visual reconstruction model and road surface reconstruction model, including:

[0054] The tactile samples from the second training set are input into the initial pseudo-visual reconstruction model, which generates pseudo-visual samples corresponding to the tactile samples based on the visual-tactile association model.

[0055] Tactile samples and pseudo-visual samples are combined into sample pairs and input into the initial road reconstruction model. The classifier and semantic segmenter in the initial road reconstruction model are used to obtain the segmentation results of the corresponding road type, road anomaly box, road friction coefficient, road depth, road height and road features for the sample pair.

[0056] Based on the sample, the network parameters of the initial pseudo-visual reconstruction model and the initial road reconstruction model are optimized according to the corresponding road surface type, road surface anomaly box, road surface friction coefficient, road surface depth, road surface height and the label corresponding to the tactile sample, until the final pseudo-visual reconstruction model and road surface reconstruction model are obtained.

[0057] In this embodiment, the perception model based on prior knowledge can extract multimodal features of vision and touch and construct a visual-touch association model under good lighting conditions. During real-time road reconstruction, the pseudo-visual reconstruction model can quickly reconstruct the pseudo-visual features of the road surface based on the pre-trained visual-touch association model and the real-time input tactile data. Then, the road reconstruction model is used to reconstruct the multi-dimensional stereoscopic features of the road surface at night, which not only ensures timeliness but also improves the perception accuracy at night.

[0058] In this embodiment of the invention, the visual-tactile association model is a backbone network, the pseudo-visual reconstruction model is a generative adversarial network, and the road surface reconstruction model is built based on a transformer network. The road surface reconstruction model includes a classifier and a semantic segmenter. Due to the excellent performance of transformer networks in performing corresponding temporal classification tasks, the road surface reconstruction model built based on transformer networks has advantages such as high accuracy, high robustness, and low response time when performing road surface reconstruction tasks.

[0059] Regarding step 104:

[0060] After obtaining the pseudo-visual data in step 102, the visual-touch fusion data composed of tactile data and pseudo-visual data is input into the road reconstruction model in the perception model. This model can classify common road types (such as asphalt road, cement road, masonry road, water-crossing road, sand road, etc.) and road anomalies (such as manhole covers, speed bumps, road damage and potholes, road bulges, and laid rails, etc.), and reconstruct road depth information such as road depth and road unevenness, outputting real-time road depth heat map and road unevenness level.

[0061] like Figure 2 , Figure 3 As shown, this embodiment of the invention provides a multi-dimensional stereoscopic feature perception device for nighttime road surfaces based on visual-touch fusion. The device embodiment can be implemented through software, hardware, or a combination of both. From a hardware perspective, as... Figure 2 The diagram shown is a hardware architecture diagram of a computing device for a nighttime road surface multi-dimensional stereoscopic feature perception device based on visual-touch fusion, provided in an embodiment of the present invention. (Except for...) Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, the computing device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing packets. Taking software implementation as an example, such as... Figure 3 As shown, a device in a logical sense is formed by the CPU of its computing device reading the corresponding computer program from the non-volatile memory into memory and running it. This embodiment provides a nighttime road surface multi-dimensional stereoscopic feature perception device based on visual-touch fusion, installed in an onboard computer, and the device includes:

[0062] The acquisition unit 301 is used to acquire visual data collected by the vehicle surround view camera system and tactile data collected by the tire tactile perception system in real time.

[0063] The reconstruction unit 302 is used to input tactile data into a pre-trained perception model when the visual data is invalid, so as to generate pseudo-visual data corresponding to the tactile data by using the pseudo-visual reconstruction model and the visual-tactile association model in the perception model; wherein, the perception model includes a visual-tactile association model, a pseudo-visual reconstruction model and a road surface reconstruction model, the visual-tactile association model is connected to the pseudo-visual reconstruction model, and the pseudo-visual reconstruction model is connected to the road surface reconstruction model.

[0064] The sensing unit 303 is used to obtain real-time road perception results based on the visual-touch fusion data pair composed of the tactile data and pseudo-visual data, using the road reconstruction model in the sensing model.

[0065] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on a nighttime road surface multi-dimensional stereoscopic feature perception device based on visual-touch fusion. In other embodiments of the present invention, a nighttime road surface multi-dimensional stereoscopic feature perception device based on visual-touch fusion may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0066] The information interaction and execution process between the various units in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description of the method embodiment of the present invention, and will not be repeated here.

[0067] This invention also provides a computing device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements a nighttime road surface multi-dimensional stereo feature perception method based on visual-touch fusion according to any embodiment of this invention.

[0068] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program causes the processor to perform a nighttime road surface multi-dimensional stereo feature perception method based on visual-touch fusion according to any embodiment of this invention.

[0069] Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0070] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0071] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0072] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0073] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion module connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion module execute some and all of the actual operations, thereby realizing the function of any of the above embodiments.

[0074] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0075] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for nighttime road surface multi-dimensional stereoscopic feature perception based on visual-touch fusion, characterized in that, Applied to in-vehicle computers, the method includes: Real-time acquisition of visual data collected by the vehicle surround view camera system and tactile data collected by the tire tactile perception system; When the visual data is invalid, the tactile data is input into a pre-trained perception model to generate pseudo-visual data corresponding to the tactile data using the pseudo-visual reconstruction model and the visual-tactile association model in the perception model; wherein, the perception model includes a visual-tactile association model, a pseudo-visual reconstruction model and a road surface reconstruction model, and the visual-tactile association model, the pseudo-visual reconstruction model and the road surface reconstruction model are connected in series. Using the road surface reconstruction model in the perception model, and based on the visual-touch fusion data pair composed of the tactile data and the pseudo-visual data, real-time road surface perception results are obtained; The perception model was trained and generated in the following manner: Acquire a first training set when the visual data is good and a second training set when the visual data is bad; wherein, the first training set contains several labeled visual-touch sample pairs, and the second training set contains several labeled tactile samples, and the labels are the road surface type, road surface anomaly box, road surface friction coefficient, road surface depth, and road surface height corresponding to the current sample. The visual-touch sample pairs of the first training set are input into the visual-touch association network in the pre-constructed perception network, and the output of the visual-touch association network is used to initially train the pseudo-visual reconstruction network and the road reconstruction network to obtain the visual-touch association model, the initial pseudo-visual reconstruction model and the initial road reconstruction model. Using the second training set and the visual-tactile association model, the initial pseudo-visual reconstruction model and the initial road surface reconstruction model are optimized and trained to obtain the final pseudo-visual reconstruction model and road surface reconstruction model.

2. The method according to claim 1, characterized in that, The visual samples in the first training set and the second training set are jointly collected by fisheye cameras at the four corners of the vehicle body in the vehicle surround view camera system. The tactile samples in the first training set and the second training set are simultaneously collected by a first accelerometer set at the center of the four wheels, a PVDF piezoelectric sensor embedded in the inner wall of the tire, a second accelerometer embedded in the inner wall of the tire, and a tire pressure sensor embedded in the inner wall of the tire.

3. The method according to claim 2, characterized in that, The vehicle is also equipped with a data acquisition module, a data transmission module, a data storage module, and a power supply module; The visual and tactile samples in the first and second training sets were preprocessed in the following manner: The onboard computer receives image data collected by fisheye cameras at the four corners of the vehicle body, and generates a bird's-eye view image of the road surface after stitching and correction, thus obtaining a visual sample. The data acquisition module samples the first accelerometer, the PVDF piezoelectric sensor, the second accelerometer, and the tire pressure sensor to generate a one-dimensional tactile information group in real time. The data transmission module is used to transmit the one-dimensional tactile information group to the data storage module; The onboard computer obtains the one-dimensional tactile information group from the data storage module, and performs outlier removal and wavelet threshold noise reduction on the one-dimensional tactile information group to obtain the corresponding tactile sample.

4. The method according to claim 3, characterized in that, The on-board computer performs outlier removal processing in the following manner: In the formula, The data values ​​of the one-dimensional tactile information group, The average value of the one-dimensional tactile information set. denoted as the standard deviation of the one-dimensional tactile information set.

5. The method according to claim 1, characterized in that, The step of optimizing and training the initial pseudo-visual reconstruction model and the initial road surface reconstruction model using the second training set and the visual-tactile association model to obtain the final pseudo-visual reconstruction model and road surface reconstruction model includes: The tactile samples of the second training set are input into the initial pseudo-visual reconstruction model, which generates pseudo-visual samples corresponding to the tactile samples based on the visual-tactile association model. The tactile samples and pseudo-visual samples are combined into a sample pair and input into the initial road reconstruction model. The classifier and semantic segmenter in the initial road reconstruction model are used to obtain the segmentation results of the road type, road anomaly box, road friction coefficient, road depth, road height and road features corresponding to the sample pair. Based on the sample, the network parameters of the corresponding road surface type, road surface anomaly box, road surface friction coefficient, road surface depth, road surface height, and the label corresponding to the tactile sample are optimized until the final pseudo-visual reconstruction model and road surface reconstruction model are obtained.

6. The method according to any one of claims 1-4, characterized in that, The visual-tactile association model is a backbone network, the pseudo-visual reconstruction model is a generative adversarial network, and the road reconstruction model is built based on a transformer network.

7. A nighttime road surface multi-dimensional stereoscopic feature perception device based on visual-touch fusion, used to implement the method as described in any one of claims 1-6, characterized in that, The device, located in an in-vehicle computer, includes: The acquisition unit is used to acquire visual data collected by the vehicle surround view camera system and tactile data collected by the tire tactile perception system in real time. A reconstruction unit is used to input the tactile data into a pre-trained perception model when the visual data is invalid, so as to generate pseudo-visual data corresponding to the tactile data using the pseudo-visual reconstruction model and the visual-tactile association model in the perception model; wherein, the perception model includes a visual-tactile association model, a pseudo-visual reconstruction model and a road surface reconstruction model, the visual-tactile association model is connected to the pseudo-visual reconstruction model, and the pseudo-visual reconstruction model is connected to the road surface reconstruction model; The perception unit is used to obtain real-time road perception results based on the visual-touch fusion data pair composed of the tactile data and the pseudo-visual data, using the road reconstruction model in the perception model.

8. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-6.

Images