Traffic marking intelligent sensing and multivariate data fusion processing method, system and device and storage medium

Abstract

The invention belongs to the technical field of intelligent traffic, and discloses a traffic marking intelligent sensing and multivariate data fusion processing method, system and device and a storage medium. The method comprises the steps of obtaining a road image collected by a vehicle-mounted camera in real time; analyzing and processing the road image to obtain visual perception data of the road marking; the method comprises the following steps: generating road digital data based on positioning information of a current road and stored road digital information; and fusing the visual perception data of the road marking and the road digital data to obtain a fused and optimized digital marking result. According to the method, the road marking information is perceived visually, the positioning information and the road digital information are combined, multivariate data fusion is carried out, the digital marking information with higher precision is provided for an automatic driving system, and the false detection condition caused by interference of other factors of the road is effectively prevented.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and particularly relates to a method, system, device and storage medium for intelligent perception of traffic markings and multi-source data fusion processing. Background Art

[0002] The perception of road markings based on image recognition is widely used in the field of intelligent transportation. By accurately identifying information such as road markings and arrows through in-vehicle cameras, it provides key support for autonomous driving, significantly improving driving safety and efficiency.

[0003] The existing patent No. CN118876962A discloses a vehicle attitude adjustment method and system for compensating lane lines based on the curve of a lighting lamp. This method fits and compensates the curve of the road lighting source image and the lane line curve to obtain the vehicle direction and lane line change situation, enabling the vehicle to maintain its position in the lane. This method depends on the layout of the lighting lamps and is affected by image noise, with limited effectiveness under poor lighting conditions. Another example is the lane number determination method, device, storage medium and program product disclosed in patent No. CN118865274A. This method determines the lane number by detecting the number of lane arrows, but it is very susceptible to road surface conditions. For example, inconsistent arrow directions may lead to incorrect judgments. Therefore, the existing methods for identifying road markings are prone to interference from other road information and rely on external lighting sources. Summary of the Invention

[0004] Aiming at the problems in the prior art that the identified road markings are easily interfered by other road information and need to rely on external lighting sources, the present invention provides a method, system, device and storage medium for intelligent perception of traffic markings and multi-source data fusion processing. It can combine visual perception of road marking information, positioning information and road digital information to perform multi-source data fusion and provide more accurate digital marking information for the autonomous driving system. The specific technical solutions are as follows:

[0005] A method for intelligent perception of traffic markings and multi-source data fusion processing includes:

[0006] Obtain the road image collected in real time by the in-vehicle camera;

[0007] Analyze and process the road image to obtain the visual perception data of the road markings;

[0008] Generate road digital data based on the positioning information of the current road and the stored road digital information;

[0009] Fuse the visual perception data of the road markings and the road digital data to obtain the fused and optimized digital marking result.

[0010] Preferably, the analysis and processing of the road image to obtain the visual perception data of the road markings include:

[0011] Input the road image into the road marking category detection model trained based on RepLK-YOLOv8, and output the road marking categories including road number and text information and road number data;

[0012] Perform corresponding processing on different road marking categories to obtain visual perception data including coordinates, text / digital content, and parameters.

[0013] Preferably, the training process of the road marking category detection model trained based on RepLK-YOLOv8 includes:

[0014] Obtain the image data of the road marking lines;

[0015] Preprocess the image data;

[0016] According to the processed image data, store the image data after the information category annotation operation on the processed image data by artificial;

[0017] Input the labeled image data into the RepLK-YOLOv8 detection model for learning and training to obtain a trained road marking category detection model capable of outputting road category results.

[0018] Preferably, the corresponding processing of different road marking categories to obtain visual perception data including coordinates, text / digital content, category, and parameters includes:

[0019] Correct the text digital image of the road number and text information, and perform end-to-end character sequence recognition after correction to output visual perception data of the text structure of coordinates and text / digital content;

[0020] Directly output visual perception data of coordinates and parameters for the road number data.

[0021] Preferably, obtaining the road number data by based on the positioning information of the current road and the stored road number information includes:

[0022] Obtain the current positioning information and the stored road number information;

[0023] Generate road number data by combining the current positioning information and the road number information.

[0024] Preferably, the fusion of the visual perception data of the road markings and the road number data to obtain the fused and optimized digital marking result includes:

[0025] Interpolate and select points from the visual perception data of road markings to obtain a discrete set of marking points;

[0026] Match the set of marking points with the road digital data to construct matching point pairs;

[0027] Construct an error function model based on the matching point pairs;

[0028] Perform non-linear optimization according to the error function model to obtain the fused and optimized digital marking result.

[0029] Preferably, the interpolating and selecting points from the visual perception data of road markings to obtain a discrete set of marking points includes:

[0030] Establish a corresponding marking curve equation according to the visual perception data of road markings;

[0031] Interpolate the marking curve equation to generate a discrete set of marking points.

[0032] A traffic marking intelligent perception and multi-source data fusion processing system, applied to the foregoing traffic marking intelligent perception and multi-source data fusion processing method, includes:

[0033] A data acquisition unit for acquiring road images collected in real time by an in-vehicle camera;

[0034] A data processing unit for analyzing and processing the road images to obtain the visual perception data of road markings;

[0035] A road digital data generation unit for obtaining road digital data through the positioning information of the current road and the stored road digital information;

[0036] A data fusion unit for fusing the visual perception data of road markings and the road digital data to obtain the fused and optimized digital marking result.

[0037] An electronic device, including:

[0038] A processor;

[0039] A memory for storing instructions executable by the processor;

[0040] Wherein, the processor realizes the foregoing traffic marking intelligent perception and multi-source data fusion processing method by running the executable instructions.

[0041] A computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the foregoing traffic marking intelligent perception and multi-source data fusion processing method is realized.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] An intelligent perception and multi - data fusion processing method for traffic markings of the present invention obtains a road image collected in real - time by an in - vehicle camera; analyzes and processes the road image to obtain visual perception data of the road markings; generates road digital data based on the positioning information of the current road and the stored road digital information; and fuses the visual perception data of the road markings and the road digital data to obtain a digitally marked line result after fusion and optimization. The present invention visually perceives road marking information, combines positioning information and road digital information, performs multi - data fusion, provides digitally marked line information with higher accuracy for the autonomous driving system, and effectively prevents misdetection caused by interference from other road factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0045] Figure 1 It is a flowchart of an intelligent perception and multi - data fusion processing method for traffic markings of the present invention.

[0046] Figure 2 It is a flowchart of an embodiment of an intelligent perception and multi - data fusion processing method for traffic markings of the present invention.

[0047] Figure 3 It is a flowchart of another embodiment of an intelligent perception and multi - data fusion processing method for traffic markings of the present invention.

[0048] Figure 4 It is a schematic diagram of an intelligent perception and multi - data fusion processing system for traffic markings of the present invention.

[0049] Figure 5 It is a schematic diagram of the RepLKDeXt structure of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0051] It should be understood that when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0052] It should also be understood that the terms used in the specification of the present invention are merely for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0053] It should be further understood that the term "and / or" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0054] For the following embodiments, please refer to Figures 1 to 5 。

[0055] The embodiments of the present application provide a method for intelligent perception of traffic markings and multi-source data fusion processing, including:

[0056] Step S1, acquiring a road image collected in real time by an in-vehicle camera;

[0057] By selecting an in-vehicle camera with high resolution and wide viewing angle, it is ensured that it can clearly capture the road scene in a certain range in front of and around the vehicle. The in-vehicle camera is firmly installed at a suitable position on the vehicle, such as the center above the windshield, the front end of the rearview mirror, and the front bumper of the car, so that its shooting field of view is not blocked, and necessary debugging and calibration are carried out. At the same time, a specific acquisition frequency is set for acquisition, for example, a certain number of images are acquired per second, ensuring that road information can be obtained in real time. When the vehicle is driving, the in-vehicle camera captures the road image in real time according to the set acquisition strategy. The acquired image data is timely stored in the data storage device equipped on the vehicle or uploaded to the cloud server for storage and analysis processing through a data cable or wireless transmission method.

[0058] Step S2, analyzing and processing the road image to obtain visual perception data of the road markings;

[0059] The preprocessed road image is input into a road marking category detection model trained based on RepLK-YOLOv8, and the output includes road digital and text information and road digital data;

[0060] The current road image is inferred and recognized by the road marking category detection model, and the road marking lines are classified into categories, specifically including:

[0061] Road digital and text information categories: speed limit signs, road texts, and other digital and text signs, etc.

[0062] Road data categories: lane lines, stop lines, lane arrows, zebra crossings, etc.

[0063] Perform corresponding processing on different road marking categories to obtain visual perception data including coordinates, text / digital content, and parameters; for example, correct the text digital image of road digital and text information, and perform end-to-end character sequence recognition after correction to output visual perception data of the text structure of coordinates, text / digital content; directly output visual perception data of coordinates and parameters for road digital data.

[0064] When the optimal model recognizes road digital and text information, it will further recognize the semantics of its text and numbers. First, it is necessary to perform text image correction (correct the text digital image of the road marking line, and through geometric transformation of the image to correct problems such as distortion, tilt, and perspective distortion of the road iconic text and digital images in the image for more accurate recognition by the subsequent text recognition module). After the corrected text image is subjected to text recognition, finally, visual perception data of [coordinates, text / digital content] of the text is obtained; when the optimal model recognizes non-road digital and text information categories, it will directly output visual perception data of [coordinates, category, parameters] of the marking line.

[0065] Step S3: Generate road digital data based on the positioning information of the current road and the stored road digital information.

[0066] Obtain the current positioning information and the stored road digital information; combine the current positioning information and the road digital information to generate road digital data.

[0067] The stored road digital information realizes the position acquisition of road markings during the process of road spraying and marking, realizes the digitization of road markings during the spraying and marking process, and stores the digitized data of road markings in the back-end server.

[0068] Step S4: Integrate the visual perception data of road markings and road digital data to obtain an integrated and optimized digital marking result.

[0069] Based on the visual perception data of in-vehicle cameras, and combined with GPS or Beidou positioning information and road digital information (road digital data), multi-source data fusion is carried out. Multi-source data fusion can find the corresponding relationship between the lane markings based on visual perception and the road digital data, and calculate the conversion relationship between the carrier coordinate system where the perceived road markings are located and the world coordinate system where the road digital information is located according to this relationship, so that the fused and optimized digital marking results are more accurate.

[0070] In this embodiment, the road marking visual perception data obtained in step S2 is matched and aligned with the road digital data generated in step S3. According to the spatial position information of the two (such as the coordinate system), the marking position and shape in the visual perception data are corresponding to the theoretical marking position in the road digital data. For possible errors and deviations, methods such as geometric transformation (such as translation, rotation, scaling, etc.) and coordinate transformation are used to adjust the two to the same coordinate system. Select a suitable fusion algorithm to perform fusion processing on the matched visual perception data and road digital data. For example, for the fusion of the marking position, the Kalman filter algorithm can be used. Combining the position information in the visual perception data and the road digital data, a more accurate estimation of the actual position of the marking can be obtained; for the fusion of the marking attributes, the weighted average method can be used. According to the reliability and accuracy of different data sources, corresponding weights are assigned to obtain a more reasonable marking attribute value.

[0071] In step S1 of this embodiment, the acquired road image data also needs to be preprocessed, including:

[0072] Grayscale processing: Convert the color image in the image data into a grayscale image, reduce the dimension and information volume of the image data, highlight the brightness information of the image, and at the same time reduce the computational complexity of subsequent processing;

[0073] Noise reduction processing: Due to the influence of factors such as the noise of the camera sensor and the shooting environment, the acquired images may contain various noise points. Removing these noises through a filtering algorithm can improve the clarity and quality of the images, and avoid noise interference with the subsequent recognition of features such as road markings. For example, replace the current pixel value with the median value in the pixel neighborhood to remove salt-and-pepper noise, or perform weighted averaging on the pixels according to the Gaussian distribution to remove Gaussian noise.

[0074] Image enhancement processing: By redistributing the grayscale values of the image pixels, expand the grayscale range of the image and enhance the overall contrast. Improve the contrast and clarity of the image, and make key information such as road markings and traffic signs more prominent.

[0075] Image cropping processing: Remove the irrelevant parts in the image and only retain the useful area (ROI), reduce unnecessary data processing, and at the same time highlight the key content, such as only retaining the area containing road markings.

[0076] In step S2 of this embodiment, a road marking category detection model is trained using the RepLK-YOLOv8 model. The RepLK-YOLOv8 model consists of a self-made lightweight backbone network and a detection head part. The backbone network is composed of ten modules (blocks). Among them, the 1st, 2nd, 4th, 6th, and 8th layers adopt the CBH structure (convolutional layer-batch normalization layer-activation function layer), and the 3rd and 9th layers are of the C2F structure (C2F performs feature transformation on the input data through two convolutional layers to enhance the network's non-linear ability and representation ability and achieve feature fusion in the channel dimension). The 5th and 7th layers adopt the RepLKDeXt structure, which consists of a 1x1 convolutional layer, a RepLKBlock layer, and a large convolutional kernel. The RepLKBlock layer is the core part of the RepLKDeXt structure. Among them, the 1x1 convolutional layers (cv1 and cv2): are used to reduce and restore the channel dimension, improve the effectiveness and computational efficiency of features, and avoid redundant parameter quantities. The large convolutional kernel convolution uses a re-parameterized large convolutional kernel convolutional layer, which can capture richer local features while maintaining computational efficiency. The schematic diagram of the RepLKDeXt structure is as Figure 5 shown. The 10th layer adopts the SPPF structure, whose main function is to perform multi-scale fusion on the features extracted by the backbone network, reduce the computational burden, and expand the receptive field. By introducing the RepLKDeXt structure, the improved backbone network combines large convolutional kernels and depth convolutions, effectively captures local features, and promotes information flow through residual connections, significantly improving the feature extraction ability and the ability to capture target details before and after. In addition, the design of RepLKDeXt ensures that while maintaining computational efficiency, it enhances the network's expressive ability, enabling the model to still achieve excellent performance in resource-constrained environments. The structure description of the Head part is as follows: The 11th and 14th layers are upsampling layers, which use the nearest neighbor interpolation method to magnify the feature map. The 12th, 15th, 18th, and 20th layers are concat layers, which mainly splice feature maps of different sizes through the channel dimension. The 17th and 20th layers adopt the CBL structure (convolutional layer-batch normalization layer-activation function layer), while the 13th, 16th, 19th, and 22nd layers are of the C2F structure, and feature transformation is performed through two convolutional layers to further enhance the network's expressive ability.

[0077] In step S3 of this embodiment, a global positioning system (GPS) or other high-precision positioning devices (such as Beidou satellite navigation system, inertial navigation system, etc.) equipped on the vehicle are used to obtain the accurate position information of the road where the vehicle is currently located in real time, including data such as longitude, latitude, and altitude. At the same time, combined with information such as the driving direction and speed of the vehicle, the specific position and driving state of the vehicle on the road are further determined.

[0078] Meanwhile, the storage process of the road marking information obtained in step S3 is as follows:

[0079] The improved marking machine is used to obtain digital road marking data. Compared with the method of specially collecting road information, the improved marking machine can collect the position of road markings during the spraying and marking process, digitize the road markings during the spraying and marking process, and store the digital road marking information, with accurate and reliable information. The basic information of the target road markings and the key point collection instruction information stored by the road marking digitization unit of the marking machine, as well as the key point information uploaded by the road marking positioning unit, constitute a complete road marking data system, and the complete road marking data system is stored in the corresponding database to assist intelligent driving. For example, the currently obtained road positioning information is integrated and processed with the called stored road digital information. According to the position and driving direction of the vehicle, the specific section where the vehicle is located is determined in the road topology, and combined with the geometric parameters of the section and the marked marking information, road digital data related to the current vehicle driving scenario is generated. For example, according to the curvature and lane width of the road, the theoretical positions and shapes of the markings at different positions are calculated; according to the road topology, information such as the road direction ahead of the vehicle and possible turning points are determined.

[0080] A traffic marking intelligent perception and multi-source data fusion processing method of the present invention obtains the road images collected in real time by the vehicle-mounted camera; analyzes and processes the road images to obtain the visual perception data of the road markings; generates road digital data based on the positioning information of the current road and the stored road digital information; and fuses the visual perception data of the road markings and the road digital data to obtain the fused and optimized digital marking result. The present invention visually perceives the road marking information, combines the positioning information and the road digital information, and performs multi-source data fusion to provide higher-precision digital marking information for the automatic driving system, effectively preventing misdetection caused by interference from other road factors.

[0081] Specifically, in a preferred embodiment of the present application, the training process of the road marking category detection model trained based on RepLK-YOLOv8 includes:

[0082] Step S21: Obtain the image data of the road marking lines;

[0083] Use in-vehicle cameras to collect road image data in multiple scenarios. According to the actual collection requirements, determine the frequency of image collection, the scenarios and time of the collected data. For example, collect a certain number of images per second to ensure real-time acquisition of road information; the collected scenario range covers different types of roads such as urban roads and highways, and also includes different weather and lighting conditions, such as night, day, rainy days, snowy days, cloudy days, sunny days, and strong light direct illumination, low light, etc. At the same time, for different states of road markings, such as incomplete markings, complete markings, clear markings, and blurred markings, targeted collection should be carried out to ensure the comprehensiveness and diversity of the collected data. And upload the collected data to the cloud server. During the storage process, add detailed metadata tags to each image, including information such as collection time, location, vehicle driving speed, weather conditions, and light intensity, to facilitate subsequent classification, screening, and analysis of the data.

[0084] Step S22: Preprocess the image data;

[0085] Perform grayscale processing, noise reduction processing, image enhancement processing, and image cropping processing on the image data. The specific implementation process is as described in the foregoing embodiments.

[0086] Step S23: Store the image data after information category annotation operation on the processed image data manually based on the processed image data;

[0087] Perform annotation on different types of markings on the processed image data through manual annotation to obtain the image data after preset type annotation; before the annotation work starts, conduct systematic training on the annotators to make the annotators clear about the different types, characteristics, annotation specifications and requirements of road markings. For example, clarify that digital and text information includes speed limit signs, road text, etc.; road digital data covers lane lines, stop lines, lane arrows, zebra crossings, etc. Through actual case demonstrations, let the annotators be familiar with the manifestation forms and annotation methods of each category in the image. Organize and store the image data after annotation categories in a specific data format (such as XML, JSON, etc.). During the storage process, integrate the annotation information with the metadata of the original image (such as collection time, location, etc.) and save them together in the annotation database.

[0088] Step S24: Input the image data with different annotated categories into the RepLK-YOLOv8 detection model for learning and training to obtain a trained road marking category detection model capable of outputting road category results.

[0089] Import the pre-trained weights of the RepLK-YOLOv8 detection model, and adjust the parameters of the model according to the characteristics and requirements of road marking detection. Set appropriate hyperparameters such as learning rate, number of iterations, batch size, etc.

[0090] The labeled image data is divided into a training set, a validation set and a test set according to a certain ratio. In this embodiment, 70% is used as a training set, 20% is used as a validation set, and 10% is used as a test set. The RepLK-YOLOv8 detection model is divided into a lightweight end-to-end target detector, a text angle calibration model and a text classifier. The labeled image data is input into the target detector for feature extraction and target positioning, and the loss function between the model prediction result and the labeled result is calculated. Through the back propagation algorithm, the gradient of the loss function to the model parameters is calculated, and then the optimizer (such as stochastic gradient descent, Adam, etc.) is used to update the model parameters, and the weights and biases of the model are continuously adjusted so that the model can better learn the characteristics and category information of the road markings, and after the model learning and training is completed, it has the ability to output road category results including road numbers and text information categories and road data category categories.

[0091] Specifically, in a preferred embodiment of the present application, the fusion of visual perception data of road markings and digital road data to obtain a fused and optimized digital road marking result includes:

[0092] Step S41, interpolating the visual perception data of the road markings to obtain a discrete marking point set;

[0093] According to the visual perception data of the road markings, the corresponding marking curve equation is established;

[0094] Interpolate the marking curve equation to generate a discrete marking point set.

[0095] Since visual perception data is usually a sparse, discrete sequence of coordinate points, directly using the original data may result in discontinuous or partially missing road marking shapes. In order to be able to more comprehensively and accurately describe the shape of road markings, it is necessary to establish a continuous curve equation to fit these discrete points. If the shape of the road marking is relatively regular and approximates a polynomial curve, you can choose polynomial fitting. For example, for straight lines or markings close to parabolic shapes, use polynomial fitting. When the shape of the road marking is more complex and there are multiple bends and turns, spline curve fitting is more appropriate. Spline curves are composed of multiple segmented polynomial curves. For example, a cubic spline curve is a cubic polynomial in each segment, which can fit complex curves well while ensuring the smoothness of the curve.

[0096] According to the selected fitting method, a specific curve equation is calculated, which can describe the shape of the road marking in a continuous form.

[0097] After obtaining the marking line curve equation, a series of discrete points need to be generated on this curve. According to the actual requirements and the actual situation of the road markings, determine the range where interpolation is required. For example, if only the markings of a certain section of the road are concerned, use the abscissa range of this section as the interpolation range. According to the density of the required discrete points, select an appropriate interpolation interval. Starting from the starting point of the interpolation range, with the preset distance as the step size, calculate the corresponding ordinate value for each abscissa value in turn. In this way, a series of discrete coordinate points are obtained, and these coordinate points form a discrete marking point set.

[0098] Step S42: Match the marking point set with the road digital data to construct matching point pairs;

[0099] The road digital data is the precise information about the road stored in advance. In order to fuse the marking point set obtained by visual perception with the road digital data, it is necessary to find the corresponding relationship between the two, that is, to construct matching point pairs.

[0100] In this embodiment, the matching method adopted is the nearest neighbor matching. For each point in the marking point set, calculate the distance from this point to all points in the road digital data (for example, use the Euclidean distance to measure the distance between two points). And by traversing all points in the road digital data, matching point pairs are obtained. At the same time, after constructing the matching point pairs, there may be some inaccurate matching situations, such as incorrect matching due to noise or data errors. By setting methods such as distance thresholds, filter out those matching point pairs with too large distances to improve the accuracy of the matching.

[0101] Step S43: Construct an error function model according to the matching point pairs;

[0102] The purpose of constructing the error function model is to measure the degree of difference between the marking point set obtained by visual perception and the road digital data. For each matching point pair, an error metric can be defined. A common error metric is the square of the Euclidean distance, and this error value reflects the deviation degree between the two points in this matching point pair.

[0103] Accumulate the error values of all matching point pairs to obtain the total error function. The smaller the value of this error function, the higher the matching degree between the marking point set obtained by visual perception and the road digital data.

[0104] Step S44: Perform non-linear optimization according to the error function model to obtain the fused and optimized digital marking result.

[0105] The error function is usually non-linear. To minimize the value of the error function, that is, to make the set of marked points obtained by visual perception as close as possible to the road digital data, non-linear optimization is required. By adopting the Levenberg-Marquardt algorithm, the damping factor is adjusted to balance the characteristics of the gradient descent method and the Gauss-Newton method. When the curvature of the error function is large, the algorithm tends to the gradient descent method; when the curvature of the error function is small, the algorithm tends to the Gauss-Newton method. This can accelerate the convergence speed while ensuring convergence.

[0106] Starting from the initial set of marked points, the coordinates of the set of marked points are continuously updated according to the selected optimization algorithm, so that the value of the error function gradually decreases. In each iteration, the new value of the error function is calculated, and it is judged whether the convergence condition is satisfied. When the convergence condition is met (such as the change in the error function is less than a preset threshold), the optimization process is stopped. The set of marked points obtained at this time is the digital marking result after fusion optimization. The digital marking result combines the information of visual perception data and road digital data, and has higher accuracy and reliability.

[0107] An embodiment of the present application provides a traffic marking intelligent perception and multi-source data fusion processing system, which is applied to the foregoing traffic marking intelligent perception and multi-source data fusion processing method, and includes:

[0108] A data acquisition unit for acquiring road images collected in real time by an in-vehicle camera;

[0109] A data processing unit for analyzing and processing the road images to obtain visual perception data of road markings;

[0110] A road digital data generation unit for obtaining road digital data by based on the positioning information of the current road and the stored road digital information;

[0111] A data fusion unit for fusing the visual perception data of road markings and road digital data to obtain a digitally marked result after fusion optimization.

[0112] In this embodiment, the function explanations of each unit are the same as those of a traffic marking intelligent perception and multi-source data fusion processing method, and the technical effects are the same, so they will not be repeated here.

[0113] An embodiment of the present application provides an electronic device, including:

[0114] A processor;

[0115] A memory for storing instructions executable by the processor;

[0116] Wherein, the processor realizes the foregoing traffic marking intelligent perception and multi-source data fusion processing method by running the executable instructions.

[0117] The technical effect of this embodiment is the same as that of the traffic marking intelligent perception and multi-source data fusion processing method in Embodiment 1, and will not be repeated here.

[0118] In this embodiment, the processor may be a central processing unit (CPU), a controller, a microcontroller, or other data processing chips.

[0119] The embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the foregoing traffic marking intelligent perception and multi-source data fusion processing method is implemented.

[0120] The technical effect of this embodiment is the same as that of the traffic marking intelligent perception and multi-source data fusion processing method in Embodiment 1, and will not be repeated here.

[0121] The present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on.

[0122] Those of ordinary skill in the art can realize that the units of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the components of each example have been generally described according to their functions in the above description. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0123] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0124] In addition, the functional units in each embodiment of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software function units.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention, and they should all be covered by the scope of the claims and the description of the present invention.

Claims

1. An intelligent perception method for traffic markings and a multi-source data fusion and processing method, characterized in that Including: Obtain the road images collected in real time by the in-vehicle camera; Analyze and process the road images to obtain the visual perception data of the road markings; Generate road digital data based on the positioning information of the current road and the stored road digital information; Fuse the visual perception data of the road markings and the road digital data to obtain the fused and optimized digital marking results.

2. The intelligent perception and multi-source data fusion processing method for traffic markings according to claim 1, wherein The analyzing and processing the road images to obtain the visual perception data of the road markings includes: Input the road images into the road marking category detection model trained based on RepLK-YOLOv8, and output the road marking categories including road digital and text information and road digital data; Perform corresponding processing on different road marking categories to obtain the visual perception data including coordinates, text / digital content, and parameters.

3. The intelligent perception and multi-source data fusion processing method for traffic markings according to claim 2, wherein, The training process of the road marking category detection model trained based on RepLK-YOLOv8 includes: Obtain the image data of the road marking lines; Preprocess the image data; According to the processed image data, store the image data after the information category annotation operation on the processed image data manually; Input the labeled image data into the RepLK-YOLOv8 detection model for learning and training to obtain the trained road marking category detection model capable of outputting the road category results.

4. A method for intelligent perception of traffic markings and multi-source data fusion processing according to claim 2, characterized in that, The performing corresponding processing on different road marking categories to obtain the visual perception data including coordinates, text / digital content, category, and parameters includes: Correct the text digital images of the road digital and text information, and perform end-to-end character sequence recognition after correction to output the visual perception data of the text structure including coordinates and text / digital content; Directly output the visual perception data of coordinates and parameters for the road digital data.

5. A method for intelligent perception of traffic markings and multi-source data fusion processing according to claim 1, characterized in that, The obtaining the road digital data based on the positioning information of the current road and the stored road digital information includes: Obtain the current positioning information and the stored road digital information; Combine the current positioning information and the road digital information to generate the road digital data.

6. A method for intelligent perception of traffic markings and multi-source data fusion processing according to claim 5, characterized in that, The fusing the visual perception data of the road markings and the road digital data to obtain the fused and optimized digital marking results includes: Interpolate and sample the visual perception data of the road markings to obtain a discrete marking point set; Match the marking point set with the road digital data to construct matching point pairs; Construct an error function model according to the matching point pairs; Perform nonlinear optimization according to the error function model to obtain the fused and optimized digital marking results.

7. A method for intelligent perception of traffic markings and multi-source data fusion processing according to claim 6, characterized in that The interpolating and sampling the visual perception data of the road markings to obtain a discrete marking point set includes: Establish a corresponding marking curve equation according to the visual perception data of the road markings; Interpolate the marking curve equation to generate a discrete marking point set.

8. An intelligent perception and multi-source data fusion processing system for traffic markings, characterized in that, Applied to a traffic marking intelligent perception and multi-source data fusion processing method according to any one of claims 1 to 7, including: A data acquisition unit for obtaining the road images collected in real time by the in-vehicle camera; A data processing unit for analyzing and processing the road images to obtain the visual perception data of the road markings; A road digital data generation unit that obtains road digital data by using the positioning information of the current road and the stored road digital information; A data fusion unit for fusing the visual perception data of road markings and the road digital data to obtain a fused and optimized digital marking result.

9. An electronic device, characterized in that, Comprising: A processor; A memory for storing processor-executable instructions; Wherein, the processor realizes a traffic marking intelligent perception and multi-source data fusion processing method as described in any one of claims 1-7 by running the executable instructions.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that, When the instruction is executed by the processor, it realizes a traffic marking intelligent perception and multi-source data fusion processing method as described in any one of claims 1-7.

Images