A lane monitoring method, device, and electronic equipment

Abstract

This invention provides a lane monitoring method, device, electronic device, and storage medium. The method includes: acquiring a road image and vehicle positioning information of a vehicle's location; acquiring road parameters corresponding to the vehicle using the vehicle positioning information; determining a first visual processing result corresponding to the road image; filtering the first visual processing result to determine a second visual processing result corresponding to the road image; determining lane information corresponding to the vehicle's location through fusion processing of the second visual processing result and the road parameters; and monitoring the vehicle's switching between different lanes based on the lane information corresponding to the vehicle's location. This achieves lane-level positioning of the vehicle and accurate monitoring of the vehicle's switching between different lanes. The solution can be applied to fields including, but not limited to, autonomous driving, vehicle-to-everything (V2X) communication, and intelligent transportation.

Description

Technical Field

[0001] This invention relates to image processing technology, and more particularly to lane monitoring methods, devices, and electronic equipment. Background Technology

[0002] Autonomous driving involves using onboard sensor systems to perceive the vehicle's surroundings and, based on the obtained road images, other vehicle positions, and obstacle information, controlling the vehicle's steering and speed to ensure safe and reliable driving on the road. In this process, automatic lane changing, applied in the field of autonomous driving, requires autonomous vehicles to autonomously select driving lanes and perform lane-changing operations. Timely and accurate lane monitoring can better complete driving tasks, avoid traffic congestion, and improve traffic efficiency. However, related technologies for lane monitoring often rely on expensive vehicle monitoring equipment and a large amount of monitoring data, which increases the hardware cost of lane monitoring. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a lane monitoring method, device, and electronic device, which can determine the lane information corresponding to the vehicle's location by visually processing the road image of the vehicle's location and fusing the visual processing result with the road parameters; based on the lane information corresponding to the vehicle's location, the method can monitor the vehicle's switching between different lanes, obtain accurate and timely lane monitoring results, and acquire the road image of the vehicle's location at low cost, which is conducive to the promotion of autonomous driving technology.

[0004] The technical solution of this invention is implemented as follows:

[0005] This invention provides a lane monitoring method, comprising:

[0006] Obtain road images and vehicle location information of the vehicle's location;

[0007] The road parameters corresponding to the vehicle are obtained through the vehicle positioning information;

[0008] Visual processing is performed on the road image of the vehicle's location to determine a first visual processing result corresponding to the road image;

[0009] The first visual processing result corresponding to the road image is filtered to determine the second visual processing result corresponding to the road image.

[0010] By fusing the second visual processing result and the road parameters, the lane information corresponding to the vehicle's location is determined;

[0011] Based on the lane information corresponding to the vehicle's location, the system monitors the vehicle's switching between different lanes.

[0012] This invention also provides a lane monitoring device, comprising:

[0013] The information transmission module is used to acquire road images and vehicle positioning information of the vehicle's location;

[0014] The information processing module is used to obtain the road parameters corresponding to the vehicle through the vehicle positioning information;

[0015] The information processing module is used to perform visual processing on the road image of the vehicle's location and determine a first visual processing result corresponding to the road image.

[0016] The information processing module is used to filter the first visual processing result corresponding to the road image and determine the second visual processing result corresponding to the road image.

[0017] The information processing module is used to determine the lane information corresponding to the vehicle's location by fusing the second visual processing result and the road parameters.

[0018] The information processing module is used to monitor the vehicle switching between different lanes based on the lane information corresponding to the vehicle's location.

[0019] In the above scheme,

[0020] The information processing module is used to obtain map data that matches the vehicle positioning information through the vehicle positioning information;

[0021] The information processing module is used to perform data parsing and processing on the map data to obtain the road parameters corresponding to the vehicle, the road parameters corresponding to the vehicle including:

[0022] The number of lanes at the vehicle's location, the road attribute information at the vehicle's location, and the road monitoring information at the vehicle's location.

[0023] In the above scheme,

[0024] The information processing module is used to process the road image through a multi-channel invertible residual network in the road image processing model to obtain a first feature vector that matches the road image.

[0025] The information processing module is used to process the first feature vector through the squeezing excitation network in the road image processing model to obtain weight parameters and a second feature vector that match the road image.

[0026] The information processing module is used to segment the road image based on weight parameters and a second feature vector that match the road image, and through the activation function of the road image processing model and the corresponding convolutional neural network, to obtain the segmentation result of the road image.

[0027] The information processing module is used to reconstruct the segmentation results of the road image to obtain a first visual processing result corresponding to the road image.

[0028] In the above scheme,

[0029] The information processing module is used to perform weighted processing on the second feature vector corresponding to the road image based on the weight parameters matched by the road image, so as to obtain the intermediate feature information corresponding to the road image;

[0030] The information processing module is used to map the intermediate feature information into a probability vector corresponding to the segmentation result, and select the category corresponding to the maximum value of the probability vector as the segmentation result of the road image.

[0031] The segmentation results are used to determine the lane lines corresponding to the road image.

[0032] In the above scheme,

[0033] The information processing module is used to perform coordinate transformation processing on the segmentation results of the road image, and to perform lane line fitting and reconstruction processing on the coordinate transformation results to obtain lane line fitting and reconstruction results.

[0034] The information processing module is used to determine the leftmost lane number and the confidence level of the leftmost lane number corresponding to the vehicle based on the lane line fitting and reconstruction results.

[0035] The information processing module is used to determine the right lane number and the confidence level of the right lane number corresponding to the vehicle based on the lane line fitting and reconstruction results.

[0036] The information processing module is used to determine the lane line corresponding to the vehicle based on the lane line fitting and reconstruction results.

[0037] The information processing module is used to combine the left lane number, the confidence level of the left lane number, the right lane number, the confidence level of the right lane number, and the lane line corresponding to the vehicle to obtain a first visual processing result corresponding to the road image.

[0038] In the above scheme,

[0039] The information processing module is used to filter the left and right lane numbers in the first visual processing result when both the left lane number and the right lane number are 0, and to combine the confidence scores of the left lane number, the right lane number, and the lane line corresponding to the vehicle to obtain a second visual processing result corresponding to the road image.

[0040] The information processing module is used to filter the confidence scores of the left lane number and the right lane number when both the confidence scores of the left lane number and the right lane number are less than the confidence score threshold, and to combine the left lane number, the right lane number, and the lane line corresponding to the vehicle to obtain a second visual processing result corresponding to the road image.

[0041] The information processing module is used to filter the lane line corresponding to the vehicle when the display effect of the lane line is less than the display effect threshold, and to combine the left lane number, the right lane number, the confidence of the left lane number and the confidence of the right lane number to obtain a second visual processing result corresponding to the road image.

[0042] The information processing module is used to obtain the speed parameters of the vehicle. When the speed parameters of the vehicle are less than the speed threshold, the left lane number, right lane number, confidence level of the left lane number, confidence level of the right lane number, and lane line corresponding to the vehicle in the first visual processing result are discarded, and the second visual processing result corresponding to the road image is determined to be empty.

[0043] In the above scheme,

[0044] The information processing module is used to fuse the left lane number, the confidence level of the left lane number, the right lane number, the confidence level of the right lane number, the lane line corresponding to the vehicle, and the number of lanes at the vehicle's location in the road parameters in the second visual processing result to obtain the fusion processing result.

[0045] The information processing module is used to determine the lane information corresponding to the vehicle's location based on the value of the fusion processing result and the comparison result between the total number of visual lanes and the number of lanes at the vehicle's location in the road parameters.

[0046] In the above scheme,

[0047] The information processing module is used to determine the total number of visual lanes based on the left lane number and the right lane number when both the left lane number and the right lane number are not 0, and both the confidence scores of the left lane number and the right lane number are greater than the confidence threshold.

[0048] The information processing module is used to determine the lane information corresponding to the vehicle's location as the leftmost lane number when the total number of visual lanes is equal to the number of lanes at the vehicle's location in the road parameters.

[0049] In the above scheme,

[0050] The information processing module is used to determine the lane information corresponding to the vehicle's location as the left lane number when the total number of visual lanes is not equal to the number of lanes at the vehicle's location, the left lane number is less than or equal to the right lane number, and the confidence level of the left lane number is greater than or equal to the confidence level of the right lane number.

[0051] The information processing module is used to determine the lane information corresponding to the vehicle's location as the left lane number when the total number of visual lanes is not equal to the number of lanes at the vehicle's location, the left lane number is less than or equal to the right lane number, and the right lane number is greater than 3.

[0052] In the above scheme,

[0053] The information processing module is used to determine the left lane number based on the lane information corresponding to the vehicle's location and the right lane number when the total number of visual lanes is not equal to the number of lanes at the vehicle's location, the left lane number is greater than or equal to the right lane number, and the confidence level of the left lane number is less than or equal to the confidence level of the right lane number.

[0054] The information processing module is used to determine the left lane number based on the lane information corresponding to the vehicle's location and the right lane number when the total number of visual lanes is not equal to the number of lanes at the vehicle's location, the left lane number is greater than or equal to the right lane number, and the right lane number is greater than 3.

[0055] In the above scheme,

[0056] The information processing module is configured to perform the following operations when both the left-numbered lane number and the right-numbered lane number are not 0, and either the confidence level of the left-numbered lane number or the confidence level of the right-numbered lane number is greater than a confidence threshold:

[0057] The information processing module is used to determine the lane information corresponding to the vehicle's location as the left-hand lane number when the confidence level of the left-hand lane number is greater than a confidence threshold.

[0058] The information processing module is used to determine the left lane number based on the lane information corresponding to the vehicle's location and the right lane number when the confidence level of the right lane number is greater than the confidence level threshold.

[0059] In the above scheme,

[0060] The information processing module is configured to perform the following operations when the left lane number is not 0, or when neither of the right lane numbers is 0:

[0061] When the leftmost lane number is greater than 0, the lane information corresponding to the vehicle's location is determined to be the leftmost lane number;

[0062] When the right lane number is greater than 0, the left lane number is determined based on the lane information corresponding to the vehicle's location and the right lane number.

[0063] In the above scheme,

[0064] The information processing module is used to determine a buffer range that matches the environment of the vehicle;

[0065] The information processing module is used to perform median filtering on the second visual processing result in the cache interval to obtain the median filtering result.

[0066] The information processing module is used to determine the lane information corresponding to the vehicle's location at different monitoring times based on the median filtering results.

[0067] The information processing module is used to determine the switching status of the vehicle in different lanes based on the lane information corresponding to the vehicle's location in two consecutive monitoring times.

[0068] This invention also provides an electronic device, the electronic device comprising:

[0069] Memory, used to store executable instructions;

[0070] The processor, when running the executable instructions stored in the memory, implements the aforementioned lane monitoring method.

[0071] This invention also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the aforementioned lane monitoring method.

[0072] The embodiments of the present invention have the following beneficial effects:

[0073] This invention acquires road images and vehicle positioning information of a vehicle's location; obtains road parameters corresponding to the vehicle using the vehicle positioning information; performs visual processing on the road image of the vehicle's location to determine a first visual processing result corresponding to the road image; filters the first visual processing result to determine a second visual processing result corresponding to the road image; and determines lane information corresponding to the vehicle's location through fusion processing of the second visual processing result and the road parameters; based on the lane information corresponding to the vehicle's location, it monitors the vehicle's switching between different lanes. This not only achieves lane-level positioning of the vehicle using the lane information corresponding to its location, enabling accurate monitoring of the vehicle's switching between different lanes, but also utilizes a simple and low-cost device structure for acquiring road images of the vehicle's location, which is beneficial for the promotion of autonomous driving technology. Therefore, this solution can be applied to fields including, but not limited to, autonomous driving, vehicle networking, and intelligent transportation. Attached Figure Description

[0074] Figure 1 This is a schematic diagram illustrating the application environment of a lane monitoring method provided in an embodiment of the present invention;

[0075] Figure 2 A schematic diagram of the composition structure of an electronic device provided in an embodiment of the present invention;

[0076] Figure 3 This is an optional flowchart illustrating the lane monitoring method provided in an embodiment of the present invention;

[0077] Figure 4 This is a schematic diagram of the vehicle coordinate system in an embodiment of the present invention;

[0078] Figure 5 This is a schematic diagram of the road parameters corresponding to the vehicle in an embodiment of the present invention;

[0079] Figure 6 This is a schematic flowchart illustrating the process of obtaining the first visual processing result in the lane monitoring method provided in this embodiment of the invention.

[0080] Figure 7 This is a schematic diagram of the SE-Block structure in the lane monitoring method provided in this embodiment of the invention.

[0081] Figure 8 This is a schematic diagram of the RevNet structure working in the lane monitoring method provided in this embodiment of the invention;

[0082] Figure 9 This is a schematic diagram of the process for obtaining the first visual processing result in the lane monitoring method provided in this embodiment of the invention;

[0083] Figure 10This is a schematic diagram of the lane line fitting and reconstruction results in an embodiment of the present invention;

[0084] Figure 11 This is a schematic diagram of the lane line fitting and reconstruction results in an embodiment of the present invention;

[0085] Figure 12 This is a schematic diagram of the first visual processing result filtering process provided in an embodiment of the present invention;

[0086] Figure 13 This is a flowchart illustrating the process of determining lane information corresponding to the location of a vehicle, provided in an embodiment of the present invention.

[0087] Figure 14 This is a schematic diagram illustrating the monitoring of a vehicle switching between different lanes in an embodiment of the present invention. Detailed Implementation

[0088] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0089] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0090] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0091] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0092] 1) Responding to: used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0093] 2) Confidence level: The confidence interval of a probability sample is an interval estimate of a population parameter for that sample. The confidence interval shows the degree to which the true value of this parameter has a certain probability of falling around the measured result. The confidence interval obtains the range of confidence regarding the measured value of the measured parameter.

[0094] 3) Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that include convolutional computations and have a deep structure. They are one of the representative algorithms of deep learning. CNNs have representation learning capabilities and can perform shift-invariant classification of input information according to their hierarchical structure.

[0095] 4) Soft max: Normalized exponential function, a generalization of the logistic function. It can "compress" a K-dimensional vector containing arbitrary real numbers into another K-dimensional real vector, such that each element is in the range [0, 1], and the sum of all elements is 1.

[0096] Before introducing the lane monitoring method provided in this application, a brief explanation of lane monitoring in related technologies is provided. In the implementation of autonomous driving technology, lane monitoring can be achieved in the following ways: 1) Based on real-time kinematic (RTK) carrier phase differential technology, the difference between the carrier phase observations of two measurement stations is processed in real time. The carrier phase collected by the base station is sent to the user receiver for differential calculation of coordinates, achieving centimeter / decimeter-level positioning accuracy. However, RTK-based positioning technology is highly dependent on equipment and also requires high-precision map data for positioning matching. The cost of acquiring high-precision maps is too high, hindering large-scale deployment. 2) Based on sensor deployment (geomagnetic sensing), vehicle position perception and recognition can be achieved. However, this method is costly, impractical for road modifications, and not suitable for large-scale deployment. 3) Based on lidar ranging and 3D point cloud feature scanning, accurate vehicle position tracking can be achieved. However, the millimeter-wave radar required for lidar ranging and 3D point cloud feature scanning is expensive, hindering large-scale deployment.

[0097] Figure 1 This is a schematic diagram illustrating a usage scenario of the lane monitoring method provided in an embodiment of the present invention. See also: Figure 1The terminals (including terminal 10-1 and vehicle-mounted terminal 10-2) are equipped with a corresponding client capable of performing lane monitoring functions. The acquisition of road images of the vehicle's location can be achieved using a monocular camera. The monocular camera can be installed in different locations depending on the vehicle's shape; for example, it can be installed on the windshield, in the center of the air intake grille (replacing existing millimeter-wave radar), or on the outer edge of the front of the roof (replacing binocular or tri-lens cameras). Furthermore, the monocular camera can also be replaced by other devices with image acquisition capabilities (such as a dashcam) to save on hardware costs for acquiring road images of the vehicle's location. After acquiring the road images and vehicle positioning information, they can be stored in a server or cloud server cluster for use by the vehicle-mounted terminal 10-2. Alternatively, the server or cloud server cluster can execute the lane monitoring method provided in this application and transmit the monitoring results of the vehicle switching between different lanes to the vehicle-mounted terminal 10-2. The terminal connects to the lane monitoring device 0 via network 300, which can be a wide area network, a local area network, or a combination of both. Data transmission is achieved using a wireless link. After acquiring road images and vehicle positioning information of the vehicle's location, the vehicle-mounted terminal 10-2 can obtain the road parameters corresponding to the vehicle using the vehicle positioning information. It then performs visual processing on the road image of the vehicle's location to determine a first visual processing result corresponding to the road image. The first visual processing result is then filtered to determine a second visual processing result corresponding to the road image. The second visual processing result and the road parameters are fused to determine the lane information corresponding to the vehicle's location. Based on the lane information corresponding to the vehicle's location, the system monitors the vehicle's switching between different lanes.

[0098] The structure of the lane monitoring device according to an embodiment of the present invention will be described in detail below. The lane monitoring device can be implemented in various forms, such as a dedicated terminal with image processing and data calculation functions, or a vehicle network server with image processing and data processing functions, as described above. Figure 1 Lane monitoring device 0. Figure 2 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of the present invention. It can be understood that... Figure 2 This is only an exemplary structure of the lane monitoring device, not the entire structure; it can be implemented as needed. Figure 2 The structure shown may be part or all of the structure.

[0099] The lane monitoring device provided in this embodiment of the invention includes at least one processor 201, a memory 202, a user interface 203, and at least one network interface 204. The various components in the lane monitoring device are coupled together via a bus system 205. It can be understood that the bus system 205 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 205 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 205.

[0100] The user interface 203 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0101] It is understood that memory 202 can be volatile memory or non-volatile memory, or both. In this embodiment of the invention, memory 202 is capable of storing data to support the operation of a terminal (such as 10-1). Examples of this data include any computer programs used to operate on the terminal (such as 10-1), such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0102] In some embodiments, the lane monitoring device provided in this invention can be implemented using a combination of hardware and software. For example, the lane monitoring device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the lane monitoring method provided in this invention. For instance, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0103] As an example of the lane monitoring device provided in this embodiment of the invention, which is implemented by combining software and hardware, the lane monitoring device provided in this embodiment of the invention can be directly embodied as a combination of software modules executed by processor 201. The software modules can be located in a storage medium, which is located in memory 202. Processor 201 reads the executable instructions included in the software modules in memory 202 and combines them with necessary hardware (e.g., including processor 201 and other components connected to bus 205) to complete the lane monitoring method provided in this embodiment of the invention.

[0104] As an example, processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0105] As an example of the hardware implementation of the lane monitoring device provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor 201 in the form of a hardware decoding processor. For example, it can be executed by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the lane monitoring method provided in this embodiment of the invention.

[0106] In this embodiment of the invention, the memory 202 is used to store various types of data to support the operation of the lane monitoring device. Examples of such data include: any executable instructions for operating on the lane monitoring device, such as executable instructions that can be included in a program implementing the lane monitoring method of this embodiment of the invention.

[0107] In other embodiments, the lane monitoring device provided in this invention can be implemented in software. Figure 2A lane monitoring device 2020 stored in memory 202 is shown. This device can be software in the form of programs and plugins, and includes a series of modules. As an example of a program stored in memory 202, it may include the lane monitoring device 2020. The lane monitoring device 2020 includes the following software modules: an information transmission module 2081 and an information processing module 2082. When the software modules in the lane monitoring device 2020 are read into RAM by processor 201 and executed, the lane monitoring method provided in this embodiment of the invention will be implemented. The functions of each software module in the lane monitoring device 2020 are described below:

[0108] The information transmission module 2081 is used to acquire road images and vehicle positioning information of the vehicle's location.

[0109] The information processing module 2082 is used to obtain the road parameters corresponding to the vehicle through the vehicle positioning information.

[0110] The information processing module 2082 is used to perform visual processing on the road image of the location of the vehicle and determine a first visual processing result corresponding to the road image.

[0111] The information processing module 2082 is used to filter the first visual processing result corresponding to the road image and determine the second visual processing result corresponding to the road image.

[0112] The information processing module 2082 is used to determine the lane information corresponding to the vehicle's location by fusing the second visual processing result and the road parameters.

[0113] The information processing module 2082 is used to monitor the vehicle switching between different lanes based on the lane information corresponding to the vehicle's location.

[0114] according to Figure 2 The electronic device shown, in one aspect of this application, also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform various embodiments and combinations of embodiments provided in the various optional implementations of the lane monitoring method described above.

[0115] Continue to combine Figure 2 The lane monitoring device shown illustrates the lane monitoring method provided in the embodiments of the present invention. Figure 3 This is an optional flowchart illustrating the lane monitoring method provided in an embodiment of the present invention. It can be understood that... Figure 3 The lane monitoring method shown can be applied to in-vehicle terminals or vehicle network servers to accurately monitor vehicles switching between different lanes. Figure 3 The steps shown can be performed by an in-vehicle terminal or a vehicle-to-everything (V2X) server, such as a dedicated monocular camera with image processing capabilities (including data calculation functions), a V2X server, or a server cluster. The following section addresses... Figure 3 The steps shown are explained.

[0116] Step 301: The lane monitoring device acquires road images and vehicle positioning information of the vehicle's location.

[0117] refer to Figure 4 , Figure 4 This is a schematic diagram of the vehicle coordinate system in an embodiment of the present invention. To monitor the lane the vehicle is in during operation, a vehicle coordinate system (VCS) needs to be established. The vehicle coordinate system is a special three-dimensional moving coordinate system O-xyz used to describe the motion of the vehicle. The origin O of the vehicle coordinate system coincides with the center of mass. When the vehicle is stationary on a horizontal road surface, the X-axis is parallel to the ground and points forward, the Y-axis points to the driver's left, and the Z-axis passes through the vehicle's center of mass and points upward. The center of the vehicle's rear axle is the origin of the coordinate system. It should be noted that... Figure 4 This is merely an illustrative example of a vehicle coordinate system. When implementing the lane monitoring method of this application, different vehicle coordinate systems (e.g., left-hand system, right-hand system) may be established according to the specific state of the vehicle and the actual road conditions. This application does not impose any specific restrictions on this.

[0118] The acquisition of road images of the vehicle's location can be achieved using a monocular camera. The monocular camera can be installed in various locations depending on the vehicle's shape; for example, it can be mounted on the windshield, in the center of the air intake grille (replacing existing millimeter-wave radar), or on the outer edge of the front of the roof (replacing binocular or tri-lens cameras). For terminals with map navigation capabilities, the monocular camera can also be replaced by other devices with image acquisition functions (such as a dashcam), saving on hardware costs for acquiring road images of the vehicle's location. This allows the lane monitoring method provided in this application to be used in older vehicles where monocular cameras cannot be installed, resulting in a better user experience.

[0119] When acquiring vehicle location information, the vehicle's position can be monitored in real time using the GPS or BeiDou positioning module carried in the vehicle. Alternatively, historical state information collected during historical positioning periods can be tracked. This historical state information includes, but is not limited to, GPS information, vehicle control information, vehicle visual perception information, and IMU (Inertial Measurement Unit) information. Based on the collected historical information, a current location point information P is calculated and output, such as the vehicle's latitude and longitude coordinates.

[0120] Step 302: The lane monitoring device obtains the road parameters corresponding to the vehicle through the vehicle positioning information.

[0121] In some embodiments of the present invention, obtaining the road parameters corresponding to the vehicle through the vehicle positioning information can be achieved in the following ways:

[0122] Using the vehicle location information, map data matching the vehicle location information is obtained; the map data is parsed to obtain the road parameters corresponding to the vehicle; wherein, the road parameters corresponding to the vehicle include at least one of the following: the number of lanes at the vehicle's location, the road attribute information at the vehicle's location, and the road monitoring information at the vehicle's location. When obtaining the road parameters corresponding to the vehicle, the corresponding road position of the vehicle can be determined based on the vehicle's location information, and then the map information corresponding to the current position can be obtained. The total number of lanes at the current position is obtained from the map information corresponding to the current position and recorded as Total Lane Cnt, along with the road attribute information, to determine whether the vehicle's current position is near an intersection, a highway ramp, or a main / auxiliary road. (Reference) Figure 5 , Figure 5 This is a schematic diagram of the road parameters corresponding to the vehicle in an embodiment of the present invention, such as... Figure 5 As shown, based on the location information of vehicle 500, the corresponding road position of vehicle 500 is determined, and then the map information corresponding to the current position is obtained. From the map information corresponding to the previous position, the total number of lanes at the current position is obtained as 2. At the same time, the map data can be used to determine... Figure 5 The signage at the intersection, or the determination Figure 5 The road shown has a turn sign, indicating that there is a fork in the road and a highway ramp ahead of vehicle 500.

[0123] Step 303: The lane monitoring device performs visual processing on the road image of the vehicle's location to determine the first visual processing result corresponding to the road image.

[0124] The determination of the first visual processing result corresponding to the road image can be achieved through a road image processing model provided by artificial intelligence technology, which is pre-trained and deployed in the vehicle-to-everything (V2X) server. Artificial intelligence is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0125] The deep learning used in road image processing models can include artificial neural networks, such as convolutional neural networks (CNN), recurrent neural networks (RNN), and deep neural networks (DNN).

[0126] In some embodiments of the present invention, reference is made to Figure 6 , Figure 6 This is a schematic flowchart illustrating the process of obtaining the first visual processing result in the lane monitoring method provided in this embodiment of the invention. It can be understood that... Figure 6 The lane monitoring method shown can be applied to in-vehicle terminals or vehicle network servers to accurately monitor vehicles switching between different lanes. Figure 6 The steps shown can be performed by a road image processing model deployed in an in-vehicle terminal or a vehicle-to-everything (V2X) server. The following section addresses... Figure 6 The steps shown are explained.

[0127] Step 601: Process the road image using a multi-channel invertible residual network in the road image processing model to obtain a first feature vector that matches the road image.

[0128] Before introducing the network structure of the road image processing model, it is necessary to first briefly introduce the processing procedures of SE-Block and RevNet, referring to... Figure 7 and Figure 7 , Figure 7This is a schematic diagram of the SE-Block structure in the lane monitoring method provided in this embodiment of the invention. SE-Block calculates the weighting for each channel through a series of pooling and activation functions, and finds the optimal weights through a loss function and backpropagation. Higher weights are assigned to some more meaningful feature channels, while lower weights are assigned to noisy channels. This better integrates information to achieve the best classification result. Therefore, by adaptively weighting different channels, the features of important channels are amplified while the features of noisy channels are reduced, thereby improving classification accuracy.

[0129] Figure 8 This is a schematic diagram of the RevNet structure in the lane monitoring method provided in this embodiment of the invention. (a) and (b) show the data processing process of the residual network. RevNet changes the previous method of processing images through layer downsampling and irreversible convolutional layers. It introduces reversible RevNet Blocks for image processing, making each step of the operation reversible, thereby preserving all information and reducing the variability of the network caused by information loss. At the same time, it does not need to store a large number of intermediate results, which greatly reduces the memory requirements. Using the trained F and G functions, the outputs y1 and y2 can be calculated from the inputs x1 and x2, and the inputs x1 and x2 can be calculated from the outputs y1 and y2. Moreover, no additional inversion operation is required during backpropagation. Therefore, it is not necessary to store the results such as x1+F(x2) generated in the intermediate process, thus saving a lot of space.

[0130] The Multi-Layer Reversible Residual Network (MLRB) consists of convolutions between different channels, with both input and output computed using the same convolutions, achieving reversibility. The activations of each layer can be computed from the activations of the next layer, enabling backpropagation without storing activations in memory. The result is a network architecture where activation storage requirements are depth-independent and typically at least an order of magnitude smaller than ResNets of the same size.

[0131] Step 602: The first feature vector is processed by the squeezing excitation network in the road image processing model to obtain weight parameters and a second feature vector that match the road image.

[0132] In some embodiments of the present invention, when the number of layers in the multi-channel reversible residual network of the road image processing model is more than one, the activation result is processed by the classification layer in the squeeze excitation network to determine a third feature vector that matches the road image; the third feature vector is iteratively processed by the reversible residual network layer and convolutional layer corresponding to each layer in the multi-channel reversible residual network until the activation result is classified by the classification layer in the squeeze excitation network to obtain the transformed weight parameters and second feature vector that match the road image.

[0133] In this network, the pooling layer can be a global pooling layer, used to pool each channel of the image to a 1*1 size, i.e., calculating the average value of each channel. For each level of the multi-channel reversible residual network and its corresponding convolutional layer, the corresponding global pooling layer of the squeeze excitation network can be connected. This means that the processing results of each level of the multi-channel reversible residual network are input into the corresponding squeeze excitation network for processing, calculating the weights for each channel. The optimal weights are then found through a loss function and backpropagation, integrating the weight parameters of different channels to achieve the best segmentation result and improve segmentation accuracy.

[0134] Step 603: Based on the weight parameters and second feature vector that match the road image, and through the activation function of the road image processing model and the corresponding convolutional neural network, the road image is segmented to obtain the segmentation result of the road image.

[0135] In the process of segmenting a road image to obtain the segmentation result, the weight parameters that match the road image can be obtained first. Based on the weight parameters that match the road image, the second feature vector corresponding to the road image is weighted to obtain the intermediate feature information corresponding to the road image. The intermediate feature information is mapped to the probability vector corresponding to the segmentation result, and the category corresponding to the maximum value of the probability vector is selected as the segmentation result of the road image. The segmentation result is used to determine the lane lines corresponding to the road image.

[0136] Step 604: Reconstruct the segmentation results of the road image to obtain a first visual processing result corresponding to the road image.

[0137] To further illustrate the process of reconstructing the segmentation results of the road image to obtain the first visual processing result corresponding to the road image, refer to... Figure 9 , Figure 9This is a schematic flowchart illustrating the process of obtaining the first visual processing result in the lane monitoring method provided in this embodiment of the invention. The process of reconstructing the segmentation result of the road image includes the following steps:

[0138] Step 6041: Perform coordinate transformation processing on the segmentation result of the road image, and perform lane line fitting and reconstruction processing on the coordinate transformation result to obtain the lane line fitting and reconstruction result.

[0139] Step 6042: Based on the lane line fitting and reconstruction results, determine the leftmost lane number and the confidence level of the leftmost lane number corresponding to the vehicle.

[0140] Step 6043: Based on the lane line fitting and reconstruction results, determine the right lane number and the confidence level of the right lane number corresponding to the vehicle.

[0141] Step 6044: Based on the lane line fitting and reconstruction results, determine the lane line corresponding to the vehicle.

[0142] In the field of map navigation, lane-level positioning of vehicles is fundamental to achieving accurate navigation. Lane-level positioning is crucial for determining the vehicle's lateral position and formulating navigation strategies. Based on the results of lane-level positioning, lane-level path planning and guidance can also be performed. Through steps 6041-6044, the lane line fitting and reconstruction results during vehicle movement can be obtained, as referenced... Figure 10 , Figure 10 This is a schematic diagram of the lane line fitting and reconstruction results in an embodiment of the present invention. The obtained first visual processing results include: the leftmost lane number, the confidence score of the leftmost lane number, the rightmost lane number, the confidence score of the rightmost lane number, and the lane line corresponding to the vehicle. Based on the obtained parameters, lane line equation information can be constructed. The lane line equation can be expressed as a quadratic polynomial, a cubic polynomial, or other forms, such as:

[0143] y=d+a*x+b*x^2+c*x^3 or y=d+a*x+b*x^2

[0144] Where a, b, c, and d are the fitting coefficients of the polynomial, and d represents the value of y when x=0. In a physical sense, d represents the distance from the vehicle to the lane line, with positive values ​​on the left and negative values ​​on the right. This result is used to detect vehicle lane changes. Figure 10The diagram illustrates a two-dimensional Cartesian coordinate system O-xy (the vehicle's 3D coordinate system excluding the z-axis). The origin O is located at the center of the vehicle's rear axle. The X-axis is parallel to the ground and points forward, while the Y-axis points to the driver's left. L1 and L2 represent the first and second lane lines closest to the vehicle's left, respectively, while R1 and R2 represent the first and second lane lines closest to the vehicle's right, respectively. Based on the fitting results, the intercept information d_L1, d_L2, d_R1, and d_R2 from each lane line to the vehicle can be obtained. When monitoring the vehicle's lane, it is recommended to track only the two closest lane lines, L1 and R1.

[0145] Step 6045: Combine the left lane number, the confidence level of the left lane number, the right lane number, the confidence level of the right lane number, and the lane line corresponding to the vehicle to obtain a first visual processing result corresponding to the road image.

[0146] refer to Figure 11 , Figure 11 This is a schematic diagram of the lane line fitting and reconstruction results in an embodiment of the present invention, wherein, for a 4-lane road monitoring scenario, Figure 11 The solid lines on the left and right sides represent the road edges. For the current vehicle, Figure 11 The top section shows the left-counting rule, counting from the first lane on the left to the current lane, resulting in lane 2 on the left. The bottom section shows the right-counting rule, counting from the first lane on the right to the current lane, resulting in lane 3 on the right. When implementing the lane detection method of this application, `visLeft` and `visConfLeft` can represent the left-counting lane number and its confidence level obtained through visual processing, and `visRight` and `visConfRight` can represent the right-counting lane number and its confidence level obtained through visual processing. `visLineLeft` and `visLineRight` represent the left and right lane lines of the vehicle obtained visually. Due to varying road complexities in actual use, in situations where the current lane number cannot be distinguished, such as at intersections or due to occlusion, both `visLeft` and `visRight` can be set to 0 to indicate that it cannot be determined. In some street scenarios where one side is occluded and the other is not, one of `visLeft` and `visRight` can be set to 0, while the other remains non-zero. It should be noted that the lane monitoring method provided in this application can adapt to different traffic rules (for example, different traffic rules allow vehicles to change lanes in different directions for overtaking), thereby increasing the practicality and reliability of the lane monitoring method in this application.

[0147] Step 304: The lane monitoring device filters the first visual processing result corresponding to the road image to determine the second visual processing result corresponding to the road image.

[0148] Figure 12 This is a schematic diagram of the first visual processing result filtering process provided in an embodiment of the present invention. It can be understood that... Figure 12 The lane monitoring method shown can be applied to in-vehicle terminals or vehicle network servers to accurately monitor vehicles switching between different lanes. Figure 12 The steps shown can be performed by an in-vehicle terminal or a vehicle-to-everything (V2X) server, such as a dedicated monocular camera with image processing capabilities (including data calculation functions), a V2X server, or a server cluster. The following section addresses... Figure 10 The steps shown are explained.

[0149] Step 1201: When both the left lane number and the right lane number are 0, filter the left lane number and the right lane number in the first visual processing result, and combine the confidence scores of the left lane number, the confidence scores of the right lane number, and the lane line corresponding to the vehicle to obtain the second visual processing result corresponding to the road image.

[0150] During vehicle operation, if both visLeft and visRight are 0, it may include at least one of the following situations: 1) The vehicle is traveling in a one-way street with only one lane; 2) Other vehicles in the adjacent lane are obstructing the view of the current vehicle; 3) The driver's view is obstructed (due to adverse weather or lighting conditions) and the driver cannot visually determine the vehicle's current position. Therefore, if both visLeft and visRight are 0, it means that the image captured by the monocular camera cannot be used to determine the absolute position of the vehicle in the lane, nor can it monitor lane changes.

[0151] Step 1202: When the confidence scores of the left lane number and the right lane number are both less than the confidence threshold, the confidence scores of the left lane number and the right lane number are filtered, and the left lane number, the right lane number, and the lane line corresponding to the vehicle are combined to obtain a second visual processing result corresponding to the road image.

[0152] Different confidence thresholds visThrehold can be set according to different vehicle types and road information. When at least one of visLeft and visRight is not 0, and the confidence of at least one non-zero value is greater than visThrehold, the confidence of the left lane number and the confidence of the right lane number can be used.

[0153] Meanwhile, if the confidence scores of both the left and right lane numbers are less than the confidence threshold, then the confidence scores of the left and right lane numbers are discarded, and only the left and right lane numbers, as well as the lane line corresponding to the vehicle, are used.

[0154] Step 1203: When the display effect of the lane line corresponding to the vehicle is less than the display effect threshold, the lane line corresponding to the vehicle is filtered, and the left lane number, the right lane number, the confidence of the left lane number and the confidence of the right lane number are combined to obtain the second visual processing result corresponding to the road image.

[0155] Specifically, when the displayed effect of the lane line corresponding to the vehicle is less than the display effect threshold, it means that the lane line corresponding to the vehicle cannot be recognized. Changing lanes at this time would be dangerous. Therefore, it is advisable to continue driving.

[0156] Step 1204: Obtain the speed parameters of the vehicle. When the speed parameters of the vehicle are less than the speed threshold, discard the left lane number, right lane number, confidence score of the left lane number, confidence score of the right lane number, and lane line corresponding to the vehicle in the first visual processing result, and determine that the second visual processing result corresponding to the road image is empty.

[0157] Specifically, when the vehicle speed parameter is less than the speed threshold, it means that lane change cannot be performed. Therefore, the left lane number, right lane number, confidence level of the left lane number, confidence level of the right lane number, and lane line corresponding to the vehicle in the first visual processing result are discarded, and lane monitoring can be retried when the vehicle speed parameter is greater than or equal to the speed threshold.

[0158] When passing Figure 12 The processing steps shown enable filtering of the first visual processing result corresponding to the road image to obtain the second visual processing result corresponding to the road image, after which step 305 can be executed.

[0159] Step 305: The lane monitoring device determines the lane information corresponding to the vehicle's location by fusing the second visual processing result and the road parameters.

[0160] In some embodiments of the present invention, the second visual processing result and the road parameters are fused to obtain the lane information corresponding to the vehicle's location, which can be achieved in the following ways:

[0161] The left lane number, left lane number confidence score, right lane number, right lane number confidence score, lane line corresponding to the vehicle, and number of lanes at the vehicle's location in the road parameters are fused together to obtain a fusion processing result. Based on the value of the fusion processing result and the comparison between the total number of visual lanes and the number of lanes at the vehicle's location in the road parameters, the lane information corresponding to the vehicle's location is determined.

[0162] Figure 13 This is a flowchart illustrating the process of determining lane information corresponding to the location of a vehicle, provided in an embodiment of the present invention. It can be understood that... Figure 13 The lane monitoring method shown can be applied to in-vehicle terminals or vehicle network servers to accurately monitor vehicles switching between different lanes. Figure 13 The steps shown specifically include:

[0163] Step 1301: Determine whether both the left lane number and the right lane number are not 0, and whether the confidence scores of the left lane number and the right lane number are both greater than the confidence threshold. If so, proceed to step 1302; otherwise, proceed to step 1303.

[0164] Step 1302: Determine the total number of visual lanes based on the left-numbered lane number and the right-numbered lane number; when the total number of visual lanes is equal to the number of lanes at the vehicle's location in the road parameters, determine the lane information corresponding to the vehicle's location as the left-numbered lane number.

[0165] When both visLeft and visRight are non-zero, and both visConfLeft and visConfRight are greater than the threshold visThrehol, the total visual lane count visLaneCnt can be calculated as visLeft + visRight – 1, and compared with TotalLaneCnt obtained from map data. If they are equal, then laneIndexLeft = visLeft. If they are not equal, the following cases should be considered:

[0166] 1) If visLeft <= visRight, and visConfLeft >= visConfRight or visRight > 3, then laneIndexLeft = visLeft.

[0167] 2) If visLeft>= visRight, and visConfLeft<= visConfRight or visLeft>3, then laneIndexLeft = TotalLaneCnt – visRight + 1.

[0168] 3) If 1) and 2) are not satisfied, then laneIndexLeft = visLeft.

[0169] Step 1303: Determine whether the following conditions are met: the total number of lanes in vision is not equal to the number of lanes at the location of the vehicle, the left lane number is less than or equal to the right lane number, and the confidence level of the left lane number is greater than or equal to the confidence level of the right lane number. If yes, proceed to step 1304; otherwise, proceed to step 1305.

[0170] Step 1304: Determine the lane information corresponding to the vehicle's location as the left-numbered lane number; when the total number of visual lanes is not equal to the number of lanes at the vehicle's location, the left-numbered lane number is less than or equal to the right-numbered lane number, and the right-numbered lane number is greater than 3, determine the lane information corresponding to the vehicle's location as the left-numbered lane number.

[0171] Step 1305: Determine whether the following conditions are met: the total number of lanes in vision is not equal to the number of lanes at the location of the vehicle, the left lane number is greater than or equal to the right lane number, and the confidence level of the left lane number is less than or equal to the confidence level of the right lane number. If yes, proceed to step 1306; otherwise, proceed to step 1307.

[0172] Step 1306: Determine the left lane number based on the lane information corresponding to the vehicle's location and the right lane number; when the total number of visual lanes is not equal to the number of lanes at the vehicle's location, the left lane number is greater than or equal to the right lane number, and the right lane number is greater than 3, determine the left lane number based on the lane information corresponding to the vehicle's location and the right lane number.

[0173] Step 1307: Determine whether the left lane number and the right lane number are both not 0, and the confidence level of the left lane number, or the confidence level of the right lane number, is greater than the confidence level threshold. If yes, proceed to step 1308; otherwise, proceed to step 1309.

[0174] Step 1308: When the confidence level of the left lane number is greater than the confidence threshold, the lane information corresponding to the vehicle's location is the left lane number; or when the confidence level of the right lane number is greater than the confidence threshold, the left lane number is determined based on the lane information corresponding to the vehicle's location and the right lane number.

[0175] Step 1309: Determine whether the condition is met that the left lane number is not 0 or the right lane number is not 0. If yes, proceed to step 1310; otherwise, return to step 1301 and continue monitoring the vehicle's lane.

[0176] Step 1310: When the left lane number is greater than 0, determine the lane information corresponding to the vehicle's location as the left lane number; or when the right lane number is greater than 0, determine the left lane number based on the lane information corresponding to the vehicle's location and the right lane number.

[0177] Specifically, when both visLeft and visRight are not 0, but only one of visConfLeft and visConfRight is greater than the threshold visThrehold, two cases can be handled:

[0178] a.) visConfLeft>visThrehold: laneIndexLeft = visLeft;

[0179] b.) visConfRight>visThrehold: laneIndexLeft = TotalLaneCnt –visRight + 1.

[0180] Furthermore, when only one of visLeft and visRight is not zero, it can be determined from the filtering process of the first visual processing result in the previous embodiment that the threshold of this non-zero result must be greater than visThrehold. Therefore, we can continue processing in two cases:

[0181] a.) visLeft>0: laneIndexLeft = visLeft;

[0182] b.) visRight>0: laneIndexLeft = TotalLaneCnt – visRight + 1.

[0183] After determining the lane information corresponding to the vehicle's location, proceed to step 306.

[0184] Step 306: The lane monitoring device monitors the vehicle's switching between different lanes based on the lane information corresponding to the vehicle's location.

[0185] When monitoring a vehicle switching between different lanes, the following methods can be used:

[0186] A buffer interval matching the vehicle's environment is determined; the second visual processing result within the buffer interval is subjected to median filtering to obtain a median filtering result; based on the median filtering result, lane information corresponding to the vehicle's location at different monitoring times is determined; based on the lane information corresponding to the vehicle's location at two consecutive monitoring times, the switching state of the vehicle in different lanes is determined. (Referring to...) Figure 14 , Figure 14 This is a schematic diagram illustrating the monitoring of vehicle lane switching in this embodiment of the invention. When monitoring vehicle lane switching, a buffer register of length N can be configured to cache information from N previous lane index left values. The value of N depends on the acquisition frequency of the road image at the vehicle's location and the required buffering time. For example, if the frequency is 10Hz and the buffering time is 1s, then N = 10 * 1 = 10. During this process, due to potential occlusion and recognition errors during road image acquisition, the lane index left result in the buffer may exhibit unnecessary jumps. In this case, a median filter is applied to the result in the buffer to remove abnormal short-term jumps, ultimately outputting the tracking result lane index Trace Left. We also retain the tracking result from the previous moment, which is lastLane index Trace Left.

[0187] In some embodiments of the present invention, when monitoring lane lines, the lateral displacement of the vehicle is tracked by tracking the information of the left and right lane lines (lane line equations). This allows for the identification of lane line crossing information, i.e., lane change actions. The d-value (lane intercept) of the lane line equation represents the distance from the current vehicle to the lane line. The lane line tracking result is named laneChange, with values ​​of -1, 0, and +1, representing a lane change to the left (-1), a lane change to the right (+1), and no lane change (0).

[0188] like Figure 14 As shown, based on the lane information corresponding to the vehicle's location at two consecutive monitoring times, when determining the vehicle's switching state in different lanes, local map data can be used to determine whether the current vehicle is in a driving area where the number of lanes may jump, such as intersections, main and auxiliary road areas, and highway ramp areas. If it is determined that the current vehicle is in a driving area where the number of lanes may jump, the laneIndexTraceLeft output can be used as the final result of lane-level localization at the current time. Otherwise, further processing is performed.

[0189] 1) If the difference between laneIndexTraceLeft and lastLaneIndexTraceLeft is greater than or equal to 2, that is:

[0190] | laneIndexTraceLeft – lastLaneIndexTraceLeft | >= 2 indicates that in a non-special area, the vehicle changes between two adjacent moments, which does not match the actual situation. In this case, let laneIndexTraceLeft = lastLaneIndexTraceLeft, and continue to output the result of the previous moment.

[0191] 2) If the values ​​of laneIndexTraceLeft and lastLaneIndexTraceLeft differ by 1, it indicates that a lane change has occurred. Let diff = laneIndexTraceLeft – lastLaneIndexTraceLeft. At this point, check the result of the lane line tracking module. If diff and laneChange are the same, output laneIndexTraceLeft; otherwise, let laneIndexTraceLeft = lastLaneIndexTraceLeft and continue outputting the result from the previous moment.

[0192] The lane monitoring method provided in this application, such as Figure 14 As shown, when monitoring vehicles switching between different lanes, the number of cars using the lane monitoring method provided in this application is constantly increasing. Therefore, to reduce the data processing pressure on the vehicle network server, the road image and vehicle positioning information of the vehicle's location can be stored in a blockchain network or cloud server to monitor vehicles switching between different lanes. This invention can be implemented using cloud technology or blockchain network technology. Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network or local area network to achieve data computation, storage, processing, and sharing. It can also be understood as a general term for network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business models. The backend services of technical network systems require a large amount of computing and storage resources, such as video websites, image websites, and many portal websites; therefore, cloud technology needs to be supported by cloud computing.

[0193] It's important to note that cloud computing is a computing model that distributes computing tasks across a resource pool comprised of numerous computers, enabling various application systems to access computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, resources in the "cloud" are infinitely scalable, readily available, and can be used on demand, expanded at any time, and paid for based on usage. As the foundational providers of cloud computing capabilities, they establish cloud resource pool platforms, often referred to as cloud platforms or Infrastructure as a Service (IaaS). These platforms deploy various types of virtual resources within the resource pool for external customers to choose from. The cloud resource pool primarily includes: computing devices (which can be virtualized machines containing operating systems), storage devices, and network devices.

[0194] The present invention has the following beneficial technical effects:

[0195] This invention acquires road images and vehicle positioning information of a vehicle's location; obtains road parameters corresponding to the vehicle using the vehicle positioning information; performs visual processing on the road image of the vehicle's location to determine a first visual processing result corresponding to the road image; filters the first visual processing result to determine a second visual processing result corresponding to the road image; and determines lane information corresponding to the vehicle's location through fusion processing of the second visual processing result and the road parameters; based on the lane information corresponding to the vehicle's location, it monitors the vehicle's switching between different lanes. This not only achieves lane-level positioning of the vehicle using the lane information corresponding to its location, enabling accurate monitoring of the vehicle's switching between different lanes, but also utilizes a simple and low-cost device structure for acquiring road images of the vehicle's location, which is beneficial for the promotion of autonomous driving technology. Therefore, this solution can be applied to fields including, but not limited to, autonomous driving, vehicle networking, and intelligent transportation.

[0196] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lane-level positioning monitoring method, characterized by, The method comprises: acquiring a road image of a position where a vehicle is located and vehicle positioning information; acquiring road parameters corresponding to the vehicle through the vehicle positioning information; processing the road image through a multi-channel reversible residual network in a road image processing model to obtain a first feature vector matching the road image; processing the first feature vector through an extrusion excitation network in the road image processing model to obtain weight parameters and a second feature vector matching the road image; segmenting the road image based on the weight parameters and the second feature vector matching the road image and through an activation function and a corresponding convolutional neural network of the road image processing model to obtain a segmentation result of the road image; performing coordinate conversion processing on the segmentation result of the road image and lane line fitting reconstruction processing on a coordinate conversion processing result to obtain a lane line fitting reconstruction result; determining a left-numbered lane number and a left-numbered lane number confidence corresponding to the vehicle based on the lane line fitting reconstruction result; determining a right-numbered lane number and a right-numbered lane number confidence corresponding to the vehicle based on the lane line fitting reconstruction result; determining a lane line corresponding to the vehicle based on the lane line fitting reconstruction result; combining the left-numbered lane number, the left-numbered lane number confidence, the right-numbered lane number, the right-numbered lane number confidence, and the lane line corresponding to the vehicle to obtain a first visual processing result corresponding to the road image; filtering the first visual processing result corresponding to the road image to obtain a second visual processing result corresponding to the road image; fusing the second visual processing result and the road parameters to obtain lane information corresponding to the position where the vehicle is located; monitoring switching of the vehicle in different lanes based on the lane information corresponding to the position where the vehicle is located.

2. The method of claim 1, wherein, The acquiring of the road parameters corresponding to the vehicle through the vehicle positioning information comprises: acquiring map data matching the vehicle positioning information through the vehicle positioning information; performing data analysis processing on the map data to obtain the road parameters corresponding to the vehicle; The road parameters corresponding to the vehicle comprise at least one of the following: a number of lanes at the position where the vehicle is located, road attribute information at the position where the vehicle is located, and road monitoring information at the position where the vehicle is located.

3. The method of claim 1, wherein, The segmenting of the road image based on the weight parameters and the second feature vector matching the road image and through the activation function and the corresponding convolutional neural network of the road image processing model to obtain the segmentation result of the road image comprises: performing weighted processing on the second feature vector corresponding to the road image based on the weight parameters matching the road image to obtain intermediate feature information corresponding to the road image; mapping the intermediate feature information into a probability vector corresponding to the segmentation result and selecting a class corresponding to a maximum value of the probability vector as the segmentation result of the road image; The segmentation result is used to determine a lane line corresponding to the road image.

4. The method of claim 1, wherein, The first visual processing result corresponding to the road image is filtered to obtain a second visual processing result corresponding to the road image, including: When the left lane number and the right lane number are both 0, the left lane number and the right lane number in the first visual processing result are filtered, and the left lane number confidence, the right lane number confidence, and the lane line corresponding to the vehicle are combined to obtain the second visual processing result corresponding to the road image; When the left lane number confidence and the right lane number confidence are both less than a confidence threshold, the left lane number confidence and the right lane number confidence are filtered, and the left lane number and the right lane number, and the lane line corresponding to the vehicle are combined to obtain the second visual processing result corresponding to the road image; When the display effect of the lane line corresponding to the vehicle is less than a display effect threshold, the lane line corresponding to the vehicle is filtered, and the left lane number, the right lane number, the left lane number confidence, and the right lane number confidence are combined to obtain the second visual processing result corresponding to the road image; The speed parameter of the vehicle is obtained, and when the speed parameter of the vehicle is less than a speed threshold, the left lane number, the right lane number, the left lane number confidence, the right lane number confidence, and the lane line corresponding to the vehicle in the first visual processing result are discarded, and it is determined that the second visual processing result corresponding to the road image is empty.

5. The method of claim 1, wherein, The second visual processing result and the road parameter are fused to obtain lane information corresponding to the position of the vehicle, including: The left lane number, the left lane number confidence, the right lane number, the right lane number confidence, the lane line corresponding to the vehicle, and the number of lanes at the position of the vehicle in the road parameter in the second visual processing result are fused to obtain a fusion processing result; According to the value of the fusion processing result and the comparison result of the total number of lanes and the number of lanes at the position of the vehicle in the road parameter, the lane information corresponding to the position of the vehicle is determined.

6. The method of claim 5, wherein, According to the value of the fusion processing result and the comparison result of the total number of lanes and the number of lanes at the position of the vehicle in the road parameter, the lane information corresponding to the position of the vehicle is determined, including: When the left lane number and the right lane number are both not 0, and the left lane number confidence and the right lane number confidence are both greater than a confidence threshold, the total number of lanes is determined based on the left lane number and the right lane number; When the total number of lanes is equal to the number of lanes at the position of the vehicle in the road parameter, it is determined that the lane information corresponding to the position of the vehicle is the left lane number.

7. The method of claim 5, wherein, According to the value of the fusion processing result and the comparison result of the total number of lanes and the number of lanes at the position of the vehicle in the road parameter, the lane information corresponding to the position of the vehicle is determined, including: determining the lane information corresponding to the position of the vehicle as the left lane number when the visual total lane number is not equal to the lane number of the position of the vehicle, the left lane number is less than or equal to the right lane number, and the left lane number confidence is greater than or equal to the right lane number confidence; determining the lane information corresponding to the position of the vehicle as the left lane number when the visual total lane number is not equal to the lane number of the position of the vehicle, the left lane number is less than or equal to the right lane number, and the right lane number is greater than 3.

8. The method of claim 5, wherein, The determining the lane information corresponding to the position of the vehicle according to the value of the fusion processing result and the comparison result of the visual total lane number and the lane number of the position of the vehicle in the road parameter comprises: determining the left lane number based on the lane information corresponding to the position of the vehicle and the right lane number when the visual total lane number is not equal to the lane number of the position of the vehicle, the left lane number is greater than or equal to the right lane number, and the left lane number confidence is less than or equal to the right lane number confidence; determining the left lane number based on the lane information corresponding to the position of the vehicle and the right lane number when the visual total lane number is not equal to the lane number of the position of the vehicle, the left lane number is greater than or equal to the right lane number, and the right lane number is greater than 3.

9. The method of claim 5, wherein, The determining the lane information corresponding to the position of the vehicle according to the value of the fusion processing result and the comparison result of the visual total lane number and the lane number of the position of the vehicle in the road parameter comprises: when the left lane number and the right lane number are both not 0, and the left lane number confidence or the right lane number confidence is greater than a confidence threshold, performing the following operations: when the left lane number confidence is greater than the confidence threshold, the lane information corresponding to the position of the vehicle is the left lane number; when the right lane number confidence is greater than the confidence threshold, determining the left lane number based on the lane information corresponding to the position of the vehicle and the right lane number.

10. The method of claim 5, wherein, The determining the lane information corresponding to the position of the vehicle according to the value of the fusion processing result and the comparison result of the visual total lane number and the lane number of the position of the vehicle in the road parameter comprises: when the left lane number is not 0 or the right lane number is not 0, performing the following operations: when the left lane number is greater than 0, determining the lane information corresponding to the position of the vehicle as the left lane number; when the right lane number is greater than 0, determining the left lane number based on the lane information corresponding to the position of the vehicle and the right lane number.

11. The method of claim 1, wherein, The monitoring the switching of the vehicle in different lanes based on the lane information corresponding to the position of the vehicle comprises: determining a buffer interval matched with the environment of the vehicle; performing median filtering processing on the second visual processing result in the buffer interval to obtain a median filtering processing result; and performing the median filtering processing on the second visual processing result in the buffer interval to obtain a median filtering processing result. Determine lane information corresponding to a position of the vehicle at different monitoring time points based on the median filtering processing result; Determine a switching state of the vehicle in different lanes based on lane information corresponding to the position of the vehicle at two continuous monitoring time points.

12. A lane monitoring device, characterized by, The device comprises: An information transmission module configured to acquire a road image of a position of a vehicle and vehicle positioning information; An information processing module configured to acquire road parameters corresponding to the vehicle by the vehicle positioning information; The information processing module is configured to process the road image by a multi-channel reversible residual network in a road image processing model to obtain a first feature vector matched with the road image; process the first feature vector by an extrusion excitation network in the road image processing model to obtain a weight parameter and a second feature vector matched with the road image; perform segmentation processing on the road image based on the weight parameter and the second feature vector matched with the road image and by an activation function and a corresponding convolutional neural network of the road image processing model to obtain a segmentation result of the road image; perform coordinate conversion processing on the segmentation result of the road image and lane line fitting reconstruction processing on a coordinate conversion processing result to obtain a lane line fitting reconstruction result; determine a left-numbered lane number and a left-numbered lane number confidence corresponding to the vehicle based on the lane line fitting reconstruction result; determine a right-numbered lane number and a right-numbered lane number confidence corresponding to the vehicle based on the lane line fitting reconstruction result; determine a lane line corresponding to the vehicle based on the lane line fitting reconstruction result; and perform combination processing on the left-numbered lane number, the left-numbered lane number confidence, the right-numbered lane number, the right-numbered lane number confidence, and the lane line corresponding to the vehicle to obtain a first visual processing result corresponding to the road image. The information processing module is configured to perform filtering processing on the first visual processing result corresponding to the road image to determine a second visual processing result corresponding to the road image. The information processing module is configured to determine lane information corresponding to the position of the vehicle by fusion processing of the second visual processing result and the road parameters. The information processing module is configured to monitor switching of the vehicle in different lanes based on the lane information corresponding to the position of the vehicle.

13. The apparatus of claim 12, wherein, The information processing module is further configured to: Acquire map data matched with the vehicle positioning information by the vehicle positioning information; Perform data analysis processing on the map data to obtain road parameters corresponding to the vehicle; The road parameters corresponding to the vehicle include at least one of the following: A number of lanes at the position of the vehicle, road attribute information at the position of the vehicle, and road monitoring information at the position of the vehicle.

14. The apparatus of claim 12, wherein, The information processing module is further configured to: Perform weighting processing on the second feature vector corresponding to the road image based on the weight parameter matched with the road image to obtain intermediate feature information corresponding to the road image. Map the intermediate feature information to a probability vector corresponding to the segmentation result, and select a class corresponding to a maximum value of the probability vector as the segmentation result of the road image; The segmentation result is used to determine a lane line corresponding to the road image.

15. The apparatus of claim 12, wherein, The information processing module is further configured to: perform coordinate conversion processing on the segmentation result of the road image, and perform lane line fitting reconstruction processing on a result of the coordinate conversion processing to obtain a lane line fitting reconstruction result; determine, based on the lane line fitting reconstruction result, a left lane number corresponding to the vehicle and a left lane number confidence; determine, based on the lane line fitting reconstruction result, a right lane number corresponding to the vehicle and a right lane number confidence; determine, based on the lane line fitting reconstruction result, a lane line corresponding to the vehicle; perform combination processing on the left lane number, the left lane number confidence, the right lane number, the right lane number confidence, and the lane line corresponding to the vehicle to obtain a first visual processing result corresponding to the road image.

16. The apparatus of claim 15, wherein, The information processing module is further configured to: when the left lane number and the right lane number are both 0, perform filtering processing on the left lane number and the right lane number in the first visual processing result, and combine the left lane number confidence, the right lane number confidence, and the lane line corresponding to the vehicle to obtain a second visual processing result corresponding to the road image; when the left lane number confidence and the right lane number confidence are both less than a confidence threshold, perform filtering processing on the left lane number confidence and the right lane number confidence, and combine the left lane number and the right lane number and the lane line corresponding to the vehicle to obtain a second visual processing result corresponding to the road image; when a display effect of the lane line corresponding to the vehicle is less than a display effect threshold, perform filtering processing on the lane line corresponding to the vehicle, and combine the left lane number, the right lane number, the left lane number confidence, and the right lane number confidence to obtain a second visual processing result corresponding to the road image; obtain a speed parameter of the vehicle, and when the speed parameter of the vehicle is less than a speed threshold, discard the left lane number, the right lane number, the left lane number confidence, the right lane number confidence, and the lane line corresponding to the vehicle in the first visual processing result, and determine that a second visual processing result corresponding to the road image is empty.

17. An electronic device, comprising: The electronic device comprises: a memory configured to store executable instructions; a processor configured to execute the executable instructions stored in the memory to implement the lane-level positioning monitoring method in any one of claims 1 to 11.

18. A computer-readable storage medium, characterized in that, The executable instructions are stored in the memory and executed by the processor to implement the lane-level positioning monitoring method in any one of claims 1 to 11.

19. A computer program product comprising computer instructions, characterized in that, The computer instructions are executed by the processor to implement the lane-level positioning monitoring method in any one of claims 1 to 11.

Images