Method, system, storage medium and program product for assisted driving processing

Abstract

The present disclosure provides a method, system, storage medium and program product for assisting driving processing, and belongs to the technical field of intelligent driving. In the method, the first region and the second region are regions where lane lines are most likely to appear in images captured by a vehicle head. These two regions are taken as attention regions. A feature extraction model with an attention mechanism can strengthen the image regions related to lane lines and weaken the image regions unrelated to lane lines. In this way, a lane line positioning model can achieve high detection accuracy with fewer model parameters, thereby better balancing processing efficiency and detection accuracy.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent driving, and in particular, relates to a method and system for assisting driving processing, a storage medium and a program product. BACKGROUND

[0002] Lane line detection is one of the technologies in the technical field of intelligent driving. During the driving process of a vehicle, a driver needs to drive according to a lane line, and a driving route is planned for the driver. When the vehicle deviates from the lane line, the assisting driving processing can be performed.

[0003] In the related art, lane line detection is performed based on a deep learning model, and the assisting driving processing is performed based on the recognized lane line.

[0004] However, the deep learning model is relatively complex, and it is difficult to balance the processing efficiency and the detection accuracy. SUMMARY

[0005] To solve the problems in the related art, the present disclosure provides a method and system for assisting driving processing, a storage medium and a program product. The technical solutions are as follows.

[0006] In a first aspect, a method for assisting driving processing is provided. The method is applied to a target vehicle, the target vehicle includes a car machine and a camera, the camera is installed at the front end of the target vehicle, the shooting direction of the camera is towards the front of the target vehicle, and the method includes the following steps.

[0007] The car machine acquires an image captured by the camera.

[0008] The car machine extracts features of the image based on a first feature extraction model with a fusion attention mechanism to obtain a first feature image, wherein the attention area corresponding to the first feature extraction model with the fusion attention mechanism includes a first area and a second area, the first area extends from the middle of the upper edge of the image area to one end of the lower edge of the image area, and the second area extends from the middle of the upper edge of the image area to the other end of the lower edge of the image area.

[0009] The car machine inputs the first feature image into a lane line positioning model to obtain lane line position information output by the lane line positioning model.

[0010] If the lane line position information output by the lane line positioning model is not empty, the car machine performs assisting driving processing based on the lane line position information.

[0011] In a possible implementation, the car machine extracts features of the image based on the first feature extraction model with the fusion attention mechanism to obtain the first feature image, including the following steps.

[0012] The car machine pre-processes the image, and the pre-processing includes at least one of scaling processing of a specified proportion, graying processing, and normalization;

[0013] The car machine inputs the pre-processed image into a first feature extraction model of a fusion attention mechanism to obtain a first feature image.

[0014] In a possible implementation, the method further includes:

[0015] The car machine performs feature extraction on the image based on a second feature extraction model of the fusion attention mechanism to obtain a second feature image, wherein the attention area corresponding to the second feature extraction model of the fusion attention mechanism includes a third area, and the third area extends from the middle of the upper edge of the image area to the middle of the lower edge;

[0016] The car machine inputs the second feature image into a road surface mark recognition model to obtain a road surface mark output by the road surface mark recognition model.

[0017] The car machine performs auxiliary driving processing based on the lane line position information, including:

[0018] The car machine performs auxiliary driving processing based on the lane line position information and the road surface mark.

[0019] In a possible implementation, the method further includes:

[0020] The car machine inputs the image into an occlusion object recognition model to obtain an occlusion object type of the camera surface output by the occlusion object recognition model.

[0021] The car machine determines a target cleaning component corresponding to the occlusion object type of the camera surface based on a pre-stored correspondence relationship between occlusion object types and cleaning components.

[0022] The car machine controls the target cleaning component to clean the camera surface.

[0023] In a possible implementation, the occlusion object type includes a water droplet type, a fog type, a sludge type, and a foreign object type.

[0024] The cleaning component includes a fan, a heater, a water gun, and an electric brush.

[0025] In the correspondence relationship, the water droplet type corresponds to the fan, the fog type corresponds to the heater, the sludge type corresponds to the water gun, and the foreign object type corresponds to the electric brush.

[0026] In a possible implementation, the car machine performs an auxiliary driving process based on the lane line position information, including:

[0027] When the car machine determines that the lane line position information is partially missing, the car machine fills in the lane line position information to obtain filled-in lane line position information.

[0028] The car machine performs an auxiliary driving process based on the filled-in lane line position information.

[0029] In a possible implementation, the lane line position information includes a plurality of lane line position points.

[0030] In a second aspect, a system for an auxiliary driving process is provided, including a car machine and a camera, the camera being installed at a front end of a target vehicle, and a shooting direction of the camera being toward a front of the target vehicle.

[0031] The car machine is configured to obtain an image captured by the camera.

[0032] The car machine is configured to perform feature extraction on the image based on a first feature extraction model of a fusion attention mechanism to obtain a first feature image, where an attention region corresponding to the first feature extraction model of the fusion attention mechanism includes a first region and a second region, the first region extending from a middle of an upper edge of an image region to one end of a lower edge of the image region, and the second region extending from the middle of the upper edge of the image region to the other end of the lower edge of the image region.

[0033] The car machine is configured to input the first feature image into a lane line positioning model to obtain lane line position information output by the lane line positioning model.

[0034] The car machine is configured to perform an auxiliary driving process based on the lane line position information if the lane line position information output by the lane line positioning model is not empty.

[0035] In a possible implementation, the car machine is configured to:

[0036] The car machine is configured to perform preprocessing on the image, including at least one of scaling processing, grayscale processing, and normalization.

[0037] The car machine is configured to input the preprocessed image into the first feature extraction model of the fusion attention mechanism to obtain the first feature image.

[0038] In a possible implementation, the car machine is further configured to:

[0039] The second feature extraction model based on the fusion attention mechanism is used for feature extraction of the image to obtain a second feature image, wherein the attention region corresponding to the second feature extraction model based on the fusion attention mechanism includes a third region, and the third region extends from the middle of the upper edge of the image region to the middle of the lower edge.

[0040] The second feature image is input into a road sign identification model to obtain a road sign output by the road sign identification model.

[0041] Based on the lane line position information, an auxiliary driving process is performed, including:

[0042] Based on the lane line position information and the road sign, an auxiliary driving process is performed.

[0043] In a possible implementation, the car machine is further configured to:

[0044] The image is input into an occlusion object identification model to obtain an occlusion object type of the camera surface output by the occlusion object identification model.

[0045] Based on a pre-stored correspondence between the occlusion object type and the cleaning component, a target cleaning component corresponding to the occlusion object type of the camera surface is determined.

[0046] The target cleaning component is controlled to clean the camera surface.

[0047] In a possible implementation, the occlusion object type includes a water droplet type, a fog type, a sludge type, and a foreign object type.

[0048] The cleaning component includes a fan, a heater, a water gun, and an electric brush.

[0049] In the correspondence, the water droplet type corresponds to the fan, the fog type corresponds to the heater, the sludge type corresponds to the water gun, and the foreign object type corresponds to the electric brush.

[0050] In a possible implementation, the car machine is configured to:

[0051] When the car machine determines that the lane line position information is partially missing, the lane line position information is filled to obtain filled lane line position information.

[0052] Based on the filled lane line position information, an auxiliary driving process is performed.

[0053] In a possible implementation, the lane line position information includes a plurality of lane line position points.

[0054] In a third aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores computer program code. When the computer program code is executed by a computer device, the computer device performs the method provided in the first aspect and possible implementation manners thereof.

[0055] In a fourth aspect, a computer program product is provided, and the computer program product includes computer program code. When the computer program code is executed by a computer device, the computer device performs the method provided in the first aspect and possible implementation manners thereof.

[0056] With the method, the first region and the second region are regions where lane lines are most likely to appear in an image captured by a general vehicle head, and the two regions are taken as attention regions. The feature extraction model with the attention mechanism can strengthen the image regions related to the lane lines and weaken the image regions irrelevant to the lane lines, so that the lane line positioning model can achieve higher detection accuracy with fewer model parameters, thereby better balancing the processing efficiency and the detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0058] Figure 2 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0059] Figure 3 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0060] Figure 4 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0061] Figure 5 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0062] Figure 6 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0063] Figure 7 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0064] Figure 8 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0065] Figure 9 FIG. 1 is a structural schematic diagram of a system for assisting driving processing provided by an embodiment of the present disclosure;

[0066] Figure 10is a schematic diagram of an attention area provided by an embodiment of the present disclosure;

[0067] Figure 11 is a schematic diagram of an attention area provided by an embodiment of the present disclosure;

[0068] Figure 12 is a schematic diagram of a road width provided by an embodiment of the present disclosure;

[0069] Figure 13 is a schematic diagram of a road width provided by an embodiment of the present disclosure;

[0070] Figure 14 is a schematic diagram of a road width provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0071] Lane line detection is one of the technologies in the field of intelligent driving. During the driving process of a vehicle, a driver needs to drive according to a lane line, and a driving route is planned for the driver. When the vehicle deviates from the lane line, an auxiliary driving process can be performed.

[0072] An embodiment of the present disclosure provides a method for auxiliary driving processing. The execution subject of the method can be a target vehicle. An auxiliary driving processing system in the target vehicle can include a car machine 110 and a camera 120. As shown in Figure 1 The camera 120 is installed at the front end of the target vehicle, and the shooting direction of the camera 120 is towards the front of the target vehicle.

[0073] The car machine 110 can functionally realize the communication between the target vehicle and other devices, as shown in Figure 2 The car machine 110 can include a processor 210, a memory 220, a display component 230, and a communication component 240.

[0074] The processor 210 can be a central processing unit (CPU) or a system on chip (SoC), and the like. The processor 210 can be used to process various operation instructions, for example, to perform auxiliary driving processing.

[0075] The memory 220 can include various volatile memories or non-volatile memories, such as a solid state disk (SSD), a dynamic random access memory (DRAM) memory, and the like. The memory 220 can be used to store initial data, intermediate data, and result data used in the related processing process, for example, to store road surface marks and traffic signs.

[0076] The display component 230 can be a separate screen, or a screen integrated with the user equipment body, a projector, etc. The screen can be a touch screen or a non-touch screen (which can be displayed remotely), etc. The display component 230 is used to display system interfaces, application interfaces, etc. For example, display lane lines, etc.

[0077] The communication component 240 can be a wired network connector, an ultra wide band (UWB) technology, a wireless fidelity (WiFi) module, a Bluetooth module, a cellular network communication module, etc. The communication component 240 can be used to receive and send various instructions and data, etc. For example, obtain images captured by the camera, etc.

[0078] The camera 120 can be communicatively coupled to the vehicle machine, and can be used to capture images, etc. Figure 3 As shown, the vehicle machine can include a processor 310, a memory 320, and a communication component 330.

[0079] The processor 310 can be a CPU or a SoC, etc. The processor 310 can be used to execute various instructions related to the method, etc. For example, process instructions of the vehicle machine, etc.

[0080] The memory 320 can include various volatile memories or non-volatile memories, such as SSD, DRAM memory, etc. The memory 320 can be used to store pre-stored data, intermediate data, and result data in content type conversion and display processing, for example, store captured images, etc.

[0081] The communication component 330 can be a wired network connector, an UWB, a WiFi module, a Bluetooth module, a cellular network communication module, etc. The communication component 330 can be used to receive various instructions and data, etc. For example, send captured images to the vehicle machine, etc.

[0082] The method for assisting driving processing provided by the embodiments of the present disclosure can be as shown in the following. Figure 4 As shown, the method includes the following steps:

[0083] 401. The vehicle machine obtains images captured by the camera.

[0084] Generally, multiple cameras will be installed on a car. The camera installed at the front end of the target vehicle has a shooting direction towards the front of the target vehicle, and is used to capture the road conditions in front of the vehicle. The camera captures images in real time and sends the images to the vehicle machine.

[0085] In addition, images can also be obtained through road monitoring videos.

[0086] 402. The vehicle machine extracts features from the images based on the first feature extraction model based on the fusion attention mechanism, to obtain a first feature image.

[0087] The vehicle machine can pre-process the image, and the pre-processing includes at least one of scaling processing of a specified ratio, grayscale processing, and normalization. The grayscale refers to the light and shade degree of the color of the image, and the grayscale processing method can be a component method, a maximum value method, an average value method, and a weighted average value method, etc. The normalization is to convert the pixel value in the image to between 0-1. The vehicle machine inputs the pre-processed image into the first feature extraction model of the fusion attention mechanism.

[0088] The attention region corresponding to the first feature extraction model of the fusion attention mechanism includes a first region and a second region, the first region extends from the middle of the upper edge of the image region to one end of the lower edge of the image region, and the second region extends from the middle of the upper edge of the image region to the other end of the lower edge of the image region. For example Figure 5 As shown in the figure, AB and CD are two lane lines, and the two dashed boxes are the first region and the second region, respectively.

[0089] First, the image is subjected to initial feature extraction to obtain an initial feature image. The initial feature image is input into a channel attention module to obtain a channel feature image. The channel feature image is input into a spatial attention module to obtain a first feature image.

[0090] The first feature extraction model of the fusion attention mechanism takes ResNet34 as the backbone network, contains 5 stages, i.e., 5 convolution stages with different parameters, and performs feature extraction through multiple convolution layers. In stage 3, stage 4, and stage 5, an attention mechanism is added, which includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism performs global maximum pooling and global average pooling operations on the image to obtain a feature vector, which is multiplied with the initial feature image to obtain a channel feature image. The spatial attention mechanism performs convolution operation on the image to obtain a feature vector, which is multiplied with the channel feature image to obtain a first feature image. In this way, the feature image output by the first feature extraction model of the fusion attention mechanism can pay more attention to the image region with strong lane line features.

[0091] In addition, the first feature image output by the first feature extraction model of the fusion attention mechanism can also be input into a variable convolutional neural network to make the extraction of lane line features more accurate.

[0092] 403, the vehicle machine inputs the first feature image into the lane line positioning model to obtain lane line position information output by the lane line positioning model.

[0093] The first feature image output by the model is input into the lane line positioning model to obtain anchor boxes of multiple lane lines. The lane line positioning model selects anchor points of lane lines in units of rows, and each small box is an anchor box. For example Figure 6 As shown in FIG. 8, AB and CD are two lane lines, and the two black anchor boxes in the thickened row are anchor boxes of the lane lines.

[0094] The lane line position information can include multiple lane line position points.

[0095] 404, if the lane line position information output by the lane line positioning model is not empty, the car machine performs auxiliary driving processing based on the lane line position information.

[0096] When the car machine determines that the lane line position information has partial missing, the lane line position information can be filled in to obtain filled lane line position information.

[0097] There can be multiple ways to fill in the lane line position information.

[0098] In a first way, when the car machine determines that the lane line position information has missing lane line position points, at least one lane line position point is inserted into the multiple lane line position points based on an interpolation method to obtain filled lane line position information. Specifically, the missing lane line position points can be filled in by using interpolation methods such as Lagrange interpolation method and Newton interpolation method.

[0099] In a second way, when the car machine determines that the lane line position information has missing lane line position points, a lane line is obtained by linear fitting based on the multiple lane line position points, and at least one lane line position point is inserted into the multiple lane line position points based on the lane line to obtain filled lane line position information.

[0100] The car machine performs auxiliary driving processing based on the filled lane line position information.

[0101] The filled lane line position information can be multiple lane line position points, which can form a complete lane line. The car machine performs auxiliary driving processing according to the position of the lane line to keep the vehicle driving within the lane line.

[0102] If the lane line position information output by the lane line positioning model is empty, the road edge position information can be extracted, and then auxiliary driving processing is performed according to the road edge position information. The specific steps are described in detail in the following flow.

[0103] In a possible implementation, the processing process of the car machine for auxiliary driving processing can also include the following steps as shown in FIG. 9. Figure 7

[0104] ​701, the vehicle machine acquires the image captured by the camera.

[0105] The camera captures the image in real time and sends the image to the vehicle machine.

[0106] In addition, the image can also be obtained through road monitoring video.

[0107] 702, the vehicle machine extracts features from the image based on the second feature extraction model with fusion attention mechanism to obtain a second feature image.

[0108] The vehicle machine can preprocess the image, and the preprocessing includes at least one of scaling processing of a specified ratio, grayscale processing, and normalization. Gray scale refers to the degree of light and shade of image color, and the grayscale processing method can be component method, maximum value method, average value method, and weighted average value method, etc. Normalization is to convert the pixel value in the image to 0-1. The vehicle machine inputs the preprocessed image into the second feature extraction model with fusion attention mechanism to obtain the second feature image output by the feature extraction model with fusion attention mechanism.

[0109] The attention area corresponding to the second feature extraction model with fusion attention mechanism includes a third area, and the third area extends from the middle of the upper edge of the image area to the middle of the lower edge. For example Figure 8 As shown, AB and CD are two lane lines, and there is a deceleration road sign between the two lane lines. The dashed box is the third area.

[0110] 703, the vehicle machine inputs the second feature image into the road sign recognition model to obtain the road sign output by the road sign recognition model.

[0111] The second feature image is input into the road sign recognition model, and the road sign recognition model can be a machine learning model, which can recognize the road sign in the image to obtain the road sign output by the road sign recognition model.

[0112] When the vehicle machine determines that the road sign is partially missing, the road sign can be filled to obtain the filled road sign. The vehicle machine can pre-store images of various road signs, and the filling method can be to calculate the similarity between the current image and the pre-stored road sign image, determine the pre-stored road sign image with a similarity exceeding a similarity threshold, and fill the current image based on the image.

[0113] 704, the vehicle machine performs auxiliary driving processing based on the lane line position information and the road sign.

[0114] The vehicle machine can perform auxiliary driving processing according to the road surface mark and the lane line position information determined by the above process. For example, when the vehicle machine recognizes a road surface mark with a speed reduction prompt in front of the vehicle, if the driver does not perform a speed reduction operation, the driver can be prompted to reduce speed by voice, or the vehicle machine can automatically reduce speed.

[0115] In a possible implementation, the processing procedure of the vehicle machine performing auxiliary driving processing can further include the following steps, as shown in the figure. Figure 9

[0116] 901. The vehicle machine acquires an image captured by a camera.

[0117] The camera captures an image in real time and sends the image to the vehicle machine.

[0118] In addition, the image can also be acquired through a road monitoring video.

[0119] 902. The vehicle machine extracts features from the image based on a third feature extraction model with a fusion attention mechanism to obtain a third feature image.

[0120] The vehicle machine can perform preprocessing on the image, and the preprocessing includes at least one of scaling processing of a specified ratio, grayscale processing, and normalization. Grayscale refers to the lightness and darkness of the color of the image, and the grayscale processing method can be a component method, a maximum value method, an average value method, and a weighted average value method, etc. Normalization is to convert the pixel value in the image to between 0 and 1. The vehicle machine inputs the preprocessed image into the third feature extraction model with a fusion attention mechanism to obtain a third feature image output by the third feature extraction model with a fusion attention mechanism.

[0121] The attention region corresponding to the third feature extraction model with a fusion attention mechanism includes a fourth region, and the fourth region extends from the upper left edge of the image region to the upper right edge. For example Figure 10 AB and CD are two lane lines, the right lane line has a traffic sign with a speed limit of 60 kilometers per hour, and the dashed box is the fourth region.

[0122] 903. The vehicle machine inputs the third feature image into a traffic sign recognition model to obtain a traffic sign output by the traffic sign recognition model.

[0123] The third feature image is input into the traffic sign recognition model, and the traffic sign recognition model can be a machine learning model, which can recognize the traffic sign in the image to obtain a traffic sign output by the traffic sign recognition model.

[0124] ​When the vehicle machine determines that the traffic sign is partially missing, the vehicle machine can fill in the traffic sign to obtain a filled traffic sign. The vehicle machine can pre-store images of various traffic signs. The filling method can be to calculate the similarity between the current image and the pre-stored traffic sign image, determine the pre-stored traffic sign image with a similarity exceeding a similarity threshold, and fill in the current image based on the image.

[0125] 904. The vehicle machine performs an assisted driving process based on the lane line position information, the road surface identifier, and the traffic sign.

[0126] The vehicle machine can perform an assisted driving process based on the traffic sign, the lane line position information, and the road surface identifier determined in the above process. For example, when the vehicle machine recognizes a traffic sign prohibiting U-turn in front of the vehicle, the vehicle machine can voice prompt the driver that the front is prohibited from U-turn if the driver performs a U-turn operation.

[0127] In a possible implementation, the process of the vehicle machine performing the assisted driving process can also include the following steps, as shown in Figure 11

[0128] 1101. The vehicle machine inputs the first feature image into the road edge positioning model to obtain road edge position information output by the road edge positioning model.

[0129] In addition, the image captured by the camera can also be directly input into the road edge positioning model.

[0130] 1102. The vehicle machine performs an assisted driving process based on the road edge position information.

[0131] The vehicle machine determines the road width based on the road edge position information. The vehicle machine determines the lowest end position of each edge line of the road in the image based on the road edge position information. If the lowest end position of the edge line is at the lower edge of the image, for example, as shown in Figure 12 , the vehicle machine determines the lowest end positions A and B as the reference positions. If the lowest end position of the edge line is at the side edge of the image, for example, as shown in Figure 13 , the vehicle machine extends the edge line and the lower edge of the image respectively, and determines the intersection positions C and D of the edge line and the lower edge of the image as the reference positions. The vehicle machine determines the distance between the two reference positions as the target reference width, and looks up the road width corresponding to the target reference width based on the pre-stored correspondence between the reference width and the road width, for example, as shown in Table 1, as the road width.

[0132] Table 1

[0133] Reference width Road width 5 centimeters 3 meters 6 centimeters 3.5 meters …… ……

[0134] ​The vehicle machine performs an auxiliary driving process based on the road width.

[0135] The vehicle machine can select a current driving range according to the current road width. For example, if the road width is greater than or equal to twice a preset minimum lane width, the vehicle machine controls the target vehicle to drive on the right side of the road, and if the road width is less than twice the preset lane width, the vehicle machine controls the target vehicle to drive in the middle of the road.

[0136] In a possible implementation, when the camera surface has an obstruction, the vehicle machine can perform a process as shown in Figure 14 The process includes the following steps:

[0137] 1401. The vehicle machine inputs an image into an obstruction recognition model to obtain an obstruction type of the camera surface output by the obstruction recognition model.

[0138] The camera is installed on the outside of the vehicle, and the surface can have an obstruction, for example, rainwater obstruction when it rains, or mud obstruction, etc. The vehicle machine inputs an image captured by the camera into an obstruction recognition model, which can be a machine learning model, to obtain an obstruction type of the camera surface output by the obstruction recognition model.

[0139] 1402. The vehicle machine determines a target cleaning component corresponding to the obstruction type of the camera surface based on a pre-stored correspondence between obstruction types and cleaning components.

[0140] The vehicle machine pre-stores a correspondence between obstruction types and cleaning components, which can be as shown in Table 2. The obstruction types can include water droplet types, fog types, mud types, and foreign matter types, etc., and the cleaning components can include air blowers, heaters, water guns, and electric brushes, etc. In the correspondence, the water droplet type corresponds to the air blower, the fog type corresponds to the heater, the mud type corresponds to the water gun, and the foreign matter type corresponds to the electric brush. After determining the obstruction type, the target cleaning component is determined in the above correspondence.

[0141] Table 2

[0142] Obstruction type Cleaning component Water droplet type Fan Mist type Heater …… ……

[0143] 1403. The vehicle machine controls the target cleaning component to clean the camera surface.

[0144] The vehicle machine can control the target cleaning component to clean the camera surface, so that the camera can capture an image without obstruction.

[0145] In the embodiments of the present disclosure, the first region and the second region are regions where lane lines are most likely to appear in the image captured by the front camera. By taking these two regions as the attention regions, the feature extraction model with attention mechanism can strengthen the image regions related to the lane lines and weaken the image regions unrelated to the lane lines. In this way, the lane line positioning model can achieve higher detection accuracy with fewer model parameters, so as to better balance the processing efficiency and the detection accuracy.

[0146] All the optional technical solutions described above can be combined to form optional embodiments of the present disclosure, which will not be described one by one here.

[0147] The embodiments of the present disclosure also provide a system for assisting driving processing. The system includes a car machine 110 and a camera 120. The camera is installed at the front end of a target vehicle, and the shooting direction of the camera is towards the front of the target vehicle.

[0148] The car machine 110 is configured to obtain an image captured by the camera 120. The car machine 110 can implement the processing function of step 401 and other implicit steps.

[0149] The car machine 110 is configured to perform feature extraction on the image based on a first feature extraction model with attention mechanism to obtain a first feature image. The attention regions corresponding to the first feature extraction model with attention mechanism include a first region and a second region. The first region extends from the middle of the upper edge of the image region to one end of the lower edge of the image region, and the second region extends from the middle of the upper edge of the image region to the other end of the lower edge of the image region. The car machine 110 can implement the processing function of step 402 and other implicit steps.

[0150] The car machine 110 is configured to input the first feature image into a lane line positioning model to obtain lane line position information output by the lane line positioning model. The car machine 110 can implement the processing function of step 403 and other implicit steps.

[0151] The car machine 110 is configured to perform auxiliary driving processing based on the lane line position information if the lane line position information output by the lane line positioning model is not empty. The car machine 110 can implement the processing function of step 404 and other implicit steps.

[0152] In a possible implementation, the car machine 110 is configured to perform preprocessing on the image. The preprocessing includes at least one of scaling processing of a specified ratio, grayscale processing, and normalization. The car machine 110 is configured to input the preprocessed image into the first feature extraction model with attention mechanism to obtain the first feature image. The car machine 110 can implement the processing function of step 402 and other implicit steps.

[0153] In a possible implementation, the car machine 110 is further configured to perform feature extraction on the image based on the second feature extraction model with a fusion attention mechanism to obtain a second feature image, where the attention region corresponding to the second feature extraction model with the fusion attention mechanism includes a third region, and the third region extends from the middle of the upper edge of the image region to the middle of the lower edge; input the second feature image into the road sign recognition model to obtain a road sign output by the road sign recognition model; and perform auxiliary driving processing based on the lane line position information and the road sign. The processing functions of steps 701, 702, 703, and 704 and other implicit steps can be implemented.

[0154] In a possible implementation, the car machine 110 is further configured to input the image into the occlusion object recognition model to obtain an occlusion object type on the surface of the camera 120 output by the occlusion object recognition model; determine a target cleaning component corresponding to the occlusion object type on the surface of the camera 120 based on a pre-stored correspondence between the occlusion object type and the cleaning component; and control the target cleaning component to clean the surface of the camera. The processing functions of steps 1401, 1402, and 1403 and other implicit steps can be implemented.

[0155] In a possible implementation, the occlusion object type includes a water droplet type, a fog type, a silt type, and a foreign object type; the cleaning component includes a fan, a heater, a water gun, and an electric brush; and in the correspondence, the water droplet type corresponds to the fan, the fog type corresponds to the heater, the silt type corresponds to the water gun, and the foreign object type corresponds to the electric brush. The processing functions of step 1402 and other implicit steps can be implemented.

[0156] In a possible implementation, the car machine 110 is configured to: when the car machine determines that the lane line position information has a partial loss, fill in the lane line position information to obtain filled-in lane line position information; and perform auxiliary driving processing based on the filled-in lane line position information. The processing functions of step 404 and other implicit steps can be implemented.

[0157] In a possible implementation, the lane line position information includes a plurality of lane line position points. The processing functions of step 403 and other implicit steps can be implemented.

[0158] In the embodiments of the present disclosure, the first region and the second region are regions where lane lines are most likely to appear in images generally captured by the front of a vehicle. By taking these two regions as attention regions, the feature extraction model with the fusion attention mechanism can strengthen image regions related to lane lines and weaken image regions unrelated to lane lines, so that the lane line positioning model can achieve a higher detection accuracy with fewer model parameters, thereby better balancing the processing efficiency and the detection accuracy.

[0159] The above sequence numbers of the embodiments of the present disclosure are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0160] A person of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete the related hardware, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0161] The above only describes the preferred embodiments of the present disclosure, and does not limit the present disclosure. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for assisting driving, characterized in that, The method is applied to a target vehicle, which includes a vehicle-mounted infotainment system and a camera. The camera is mounted at the front of the target vehicle, and its shooting direction is towards the front of the target vehicle. The method includes: The vehicle's infotainment system acquires images captured by the camera; The vehicle system extracts features from the image based on a first feature extraction model with a fusion attention mechanism to obtain a first feature image. The attention region corresponding to the first feature extraction model with the fusion attention mechanism includes a first region and a second region. The first region extends from the middle of the upper edge of the image region to one end of the lower edge of the image region, and the second region extends from the middle of the upper edge of the image region to the other end of the lower edge of the image region. The vehicle system inputs the first feature image into the lane line positioning model to obtain the lane line position information output by the lane line positioning model; The vehicle system extracts features from the image based on a second feature extraction model with a fusion attention mechanism to obtain a second feature image. The attention region corresponding to the second feature extraction model with the fusion attention mechanism includes a third region, which extends from the middle of the upper edge to the middle of the lower edge of the image region. The vehicle-mounted system inputs the second feature image into the road marking recognition model to obtain the road markings output by the road marking recognition model; If the lane line position information output by the lane line positioning model is not empty and it is determined that there are some missing lane line position information, the vehicle system will fill in the missing lane line position information to obtain the filled lane line position information, and perform assisted driving processing based on the filled lane line position information and the road markings.

2. The method according to claim 1, characterized in that, The first feature extraction model based on the fusion attention mechanism extracts features from the image to obtain a first feature image, including: The vehicle system preprocesses the image, and the preprocessing includes at least one of scaling by a specified ratio, grayscale conversion, and normalization. The vehicle system inputs the preprocessed image into the first feature extraction model with a fusion attention mechanism to obtain the first feature image.

3. The method according to claim 1, characterized in that, The method further includes: The vehicle system inputs the image into the occlusion recognition model, and the occlusion recognition model outputs the type of occlusion on the surface of the camera. The vehicle system determines the target cleaning component corresponding to the type of obstruction on the camera surface based on a pre-stored correspondence between obstruction types and cleaning components. The vehicle's infotainment system controls the target cleaning component to clean the surface of the camera.

4. The method according to claim 3, characterized in that, The types of obstructions include water droplets, fog, silt, and foreign objects; The cleaning components include a fan, a heater, a water gun, and an electric brush; In the correspondence, the water droplet type corresponds to the fan, the mist type corresponds to the heater, the sludge type corresponds to the water gun, and the foreign object type corresponds to the electric brush.

5. The method according to claim 1, characterized in that, The lane line location information includes multiple lane line location points.

6. A system for assisting driving, characterized in that, The system includes a vehicle-mounted infotainment system and a camera. The camera is mounted on the front of the target vehicle and its shooting direction is towards the front of the target vehicle. The vehicle infotainment system is used to acquire images captured by the camera; The vehicle system is used to extract features from the image based on a first feature extraction model with a fusion attention mechanism to obtain a first feature image. The attention region corresponding to the first feature extraction model with the fusion attention mechanism includes a first region and a second region. The first region extends from the middle of the upper edge of the image region to one end of the lower edge of the image region, and the second region extends from the middle of the upper edge of the image region to the other end of the lower edge of the image region. The vehicle-mounted system is used to input the first feature image into the lane line positioning model to obtain the lane line position information output by the lane line positioning model. The vehicle system is used to extract features from the image based on a second feature extraction model with a fusion attention mechanism to obtain a second feature image. The attention region corresponding to the second feature extraction model with the fusion attention mechanism includes a third region, which extends from the middle of the upper edge to the middle of the lower edge of the image region. The vehicle-mounted unit is used to input the second feature image into the road marking recognition model to obtain the road markings output by the road marking recognition model; The vehicle-mounted system is configured to, if the lane line position information output by the lane line positioning model is not empty and it is determined that the lane line position information is partially missing, supplement the lane line position information to obtain supplemented lane line position information, and perform assisted driving processing based on the supplemented lane line position information and the road markings.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program code, which, when executed by a computer device, performs the method described in any one of claims 1-5.

8. A computer program product, characterized in that, The computer program product includes computer program code, which, when executed by a computer device, performs the method described in any one of claims 1-5.

Images