Vehicle lane keeping and lane changing control method, device, equipment and medium

Abstract

The invention provides a vehicle lane keeping and lane changing control method and device, equipment and a medium, and the method comprises the steps: carrying out the detection of a road image right ahead a target vehicle through an existing lane line detection model, and obtaining a lane line; according to the speed of the target vehicle, determining a pixel reference point in the image of the road in front of the target vehicle; traversing and screening pixel points, closest to the pixel reference point, in the pixel point group corresponding to each lane line by using a greedy algorithm to obtain a nearest pixel point group; the pixel points in the nearest pixel point group are sorted, the mean value of the adjacent nearest pixel points is calculated according to the sorting result, a lane center pixel point is determined, and lane keeping or lane changing of the target vehicle is controlled according to the determined lane center pixel point; the requirement of the lane keeping system for computing resources is reduced, and the real-time performance and the stability of the system are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to vehicle driving, and in particular relates to a vehicle lane keeping and lane changing control method, device, equipment and medium. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Vehicle driving is easily affected by surrounding vehicles, roads, and the vehicle's own status. The corresponding reaction time also needs to be extended, which greatly affects road traffic safety. It is understood that traffic accidents caused by lane departure account for about 20% of all traffic accidents, and fatal accidents account for about 30% of all traffic accidents. In order to reduce safety accidents caused by humans, autonomous driving technology has become a hot topic pursued by domestic and foreign research institutions and companies in recent years, and lane keeping and lane changing are one of the important components. Lane keeping is to keep the vehicle body from deviating from the lane by calculating the relative position of the lane line read by the front camera and the vehicle itself through the on-board domain control, and lane changing is to change the vehicle's movement trajectory to the adjacent lane.

[0004] At present, lane line detection-based vehicle lane keeping or lane change control wastes computing resources, and has problems with real-time performance and stability. Summary of the invention

[0005] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a vehicle lane keeping and lane changing control method, device, equipment and medium, which reduces the lane keeping system's demand for computing resources, improves the system's real-time performance and stability, and improves the comfort of passengers during lane keeping and lane changing.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a vehicle lane keeping and lane changing control method, comprising:

[0008] Acquire the road image directly in front of the target vehicle;

[0009] The acquired road image directly in front of the target vehicle is detected using a lane line detection model to determine the lane line in the road image directly in front of the target vehicle;

[0010] Determining a pixel reference point in a road image directly in front of the target vehicle according to the speed of the target vehicle;

[0011] A greedy algorithm is used to traverse and select the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group;

[0012] The pixel points in the nearest pixel point group are sorted, the average of the adjacent nearest pixel points is calculated according to the sorting result, the lane center pixel point is determined, and the lane keeping or lane changing of the target vehicle is controlled according to the determined lane center pixel point.

[0013] Furthermore, the acquired image of the road in front of the target vehicle is preprocessed, and the preprocessing includes image cropping and normalization processing.

[0014] Further, according to the speed of the target vehicle, the pixel reference point in the road image directly in front of the target vehicle is determined, specifically:

[0015] Determine the image center point of the road image directly in front of the target vehicle;

[0016] Determine a first pixel point above the center point of the image in a direction away from the target vehicle;

[0017] Determine a second pixel point below the center point of the image in a direction close to the target vehicle;

[0018] When the driving speed of the target vehicle is greater than a first speed threshold, the first pixel point is used as a pixel reference point of the target vehicle;

[0019] When the driving speed of the target vehicle is less than a first speed threshold, the second pixel point is used as a pixel reference point of the target vehicle.

[0020] Furthermore, a greedy algorithm is used to traverse and select the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group, specifically:

[0021] Calculate the distance between the ordinate of the pixel point corresponding to each lane line and the ordinate of the pixel reference point of the target vehicle;

[0022] According to the distance calculation result, the pixel point closest to the pixel reference point of the target vehicle in each lane line is determined.

[0023] Further, the pixels in the nearest pixel group are sorted, the average of the adjacent nearest pixels is calculated according to the sorting result, the lane center pixel is determined, and the lane keeping or lane changing of the target vehicle is controlled according to the determined lane center pixel, specifically:

[0024] Sort the pixels in the group of closest pixels based on the horizontal coordinates of the pixels in the group of closest pixels;

[0025] Calculate the average of the nearest adjacent pixels based on the sorting results to determine the lane pixel center point;

[0026] Calculating the distance between the lane pixel center point and the pixel reference point of the target vehicle;

[0027] The lane keeping or lane changing of the target vehicle is controlled according to the calculated distance.

[0028] Furthermore, according to the calculated distance, the lane keeping or lane changing of the target vehicle is controlled, specifically: when the lane pixel center point is the center point of the lane of the target vehicle, the target vehicle is controlled to keep driving in the lane; when the lane pixel center point is the center point of the lane next to the target vehicle, the target vehicle is controlled to change lanes.

[0029] In a second aspect, the present invention provides a vehicle lane keeping and lane changing control device, comprising:

[0030] An acquisition module is configured to: acquire an image of the road in front of the target vehicle;

[0031] A detection module is configured to: detect the acquired road image directly in front of the target vehicle using a lane line detection model to determine the lane line in the road image directly in front of the target vehicle;

[0032] A determination module is configured to: determine a pixel reference point in a road image directly in front of the target vehicle according to a speed of the target vehicle;

[0033] The screening module is configured to: use a greedy algorithm to traverse and screen the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group;

[0034] A control module is configured to: sort the pixels in the nearest pixel group, calculate the average of adjacent nearest pixels according to the sorting result, determine the lane center pixel, and control the lane keeping or lane changing of the target vehicle according to the determined lane center pixel.

[0035] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in the first aspect is completed.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0037] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in the first aspect.

[0038] One or more of the above technical solutions have the following beneficial effects:

[0039] In the present invention, the road image directly in front of the target vehicle can be detected by the existing lane line detection model to obtain the lane line; according to the speed of the target vehicle, the pixel reference point in the road image directly in front of the target vehicle is determined; the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point are traversed and screened by a greedy algorithm to obtain the nearest pixel point group; the pixel points in the nearest pixel point group are sorted, and the average of the adjacent nearest pixel points is calculated according to the sorting result to determine the lane center pixel point, and the lane keeping or lane changing of the target vehicle is controlled according to the determined lane center pixel point; the present invention performs corresponding lane keeping or lane changing control by means of pixel point calculation and comparison, thereby reducing the demand for computing resources of the lane keeping system and improving the real-time performance and stability of the system; and the method of determining the pixel reference point by the speed of the target vehicle can adapt to the current state of the target vehicle, while ensuring the accuracy of control, improving the comfort of the passengers during lane keeping and lane changing.

[0040] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0042] Figure 1 A flow chart of a vehicle lane keeping and lane changing control method proposed in an embodiment of the present invention;

[0043] Figure 2 An overall flow chart of a vehicle lane keeping and lane changing control method proposed in an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of the process of processing the image of the road directly in front of the target vehicle obtained in an embodiment of the present invention;

[0045] Figure 4 Schematic diagram of reference pixels in an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of a traversal screening process using a greedy algorithm in an embodiment of the present invention;

[0047] Figure 6 Schematic diagram of real lane lines and inferred lane lines, as well as each pixel point in an embodiment of the present invention;

[0048] Figure 7 A schematic diagram of the structure of a vehicle lane keeping and lane changing control device in an embodiment of the present invention;

[0049] Figure 8 The figure is a structural block diagram of a terminal in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0051] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0052] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0053] Embodiment 1

[0054] At present, lane detection can usually be performed through two types of lane detection algorithms: model method and feature method, and then the vehicle can be controlled to keep or change lanes. However, the model method focuses on improving the accuracy of lane detection by improving the model, and the application of large models has high computational costs; the feature method mainly detects lanes through some features such as the gradient, direction and grayscale value of the lane edge points, which is easily disturbed by the road environment, resulting in inaccurate lane detection.

[0055] like Figure 1 As shown, this embodiment discloses a vehicle lane keeping and lane changing control method, including:

[0056] Acquire the road image directly in front of the target vehicle;

[0057] The acquired road image directly in front of the target vehicle is detected using a lane line detection model to determine the lane line in the road image directly in front of the target vehicle;

[0058] Determining a pixel reference point in a road image directly in front of the target vehicle according to the speed of the target vehicle;

[0059] A greedy algorithm is used to traverse and select the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group;

[0060] The pixel points in the nearest pixel point group are sorted, the average of the adjacent nearest pixel points is calculated according to the sorting result, the lane center pixel point is determined, and the lane keeping or lane changing of the target vehicle is controlled according to the determined lane center pixel point.

[0061] In some embodiments, the acquired image of the road directly in front of the target vehicle is preprocessed, and the preprocessing includes image cropping and normalization.

[0062] In some embodiments, according to the speed of the target vehicle, the pixel reference point in the road image directly in front of the target vehicle is determined as follows:

[0063] Determine the image center point of the road image directly in front of the target vehicle;

[0064] Determine a first pixel point above the center point of the image in a direction away from the target vehicle;

[0065] Determine a second pixel point below the center point of the image in a direction close to the target vehicle;

[0066] When the driving speed of the target vehicle is greater than a first speed threshold, the first pixel point is used as a pixel reference point of the target vehicle;

[0067] When the driving speed of the target vehicle is less than a first speed threshold, the second pixel point is used as a pixel reference point of the target vehicle.

[0068] In some embodiments, a greedy algorithm is used to traverse and screen the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group, specifically:

[0069] Calculate the distance between the ordinate of the pixel point corresponding to each lane line and the ordinate of the pixel reference point of the target vehicle;

[0070] According to the distance calculation result, the pixel point closest to the pixel reference point of the target vehicle in each lane line is determined.

[0071] In some embodiments, the pixels in the nearest pixel group are sorted, the average of the adjacent nearest pixels is calculated according to the sorting result, the lane center pixel is determined, and the lane keeping or lane changing of the target vehicle is controlled according to the determined lane center pixel, specifically:

[0072] Sort the pixels in the group of closest pixels based on the horizontal coordinates of the pixels in the group of closest pixels;

[0073] Calculate the average of the nearest adjacent pixels based on the sorting results to determine the lane pixel center point;

[0074] Calculating the distance between the lane pixel center point and the pixel reference point of the target vehicle;

[0075] The lane keeping or lane changing of the target vehicle is controlled according to the calculated distance.

[0076] In some embodiments, the lane keeping or lane changing of the target vehicle is controlled according to the calculated distance, specifically: when the lane pixel center point is the center point of the lane of the target vehicle, the target vehicle is controlled to maintain the lane; when the lane pixel center point is the center point of the lane on the side of the target vehicle, the target vehicle is controlled to change lanes.

[0077] Figure 2 This is a flow chart of a vehicle lane keeping and lane changing control method proposed in this embodiment, see Figure 2 , the method comprises the following steps:

[0078] Step 1: Obtain the road image directly in front of the target vehicle.

[0079] Since the vehicle usually needs to obtain some information about the road ahead in order to achieve autonomous driving during driving, the front camera can be installed above the windshield of the vehicle to obtain the image of the road directly in front of the target vehicle through the front camera.

[0080] As an implementation method, considering range coverage and redundancy, the front camera used is an RGB short-focus camera with a field of view of approximately 30 degrees. The front camera can be an industrial camera with a pixel of 720x 1080, so that it is arranged above the windshield in the cockpit and is not blocked.

[0081] The front camera is horizontally parallel to the ground, and the ground occupies at least 50% of the entire image. During the installation of the front camera, the front camera needs to be directly in front of the vehicle so that the center point of the image captured by the front camera is consistent with the center line of the vehicle.

[0082] In the vehicle perception system, the vehicle mainly uses ground information for autonomous navigation, and information such as the sky and roadside trees is clutter in the lane keeping system. The vehicle control and computing unit used in this embodiment is the NVIDIA Orin domain control unit.

[0083] Step 2: Detect the acquired road image directly in front of the target vehicle using a lane line detection model to determine the lane line in the road image directly in front of the target vehicle.

[0084] This embodiment uses a road line detection model based on deep learning to detect the acquired road image directly in front of the target vehicle to obtain lane line information in the road image directly in front of the target vehicle.

[0085] There is a large amount of information and noise that is invalid to the algorithm in the original image of the road directly in front of the target vehicle. Therefore, this embodiment preprocesses the acquired original image of the road directly in front of the target vehicle to remove the invalid information, which can effectively speed up the operation speed and robustness of the lane line detection algorithm.

[0086] like Figure 3 As shown, the preprocessing of the acquired original image of the road in front of the target vehicle specifically includes:

[0087] Step 201: The image needs to be cropped, and the bottom 60% of the image is retained.

[0088] Since lane lines can only appear in the ground information at the bottom of the picture, cropping the picture can effectively improve the utilization of information and the performance of the deep learning model.

[0089] Step 202: Adjust the cropped image to the input size of deep learning, and perform normalization operation.

[0090] When training the lane detection model, normalization can speed up the convergence of the network model and improve the accuracy of the network model. Because the image is cropped and enlarged before the data is input into the lane detection model, the pixel coordinates output by the lane detection model are not the pixel coordinates of the real original image, and the pixel coordinates of the original image can only be obtained after the transformation.

[0091] Step 3: According to the speed of the target vehicle, a pixel reference point in the road image directly in front of the target vehicle is determined.

[0092] Take the center point O(x, y) of the image captured by the front camera. In step 1, the camera has been adjusted to face the front of the target vehicle, so point O(x, y) represents a fixed distance point in front of the target vehicle. Then, Figure 4 As shown, according to the center point O of the image, two points on the upper and lower sides of the camera image, namely the first pixel point P1 and the second pixel point P, are taken, which respectively represent fixed positions farther and closer to the target vehicle. The selection of pixel points is determined according to the calibration of the camera.

[0093] When the target vehicle is traveling at a higher speed than the first speed threshold, the pixel reference point of the target vehicle selects a farther pixel point, namely the first pixel point P1; when the vehicle is traveling at a lower speed than the first speed threshold, the pixel reference point of the target vehicle selects a closer pixel point, namely the second pixel point P2.

[0094] The first speed threshold may be determined according to actual conditions.

[0095] Step 4: Use a greedy algorithm to traverse and screen the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group.

[0096] By detecting the acquired road image directly in front of the target vehicle through the lane line detection model in step 2, multiple lane lines and the pixel coordinates of each lane line on the original image will be obtained.

[0097] like Figure 5 As shown, for each lane line, a greedy algorithm is used to calculate the pixel point D closest to the pixel point corresponding to the vehicle pixel reference point P1 or P2, specifically including:

[0098] Step 401: Initialize a result array to store the pixel points of the lane line;

[0099] Step 402: Determine whether the calculation of each lane line is completed; if so, end; if not, execute step 403;

[0100] Step 403: Initialize the shortest distance to 9999999 and the pixel subscript to 0;

[0101] Step 404: Determine whether the calculation of each pixel point of a lane line is completed. If so, save the result of this lane line in the result array; if not, execute step 405;

[0102] Step 405: Calculate the distance between the y coordinate value of the pixel point at this moment and the y value of the coordinate of the pixel reference point P1 or P2 of the vehicle;

[0103] Step 406: If the distance is less than the minimum distance, update the latest distance and the corresponding pixel index, and return to step 404.

[0104] Step 5: Sort the pixels in the nearest pixel group, calculate the average of the adjacent nearest pixels according to the sorting result, determine the lane center pixel, and control the lane keeping or lane changing of the target vehicle according to the determined lane center pixel.

[0105] After obtaining the nearest pixel group D composed of the nearest pixel points of all lane lines according to step 4, the pixel group D is sorted according to the horizontal coordinate x of the pixel sequence. Then, as Figure 6 As shown, the average value of D[i] and D[i+1] is calculated in pairs, and this value is the pixel center point M of each lane. When the vehicle speed is fast, the far point, that is, the first pixel point P1, is referenced, and when the vehicle speed is slow, the near point, that is, the second pixel point P2, is selected. Among them, [i] and [i+1] represent the lower coordinate sequence of the pixel point, such as calculating the average value between the 0th and 1st pixel points, and calculating the average value between the 1st pixel point and the 2nd pixel point.

[0106] After obtaining the pixel coordinates of the lane center, the coordinates are transmitted to the control module. The Euclidean distance between the lane center pixel point M and the reference pixel point P in front of the vehicle is calculated, and the PID control method is used to reduce the error. The input of PID is the error between the two points, and the output is the steering angle of the target vehicle. When the lane center pixel point M is the center point of the vehicle's driving lane, the vehicle can maintain the lane. When the lane center pixel point M is the center point of the lane next to the vehicle, the vehicle can complete the lane change.

[0107] like Figure 7 As shown, the purpose of this embodiment is to provide a vehicle lane keeping and lane changing control device, including:

[0108] An acquisition module is configured to: acquire an image of the road in front of the target vehicle;

[0109] A detection module is configured to: detect the acquired road image directly in front of the target vehicle using a lane line detection model to determine the lane line in the road image directly in front of the target vehicle;

[0110] A determination module is configured to: determine a pixel reference point in a road image directly in front of the target vehicle according to a speed of the target vehicle;

[0111] The screening module is configured to: use a greedy algorithm to traverse and screen the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group;

[0112] A control module is configured to: sort the pixels in the nearest pixel group, calculate the average of adjacent nearest pixels according to the sorting result, determine the lane center pixel, and control the lane keeping or lane changing of the target vehicle according to the determined lane center pixel.

[0113] refer to Figure 8 , Figure 8 The structure block diagram of a terminal 800 provided by an exemplary embodiment of the present application is shown. The terminal 800 may be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0114] Typically, the terminal 800 includes a processor 801 and a memory 802 .

[0115] The processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0116] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one program code, which is used to be executed by the processor 801 to implement the motor end cover optimization method provided in the method embodiment of the present application.

[0117] In some embodiments, the terminal 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807 and a power supply 808.

[0118] The peripheral device interface 803 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0119] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0120] The display screen 805 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 805 is a touch display screen, the display screen 805 also has the ability to collect touch signals on the surface or above the surface of the display screen 805. The touch signal can be input to the processor 801 as a control signal for processing. At this time, the display screen 805 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 805 can be one, set on the front panel of the terminal 800; in other embodiments, the display screen 805 can be at least two, respectively set on different surfaces of the terminal 800 or in a folding design; in other embodiments, the display screen 805 can be a flexible display screen, set on the curved surface or folding surface of the terminal 800. Even, the display screen 805 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 805 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0121] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0122] The audio circuit 807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 801 for processing, or input them into the radio frequency circuit 804 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 800. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 807 may also include a headphone jack.

[0123] The power supply 808 is used to power various components in the terminal 800. The power supply 808 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 808 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0124] In some embodiments, the terminal 800 further includes one or more sensors 809 , including but not limited to: an acceleration sensor 810 , a gyroscope sensor 811 , a pressure sensor 812 , an optical sensor 813 , and a proximity sensor 814 .

[0125] The acceleration sensor 810 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 800. For example, the acceleration sensor 810 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a horizontal view or a vertical view based on the gravity acceleration signal collected by the acceleration sensor 810. The acceleration sensor 810 can also be used to collect game or user motion data.

[0126] The gyroscope sensor 811 can detect the body direction and rotation angle of the terminal 800, and the gyroscope sensor 811 can cooperate with the acceleration sensor 810 to collect the user's 3D actions on the terminal 800. Based on the data collected by the gyroscope sensor 811, the processor 801 can implement the following functions: motion sensing (such as changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0127] The pressure sensor 812 can be set on the side frame of the terminal 800 and / or the lower layer of the display screen 805. When the pressure sensor 812 is set on the side frame of the terminal 800, it can detect the user's holding signal of the terminal 800, and the processor 801 performs left and right hand recognition or shortcut operation based on the holding signal collected by the pressure sensor 812. When the pressure sensor 812 is set on the lower layer of the display screen 805, the processor 801 controls the operability controls on the UI interface based on the user's pressure operation on the display screen 805. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0128] The optical sensor 813 is used to collect the ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 based on the ambient light intensity collected by the optical sensor 813. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is reduced. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera component 806 based on the ambient light intensity collected by the optical sensor 813.

[0129] The proximity sensor 814, also called a distance sensor, is usually disposed on the front panel of the terminal 800. The proximity sensor 814 is used to collect the distance between the user and the front of the terminal 800. In one embodiment, when the proximity sensor 814 detects that the distance between the user and the front of the terminal 800 is gradually decreasing, the processor 801 controls the display screen 805 to switch from the screen-on state to the screen-off state; when the proximity sensor 814 detects that the distance between the user and the front of the terminal 800 is gradually increasing, the processor 801 controls the display screen 805 to switch from the screen-off state to the screen-on state.

[0130] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the terminal 800, and the terminal 800 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0131] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the motor end cover optimization method in the above embodiment.

[0132] In an exemplary embodiment, a computer program product is further provided. The computer program product stores at least one program code. The at least one program code is loaded and executed by a processor to implement the motor end cover optimization method in the above embodiment.

[0133] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0134] The above description is only for the purpose of facilitating the technical solution of the present application to be understood by those skilled in the art, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle lane keeping and lane changing control method, characterized in that: include: Acquire the road image directly in front of the target vehicle; The acquired road image directly in front of the target vehicle is detected using a lane line detection model to determine the lane line in the road image directly in front of the target vehicle; Determining a pixel reference point in a road image directly in front of the target vehicle according to the speed of the target vehicle; A greedy algorithm is used to traverse and select the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group; The pixel points in the nearest pixel point group are sorted, the average of the adjacent nearest pixel points is calculated according to the sorting result, the lane center pixel point is determined, and the lane keeping or lane changing of the target vehicle is controlled according to the determined lane center pixel point.

2. A vehicle lane keeping and lane changing control method as claimed in claim 1, characterized in that: The acquired image of the road in front of the target vehicle is preprocessed, and the preprocessing includes image cropping and normalization processing.

3. A vehicle lane keeping and lane changing control method as claimed in claim 1, characterized in that: According to the speed of the target vehicle, the pixel reference point in the road image directly in front of the target vehicle is determined, specifically: Determine the image center point of the road image directly in front of the target vehicle; Determine a first pixel point above the center point of the image in a direction away from the target vehicle; Determine a second pixel point below the center point of the image in a direction close to the target vehicle; When the driving speed of the target vehicle is greater than a first speed threshold, the first pixel point is used as a pixel reference point of the target vehicle; When the driving speed of the target vehicle is less than a first speed threshold, the second pixel point is used as a pixel reference point of the target vehicle.

4. A vehicle lane keeping and lane changing control method as claimed in claim 1, characterized in that: The greedy algorithm is used to traverse and screen the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group, specifically: Calculate the distance between the ordinate of the pixel point corresponding to each lane line and the ordinate of the pixel reference point of the target vehicle; According to the distance calculation result, the pixel point closest to the pixel reference point of the target vehicle in each lane line is determined.

5. The vehicle lane keeping and lane changing control method according to claim 1, characterized in that: The pixel points in the nearest pixel point group are sorted, and the average of the adjacent nearest pixel points is calculated according to the sorting result to determine the lane center pixel point. According to the determined lane center pixel point, the lane keeping or lane changing of the target vehicle is controlled, specifically as follows: Sort the pixels in the group of closest pixels based on the horizontal coordinates of the pixels in the group of closest pixels; Calculate the average of the nearest adjacent pixels based on the sorting results to determine the lane pixel center point; Calculating the distance between the lane pixel center point and the pixel reference point of the target vehicle; The lane keeping or lane changing of the target vehicle is controlled according to the calculated distance.

6. A vehicle lane keeping and lane changing control method as claimed in claim 5, characterized in that: According to the calculated distance, the lane keeping or lane changing of the target vehicle is controlled, specifically: when the lane pixel center point is the center point of the lane of the target vehicle, the target vehicle is controlled to keep driving in the lane; when the lane pixel center point is the center point of the lane next to the target vehicle, the target vehicle is controlled to change lanes.

7. A vehicle lane keeping and lane changing control device, characterized in that: include: An acquisition module is configured to: acquire an image of the road in front of the target vehicle; A detection module is configured to: detect the acquired road image directly in front of the target vehicle using a lane line detection model to determine the lane line in the road image directly in front of the target vehicle; A determination module is configured to: determine a pixel reference point in a road image directly in front of the target vehicle according to a speed of the target vehicle; The screening module is configured to: use a greedy algorithm to traverse and screen the pixel point group corresponding to each lane line and the pixel point closest to the pixel reference point to obtain the closest pixel point group; A control module is configured to: sort the pixels in the nearest pixel group, calculate the average of adjacent nearest pixels according to the sorting result, determine the lane center pixel, and control the lane keeping or lane changing of the target vehicle according to the determined lane center pixel.

8. An electronic device, characterized in that: The method comprises a memory and a processor and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the method described in any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images