Target detection method and device, equipment and storage medium

Abstract

The invention provides a target detection method and device, equipment and a storage medium, and relates to the technical field of automatic driving visual perception, and the method comprises the steps: determining each lane line according to the original image data of a current lane, generating a plurality of first coordinate points of each lane line, and carrying out the sorting according to the first coordinate points, determining an area between the leftmost lane line and the rightmost lane line as a first maximum drivable area; projecting the first maximum drivable area and the plurality of first coordinate points of each lane line into the original image, and generating a second maximum drivable area in the original image and a plurality of second coordinate points of each lane line in the original image; and carrying out zero setting on an area outside the second maximum drivable area in the original image to obtain a pre-processed image, and executing target detection operation. According to the method, more key information needed by target detection is added, only the driving area is reserved in the original image, invalid information in the original image is reduced, and therefore the reasoning time of a deep learning model can be shortened.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving visual perception technology, and in particular to a target detection method, device, equipment and storage medium. Background Art

[0002] With the continuous development of autonomous driving technology, vehicles have higher and higher requirements for the perception and speed of the surrounding environment. Among them, visual perception technology has become one of the key components to realize the autonomous driving system, and visual target detection is a key technology of visual perception. Through target detection, vehicles can obtain information about the surrounding environment in real time, so as to make correct driving decisions to ensure driving safety and efficiency.

[0003] Existing visual target detection methods usually use deep learning models such as convolutional neural networks. Although these models have made significant progress in target recognition accuracy, they have high requirements on computing resources and long model inference time.

[0004] However, in the existing technology, during the preprocessing stage of target detection, too much invalid information is retained, and at the same time, the key information required for target detection is missing, which leads to too long inference time of the deep learning model and reduces the accuracy and efficiency of visual perception. Summary of the invention

[0005] The present application provides a target detection method, apparatus, equipment and storage medium to solve the problem in the prior art that in the preprocessing stage of target detection, too much invalid information is retained and key information required for target detection is missing, resulting in too long inference time of the deep learning model and reduced accuracy and efficiency of visual perception.

[0006] In a first aspect, the present application provides a target detection method, the method comprising:

[0007] Obtaining original image data of the lane where the vehicle is located;

[0008] Determine each lane line in the lane according to the original image data, and generate a plurality of first coordinate points of each lane line;

[0009] Sort the lane lines according to the multiple first coordinate points of each lane line to determine the leftmost lane line and the rightmost lane line;

[0010] Determine an area between the leftmost lane line and the rightmost lane line as a first maximum drivable area;

[0011] Projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate a second maximum drivable area in the original image and multiple second coordinate points of each lane line in the original image;

[0012] Setting the area outside the second maximum drivable area in the original image to zero to obtain a preprocessed image;

[0013] An object detection operation is performed according to the preprocessed image.

[0014] In a possible design, determining each lane line in the lane according to the original image data includes:

[0015] Generate a binary image and an instance segmentation image of the original image according to the original image data;

[0016] Clustering is performed on the pixels of the binary image and the instance segmented image to determine the lane line.

[0017] In a possible design, clustering pixels of the binary image and the instance segmented image to determine the lane line includes:

[0018] Clustering the pixels of the binary image and the instance segmented image, and determining each lane line in the binary image and the instance segmented image;

[0019] Acquire coordinate data of a plurality of first target points on each lane line in the binary image and the instance segmented image;

[0020] Convert the coordinate data of the plurality of first target points on each lane line into a bird's-eye view coordinate system to generate coordinate data of the plurality of second target points on each lane line;

[0021] The lane line equation is determined by polynomial fitting according to the coordinate data of the plurality of second target points on each of the lane lines.

[0022] In a possible design, determining the area between the leftmost lane line and the rightmost lane line as the first maximum drivable area includes:

[0023] Acquire multiple coordinate points of the first maximum drivable area according to the lane line equation coefficient of the leftmost lane line, the lane line direction coefficient of the rightmost lane line, and multiple first coordinate points of the leftmost lane line and the rightmost lane line;

[0024] The first maximum drivable area is determined according to a plurality of coordinate points of the first maximum drivable area.

[0025] In a possible design, before acquiring the original image data of the lane in which the vehicle is located, the method further includes:

[0026] Acquire calibration image data including the lane line;

[0027] An inverse perspective transformation matrix is ​​calculated based on the calibration image and the intrinsic parameter matrix and the extrinsic parameter matrix of the camera device that captures the calibration image. The inverse perspective transformation matrix is ​​used to transform the image from a perspective coordinate system to a bird's-eye view coordinate system.

[0028] In a possible design, converting the coordinate data of the plurality of first target points on each lane line into a bird's-eye view coordinate system to generate the coordinate data of the plurality of second target points on each lane line includes:

[0029] The coordinate data of the multiple first target points on each of the lane lines are converted to the bird's-eye view coordinate system through the inverse perspective transformation matrix to generate the coordinate data of the multiple second target points on each of the lane lines.

[0030] In a possible design, projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate the second maximum drivable area in the original image and the multiple second coordinate points of each lane line in the original image includes:

[0031] The first maximum drivable area and the multiple first coordinate points of each lane line are projected into the original image through the inverse perspective transformation matrix to generate the second maximum drivable area and the multiple second coordinate points of each lane line in the original image.

[0032] In a second aspect, the present application provides a target detection device, the device comprising:

[0033] An acquisition module is used to: acquire original image data of the lane where the vehicle is located;

[0034] A coordinate determination module, used to: determine each lane line in the lane according to the original image data, and generate a plurality of first coordinate points of each lane line;

[0035] A sorting module, used to: sort the lane lines according to the multiple first coordinate points of each lane line, and determine the leftmost lane line and the rightmost lane line;

[0036] An image processing module, used to: determine the area between the leftmost lane line and the rightmost lane line as a first maximum drivable area;

[0037] Projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate a second maximum drivable area in the original image and multiple second coordinate points of each lane line in the original image;

[0038] Setting the area outside the second maximum drivable area in the original image to zero to obtain a preprocessed image;

[0039] The target detection module is used to perform a target detection operation according to the preprocessed image.

[0040] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0041] The memory stores computer-executable instructions;

[0042] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0044] The target detection method, device, equipment and storage medium provided by this application achieve the following technical effects:

[0045] This method can ultimately retain the coordinate point data of multiple lane lines in the original image, adding more key information required for target detection compared to existing technologies, and can improve the accuracy of visual perception. The parts outside the drivable area of ​​the original image are set to zero, and only the drivable area is retained, reducing the invalid information in the original image, thereby reducing the inference time of the deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 Schematic diagram of the target detection method provided in the embodiment of the present application Figure 1 ;

[0048] Figure 2 Schematic diagram of the target detection method provided in the embodiment of the present application Figure 2 ;

[0049] Figure 3 It is a schematic diagram of the process of determining lane lines provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of the structure of target detection provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation methods described in the following exemplary embodiments do not represent all implementation methods consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims, rather than all embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] First, the relevant concepts or nouns involved in this application are explained:

[0054] Perspective coordinate system: Perspective coordinate system is a coordinate system used to simulate the perspective of the human eye or camera. In perspective projection, objects appear smaller when they are farther away from the observer and larger when they are closer to the observer. This coordinate system is often used in 3D graphics and computer vision to generate realistic images.

[0055] Bird's-eye view coordinate system: A bird's-eye view coordinate system is a coordinate system that looks down on a scene from a high altitude, similar to the perspective of a bird from the air. This perspective is commonly used in maps, navigation systems, and certain types of image processing applications to provide a comprehensive view of the scene.

[0056] Inverse perspective transformation matrix: The inverse perspective transformation matrix is ​​used to transform the perspective projection image back to the original 3D coordinate system. This transformation is very important in computer vision, especially when recovering 3D information from 2D images. For example, in autonomous driving, the road image can be converted into a bird's-eye view through the inverse perspective transformation for better path planning and obstacle detection.

[0057] Calibrated images: Calibrated images are images that have been calibrated by the camera. Camera calibration is the process of determining the internal and external parameters of the camera, including focal length, optical center position, distortion coefficient, etc. The calibrated images can be used for accurate measurement and 3D reconstruction because the geometric distortion of the camera has been corrected.

[0058] HLS space: HLS space is a color representation method that includes three components: hue, brightness, and saturation. Unlike the RGB color space, the HLS space is more in line with human visual perception and is often used for color analysis and processing in image processing and computer vision.

[0059] In view of the problem that in the preprocessing stage of target detection in the prior art, too much invalid information is retained and the key information required for target detection is lacking, resulting in too long inference time of the deep learning model and reducing the accuracy and efficiency of visual perception, the technical concept of the present application is: obtaining the original image data of the lane in which the vehicle is located, determining each lane line in the current lane according to the original image, and generating multiple coordinate points on each lane line, sorting each lane line according to the coordinate points on each lane line, and determining the leftmost lane line and the rightmost lane line, and determining the area between the leftmost lane line and the rightmost lane line as the maximum drivable area, projecting the area and the multiple coordinate points on each of the aforementioned lane lines into the original image, and setting the part outside the maximum drivable area in the original image to zero, and then performing target detection based on the processed image.

[0060] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0061] Embodiment 1

[0062] Figure 1 Schematic diagram of the target detection method provided in the embodiment of the present application Figure 1 , this method can be applied to the vehicle's automatic driving system. Figure 1 As shown, the method includes:

[0063] S101, obtaining original image data of the lane where the vehicle is located;

[0064] Specifically, a real-time image of the lane in which the vehicle is currently located may be obtained as raw image data through a camera mounted on the vehicle.

[0065] S102, determining each lane line in the lane according to the original image data, and generating a plurality of first coordinate points of each lane line;

[0066] Specifically, since the original image is taken by a camera and is the camera's perspective, the coordinate system of the original image is a perspective coordinate system, and the lane line equation is based on the bird's-eye view coordinate system. Therefore, the lane lines based on the perspective coordinate system in the original image data can be converted to the bird's-eye view coordinate system through an inverse perspective transformation matrix and a two-dimensional vector perspective transformation, and then the equation of each lane line in the bird's-eye view coordinate system is determined, and based on the equation of each lane line, multiple first coordinate points of each lane line are generated.

[0067] S103, sorting the lane lines according to the multiple first coordinate points of each lane line, and determining the leftmost lane line and the rightmost lane line;

[0068] S104, determining an area between the leftmost lane line and the rightmost lane line as a first maximum drivable area;

[0069] S105, projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate the second maximum drivable area in the original image and the multiple second coordinate points of each lane line in the original image;

[0070] Specifically, since the coordinate system of the original image is a perspective coordinate system, after determining the first maximum drivable area and multiple first coordinate points of each lane line in the bird's-eye view coordinate system, the inverse perspective transformation matrix is ​​used to project the first maximum drivable area and multiple first coordinate points of each lane line into the original image through inverse perspective transformation to generate the second maximum drivable area in the original image and multiple second coordinate points of each lane line in the original image.

[0071] S106, setting the area outside the second largest drivable area in the original image to zero to obtain a preprocessed image;

[0072] After the area outside the second maximum drivable area is set to zero, only the original data on the second maximum drivable area and the second coordinate point on each lane line are retained in the preprocessed image, and invalid data are cleared by setting to zero.

[0073] S107: Perform target detection operation according to the preprocessed image.

[0074] Specifically, the preprocessed image can be input into the target detection network to obtain the target detection object in the second largest drivable area, and the target detection object can be output to the control module of the automatic driving system.

[0075] The method provided in this embodiment achieves the following technical effects:

[0076] This method can ultimately retain the coordinate point data of multiple lane lines in the original image, adding more key information required for target detection compared to existing technologies, and can improve the accuracy of visual perception. The parts outside the drivable area of ​​the original image are set to zero, and only the drivable area is retained, reducing the invalid information in the original image, thereby reducing the inference time of the deep learning model.

[0077] The target detection method of the present application is described in detail using a specific embodiment below.

[0078] Embodiment 2

[0079] Figure 2 Schematic diagram of the target detection method provided in the embodiment of the present application Figure 2 .like Figure 2 As shown, the method includes:

[0080] S201, obtaining calibration image data including lane lines;

[0081] S202, calculating an inverse perspective transformation matrix according to the calibration image and the intrinsic parameter matrix and the extrinsic parameter matrix of the camera device that collects the calibration image, where the inverse perspective transformation matrix is ​​used to transform the image from the perspective coordinate system to the bird's-eye view coordinate system;

[0082] Specifically, the camera internal and external parameter matrix includes:

[0083] Intrinsic parameter matrix (intrinsic parameter matrix): describes the internal characteristics of the camera, including focal length, optical center (principal point) and lens distortion parameters. The intrinsic parameter matrix is ​​usually expressed as a 3x3 matrix;

[0084] Extrinsic parameter matrix (extrinsic parameter matrix): describes the posture of the camera relative to the world coordinate system, including position and translation. The extrinsic parameter matrix is ​​usually composed of a rotation matrix and a translation vector.

[0085] When calculating the inverse perspective matrix that transforms the perspective coordinate system of the camera image into the bird's-eye view coordinate system, the camera intrinsic and extrinsic parameter matrix mentioned above must be obtained. Calculating the inverse perspective transformation matrix includes the following steps:

[0086] Step 1: Collect the calibration image containing lane lines in the current frame, ensure that there are two lines in the image that are parallel lines in the actual scene (such as two lane lines), and the direction of the lines is consistent with the direction of the vehicle camera as much as possible. The two lines in the image are as symmetrical as possible about the image center axis. Then the lane lines can be marked in white and other non-lane line areas can be painted black.

[0087] Step 2: Set the region of interest (ROI) in the image, filter the areas irrelevant to the two lane lines in the calibration image to generate a zero matrix, and set the ROI area to 1;

[0088] Step 3: Read the calibration image, perform image dedistortion transformation and convert it to HLS space, then take the brightness layer to extract edge features and use Hough transform to detect two lines in the image. Use Hough transform to detect the white lane line on the ground (multiple straight lines will be detected at this time). In order to reduce the error, the vanishing point is defined as the minimum distance point to all the above straight lines;

[0089] Step 4: Find the intersection point (vanishing point) of the two lines, determine the trapezoidal area of ​​the original image that needs inverse perspective transformation based on the coordinates of the vanishing point, and calculate the inverse perspective transformation matrix based on the trapezoidal coordinates in the calibration image and the rectangular coordinates of the top view.

[0090] In step S202, the inverse perspective transformation matrix is ​​calculated so that in subsequent steps, the pixel points in the original image can be mapped to the corresponding points in the bird's-eye view, and the points in the bird's-eye view can be mapped to the corresponding points in the perspective view of the original image through the inverse inverse perspective transformation.

[0091] S203, obtaining original image data of the lane where the vehicle is located, and generating a binary image and an instance segmentation image of the original image according to the original image data;

[0092] Specifically, in this step, the real-time image captured by the vehicle camera is obtained as the original image data, and the original image data is input into the lane line detection network to generate a binary image of the lane line (to distinguish between the background and the lane line) and an instance segmentation image (to distinguish which lane the pixels on the lane line belong to).

[0093] The lane detection network is usually a trained deep learning model, such as a convolutional neural network (CNN). These models can extract features from images and identify the location and shape of lane lines. A binary image is an image in which each pixel belongs to either a lane line (usually represented by white) or a background (usually represented by black). This image can be obtained through threshold segmentation or the output layer of a deep learning model.

[0094] Instance segmentation is about distinguishing not only lane lines from the background, but also different lane lines. Pixels on each lane line are assigned different labels to distinguish which lane they belong to. Instance segmentation usually requires more complex models, such as Mask R-CNN or other specially designed instance segmentation networks. These models can not only detect lane lines, but also generate a separate mask for each lane line.

[0095] S204, clustering the pixels of the binary image and the instance segmentation image, determining lane lines, and generating a plurality of first coordinate points of each lane line;

[0096] Figure 3is a schematic diagram of the process of determining lane lines provided in an embodiment of the present application. Figure 3 As shown, clustering the pixels of the binary image and the instance segmentation image to determine the lane lines may include the following steps:

[0097] S301, clustering the pixels of the binary image and the instance segmentation image, and determining each lane line in the binary image and the instance segmentation image;

[0098] In this step, the binary image can be first morphologically processed to remove non-lane line black holes and reduce errors. After that, the binary image and the instance segmentation image can be logically ANDed to obtain a new image that only retains lane line pixels that exist in both images. Alternatively, the binary image and the instance segmentation image can be weighted merged to obtain a new image, and the weight can be determined based on the confidence level of the image or other indicators. After merging the binary image with the instance segmentation image, the following algorithm can be used to cluster the pixels:

[0099] DBSCAN: density-based clustering algorithm that can effectively handle noise;

[0100] K-means: A centroid-based clustering algorithm that requires the number of clusters to be specified in advance;

[0101] Mean Shift: A density-based clustering algorithm that does not require the number of clusters to be specified in advance.

[0102] This embodiment can achieve the following technical effects by clustering the pixels of the binary image and the instance segmentation image:

[0103] Improve detection accuracy: Through the clustering algorithm, pixels belonging to the same lane line can be clustered together to more accurately determine the position and shape of the lane line, which helps reduce false detections and missed detections and improves the overall detection accuracy.

[0104] Enhanced robustness: Clustering methods can effectively handle noise and discontinuities in images. For example, in actual road scenes, lane lines may be broken or blurred due to wear, occlusion, or changes in illumination. Clustering algorithms can fill these breaks by clustering adjacent pixels together, thereby improving the robustness of detection.

[0105] Real-time performance: Clustering algorithms usually have high computational efficiency and can process a large amount of pixel data in a short time. This is particularly important for autonomous driving systems that require real-time processing and can meet real-time requirements.

[0106] Adaptable to various scenarios: The clustering method can adapt to different types of lane lines (such as solid lines, dotted lines, curves, etc.) and different road scenarios (such as urban roads, highways, etc.), and has good versatility;

[0107] Reduce error propagation: Through clustering, pixels can be classified into different lane line instances, avoiding error propagation between different lane lines, thereby improving the accuracy and reliability of lane line detection.

[0108] In this step, each lane line can be classified and assigned different category numbers through clustering, and different lane lines can be marked with different colors.

[0109] S302, obtaining coordinate data of a plurality of first target points on each lane line in the binary image and the instance segmentation image;

[0110] S303, converting the coordinate data of a plurality of first target points on each lane line into a bird's-eye view coordinate system to generate coordinate data of a plurality of second target points on each lane line;

[0111] In this step, specifically, the coordinate data of multiple first target points on each lane line are converted into a bird's-eye view coordinate system using the aforementioned inverse perspective transformation matrix and two-dimensional vector perspective transformation, thereby generating coordinate data of multiple second target points on each lane line.

[0112] S304: Determine a lane line equation by polynomial fitting according to the coordinate data of multiple second target points on each lane line.

[0113] In this step, quadratic polynomial fitting can be used to determine the coefficients of each item of the lane line equation that represent the shape of the lane line, thereby determining the lane line equation. Specifically, the lane line equation can be:

[0114] y=ax 2 +bx+c

[0115] Among them, y is the ordinate of the lane line in the bird's-eye view coordinate system, and x is the abscissa of the lane line in the bird's-eye view coordinate system; a, b, and c are the lane line direction coefficients: a is the quadratic coefficient, which represents the curvature of the lane line, which determines the degree of curvature of the lane line; b is the linear coefficient, which represents the slope of the lane line, which determines the degree of inclination of the lane line; c is a constant term, which represents the position offset of the lane line in the coordinate system.

[0116] S205, sorting the lane lines according to the multiple first coordinate points of each lane line, and determining the leftmost lane line and the rightmost lane line;

[0117] In this step, the sorting result of each lane line can be retained in the final preprocessed image, thereby retaining more effective information in the preprocessed image.

[0118] S206, determining the area between the leftmost lane line and the rightmost lane line as the first maximum drivable area;

[0119] Optionally, determining an area between the leftmost lane line and the rightmost lane line as the first maximum drivable area includes:

[0120] Acquire multiple coordinate points of the first maximum drivable area according to the lane line equation coefficient of the leftmost lane line, the lane line direction coefficient of the rightmost lane line, and multiple first coordinate points of the leftmost lane line and the rightmost lane line;

[0121] The first maximum drivable area is determined according to a plurality of coordinate points of the first maximum drivable area.

[0122] S207, projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate the second maximum drivable area in the original image and the multiple second coordinate points of each lane line in the original image;

[0123] Optionally, projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate the second maximum drivable area in the original image and the multiple second coordinate points of each lane line in the original image includes:

[0124] The first maximum drivable area and multiple first coordinate points of each lane line are projected into the original image through an inverse perspective transformation matrix to generate a second maximum drivable area and multiple second coordinate points of each lane line in the original image.

[0125] S208, setting the area outside the second largest drivable area in the original image to zero to obtain a preprocessed image;

[0126] S209: Perform target detection operation according to the preprocessed image.

[0127] The embodiment of the present invention can divide the electronic device or the main control device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0128] Figure 4 This is a schematic diagram of the structure of target detection provided in the embodiment of the present application. Figure 4 As shown, the device 40 includes:

[0129] The acquisition module 401 is used to: acquire the original image data of the lane where the vehicle is located;

[0130] The coordinate determination module 402 is used to: determine each lane line in the lane according to the original image data, and generate a plurality of first coordinate points of each lane line;

[0131] The sorting module 403 is used to sort the lane lines according to the multiple first coordinate points of each lane line to determine the leftmost lane line and the rightmost lane line;

[0132] The image processing module 404 is used to: determine the area between the leftmost lane line and the rightmost lane line as the first maximum drivable area;

[0133] Projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate the second maximum drivable area in the original image and the multiple second coordinate points of each lane line in the original image;

[0134] The area outside the second largest drivable area in the original image is set to zero to obtain a preprocessed image;

[0135] The target detection module 405 is used to perform a target detection operation according to the preprocessed image.

[0136] The target detection device provided in this embodiment can execute the target detection method of the above embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0137] In the specific implementation of the aforementioned target detection device, each module can be implemented as a processor, and the processor can execute computer-executable instructions stored in the memory, so that the processor executes the above-mentioned gear control method based on a heavy-loaded vehicle.

[0138] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 50 includes: at least one processor 501 and a memory 502. The electronic device 50 also includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus 504.

[0139] In a specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the target detection method executed by the electronic device side as above.

[0140] The specific implementation process of the processor 501 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0141] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0142] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk storage.

[0143] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0144] The above-mentioned functions implemented by the electronic device and the main control device introduce the scheme provided by the embodiment of the present invention. It can be understood that in order to implement the above-mentioned functions, the electronic device or the main control device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in the embodiment of the present invention, the embodiment of the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present invention.

[0145] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned target detection method is implemented.

[0146] The computer-readable storage medium mentioned above can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0147] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in an electronic device or a main control device.

[0148] The present application also provides a computer program product, which includes: a computer program, which is stored in a readable storage medium. At least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0149] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method, characterized in that: The method comprises: Obtaining original image data of the lane where the vehicle is located; Determine each lane line in the lane according to the original image data, and generate a plurality of first coordinate points of each lane line; Sort the lane lines according to the multiple first coordinate points of each lane line to determine the leftmost lane line and the rightmost lane line; Determine an area between the leftmost lane line and the rightmost lane line as a first maximum drivable area; Projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate a second maximum drivable area in the original image and multiple second coordinate points of each lane line in the original image; Setting the area outside the second maximum drivable area in the original image to zero to obtain a preprocessed image; An object detection operation is performed according to the preprocessed image.

2. The method according to claim 1, characterized in that: The step of determining each lane line in the lane according to the original image data includes: Generate a binary image and an instance segmentation image of the original image according to the original image data; Clustering is performed on the pixels of the binary image and the instance segmented image to determine the lane line.

3. The method according to claim 2, characterized in that The clustering of the pixels of the binary image and the instance segmented image to determine the lane line includes: Clustering the pixels of the binary image and the instance segmented image, and determining each lane line in the binary image and the instance segmented image; Acquire coordinate data of a plurality of first target points on each lane line in the binary image and the instance segmented image; Convert the coordinate data of the plurality of first target points on each lane line into a bird's-eye view coordinate system to generate coordinate data of the plurality of second target points on each lane line; The lane line equation is determined by polynomial fitting according to the coordinate data of the plurality of second target points on each of the lane lines.

4. The method according to claim 3, characterized in that: The step of determining the area between the leftmost lane line and the rightmost lane line as the first maximum drivable area includes: Acquire multiple coordinate points of the first maximum drivable area according to the lane line equation coefficient of the leftmost lane line, the lane line direction coefficient of the rightmost lane line, and multiple first coordinate points of the leftmost lane line and the rightmost lane line; The first maximum drivable area is determined according to a plurality of coordinate points of the first maximum drivable area.

5. The method according to claim 3, characterized in that: Before acquiring the original image data of the lane where the vehicle is located, the method further includes: Acquire calibration image data including the lane line; An inverse perspective transformation matrix is ​​calculated based on the calibration image and the intrinsic parameter matrix and the extrinsic parameter matrix of the camera device that captures the calibration image. The inverse perspective transformation matrix is ​​used to transform the image from a perspective coordinate system to a bird's-eye view coordinate system.

6. The method according to claim 5, characterized in that The step of converting the coordinate data of the plurality of first target points on each lane line into a bird's-eye view coordinate system to generate the coordinate data of the plurality of second target points on each lane line comprises: The coordinate data of the multiple first target points on each of the lane lines are converted to the bird's-eye view coordinate system through the inverse perspective transformation matrix to generate the coordinate data of the multiple second target points on each of the lane lines.

7. The method according to claim 5, characterized in that The projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate the second maximum drivable area in the original image and the multiple second coordinate points of each lane line in the original image includes: The first maximum drivable area and the multiple first coordinate points of each lane line are projected into the original image through the inverse perspective transformation matrix to generate the second maximum drivable area and the multiple second coordinate points of each lane line in the original image.

8. A target detection device, characterized in that: The device comprises: An acquisition module is used to: acquire original image data of the lane where the vehicle is located; A coordinate determination module, used to: determine each lane line in the lane according to the original image data, and generate a plurality of first coordinate points of each lane line; A sorting module, used to: sort the lane lines according to the multiple first coordinate points of each lane line, and determine the leftmost lane line and the rightmost lane line; An image processing module, used to: determine the area between the leftmost lane line and the rightmost lane line as a first maximum drivable area; Projecting the first maximum drivable area and the multiple first coordinate points of each lane line into the original image to generate a second maximum drivable area in the original image and multiple second coordinate points of each lane line in the original image; Setting the area outside the second maximum drivable area in the original image to zero to obtain a preprocessed image; The target detection module is used to perform a target detection operation according to the preprocessed image.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images