Method and device for determining position of pavement defect

Abstract

The invention provides a method and a device for determining the position of a pavement defect. The method comprises the following steps: acquiring an image about a pavement defect through shooting equipment, acquiring point cloud data of a pavement where the pavement defect is located through radar equipment, and acquiring an internal parameter matrix and an external parameter matrix of the shooting equipment; identifying a first position coordinate of a pavement defect in the image through a defect detection model; converting the first position coordinate into a second position coordinate under a photographing equipment coordinate system according to an internal reference matrix; converting the second position coordinate into a third position coordinate in a world coordinate system according to an external parameter matrix; performing principal component analysis on the point cloud data to calculate a plane equation of the road surface; and calculating a fourth position coordinate of the pavement defect on the pavement based on the plane equation and the third position coordinate. By adopting the technical means, the problem of low pavement defect position detection precision in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle traffic technology, and in particular to a method and device for determining the location of road surface defects. Background Art

[0002] Various road surface defects will appear over time. Timely detection of road surface defects or updating of road surface defect data on the map can reduce traffic accidents and improve traffic safety. Detecting the location of road surface defects is of great significance to vehicle traffic. Currently, commonly used methods, such as the method of determining the location of road surface defects by matching road surface defect images with maps, or the method of measuring the location of road surface defects using point cloud data, all have the problem of low accuracy in determining the location of road surface defects, and a large gap between the determined location of road surface defects and the actual location of road surface defects, resulting in traffic hazards. Summary of the invention

[0003] In view of this, embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for determining a road surface defect location to solve the problem of low accuracy in detecting road surface defect locations in the prior art.

[0004] According to a first aspect of an embodiment of the present application, a method for determining the position of a road surface defect is provided, comprising: acquiring an image of a road surface defect through a photographing device, acquiring point cloud data of a road surface where the road surface defect is located through a radar device, and acquiring an internal parameter matrix and an external parameter matrix of the photographing device; identifying a first position coordinate of the road surface defect in the image through a defect detection model; converting the first position coordinate into a second position coordinate in a coordinate system of the photographing device according to the internal parameter matrix; converting the second position coordinate into a third position coordinate in a world coordinate system according to the external parameter matrix; performing principal component analysis on the point cloud data to calculate a plane equation of the road surface; and calculating a fourth position coordinate of the road surface defect on the road surface based on the plane equation and the third position coordinate.

[0005] According to a second aspect of an embodiment of the present application, a device for determining the position of a road surface defect is provided, comprising: an acquisition module, configured to acquire an image of a road surface defect through a photographing device, acquire point cloud data of a road surface where the road surface defect is located through a radar device, and acquire an intrinsic parameter matrix and an extrinsic parameter matrix of the photographing device; an identification module, configured to identify a first position coordinate of a road surface defect in the image through a defect detection model; a first conversion module, configured to convert the first position coordinate into a second position coordinate in a coordinate system of the photographing device according to the intrinsic parameter matrix; a second conversion module, configured to convert the second position coordinate into a third position coordinate in a world coordinate system according to the extrinsic parameter matrix; an analysis module, configured to perform principal component analysis on the point cloud data to calculate a plane equation of the road surface; and a calculation module, configured to calculate a fourth position coordinate of the road surface defect on the road surface based on the plane equation and the third position coordinate.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0008] Compared with the prior art, the embodiments of the present application have the following beneficial effects: obtaining images of road surface defects through a photographing device, obtaining point cloud data of the road surface where the road surface defects are located through a radar device, obtaining the internal parameter matrix and external parameter matrix of the photographing device; identifying the first position coordinates of the road surface defects in the image through a defect detection model; converting the first position coordinates into the second position coordinates in the coordinate system of the photographing device according to the internal parameter matrix; converting the second position coordinates into the third position coordinates in the world coordinate system according to the external parameter matrix; performing principal component analysis on the point cloud data to calculate the plane equation of the road surface; and calculating the fourth position coordinates of the road surface defects on the road surface based on the plane equation and the third position coordinates. The above technical means can solve the problem of low accuracy of road surface defect position detection in the prior art, thereby improving the accuracy of defect position detection and ensuring traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 It is a flow chart of a method for determining the location of a road surface defect provided in an embodiment of the present application;

[0011] Figure 2 It is a flow chart of another method for determining the location of a road surface defect provided in an embodiment of the present application;

[0012] Figure 3 It is a structural schematic diagram of a device for determining a road surface defect position provided in an embodiment of the present application;

[0013] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0015] A method and device for determining a road surface defect location according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0016] Figure 1 It is a flow chart of a method for determining the location of road defects provided in an embodiment of the present application. Figure 1 The method for determining the position of the road surface defect can be executed by a computer or a server, or by software on the computer or the server. Figure 1 As shown, the method for determining the position of the road surface defect includes:

[0017] S101, acquiring an image of a road surface defect by a photographing device, acquiring point cloud data of a road surface where the road surface defect is located by a radar device, and acquiring an internal parameter matrix and an external parameter matrix of the photographing device;

[0018] S102, identifying a first position coordinate of a road surface defect in the image by using a defect detection model;

[0019] S103, converting the first position coordinates into second position coordinates in a shooting device coordinate system according to the internal parameter matrix;

[0020] S104, converting the second position coordinates into third position coordinates in the world coordinate system according to the extrinsic parameter matrix;

[0021] S105, performing principal component analysis on the point cloud data to calculate the plane equation of the road surface;

[0022] S106, calculating a fourth position coordinate of the road surface defect on the road surface based on the plane equation and the third position coordinate.

[0023] The camera and radar devices are set on the target vehicle. The camera and radar devices can be common devices. The image and point cloud data are acquired at the same time. If the camera is always acquiring images and the radar is always acquiring point cloud data, the camera and radar devices can be synchronized according to time to match the image and point cloud data.

[0024] The defect detection model is a neural network trained to identify road surface defects in an image (both the neural network and the training method can adopt existing methods), including identifying the location coordinates and types of road surface defects in the image. Types of road surface defects include: potholes, circular or irregular depressions in the road surface; cracks, linear or mesh cracks on the road surface; rutting, permanent deformation of the road surface at the wheel tracks of vehicles; spalling, loosening and shedding of the surface material of the road surface; settlement, the entire or partial sinking of the road surface; bulges, local bulges or protrusions on the road surface; oil stains, asphalt seepage on the surface of the road surface; repair marks, marks left after repairing road surface defects, etc.

[0025] The first position coordinate is converted into the second position coordinate using the internal parameter matrix, and the second position coordinate is converted into the third position coordinate using the external parameter matrix. The third position coordinate is the road surface defect in the real world coordinate system. The plane equation of the road surface is obtained by performing principal component analysis on the point cloud data. Principal component analysis (PCA) is a statistical method used to convert a set of possibly related variables into a set of linearly independent variables, which are called principal components. In fact, point cloud data is used to solve A, B, C and D in the plane equation Ax+By+Cz+D=0. Finally, the fourth position coordinate of the road surface defect on the road surface can be calculated based on the plane equation and the third position coordinate.

[0026] According to the technical solution provided in the embodiment of the present application, an image of a road surface defect is obtained through a shooting device, point cloud data of the road surface where the road surface defect is located is obtained through a radar device, and the internal parameter matrix and external parameter matrix of the shooting device are obtained; the first position coordinate of the road surface defect in the image is identified through a defect detection model; the first position coordinate is converted into the second position coordinate in the coordinate system of the shooting device according to the internal parameter matrix; the second position coordinate is converted into the third position coordinate in the world coordinate system according to the external parameter matrix; principal component analysis is performed on the point cloud data to calculate the plane equation of the road surface; and the fourth position coordinate of the road surface defect on the road surface is calculated based on the plane equation and the third position coordinate. The above technical means can solve the problem of low accuracy of road surface defect position detection in the prior art, thereby improving the accuracy of defect position detection and ensuring traffic safety.

[0027] Further, the first position coordinates are converted into second position coordinates in the coordinate system of the shooting device according to the intrinsic parameter matrix, including: the first position coordinates include a first upper left corner coordinate, a first center coordinate and a first lower right corner coordinate, and the second position coordinates include a second upper left corner coordinate, a second center coordinate and a second lower right corner coordinate; the inverse matrix of the intrinsic parameter matrix is ​​calculated; the inverse matrix of the intrinsic parameter matrix is ​​multiplied by the first upper left corner coordinate, the first center coordinate and the first lower right corner coordinate respectively to obtain a first direction vector, a second direction vector and a third direction vector in the coordinate system of the shooting device; the second upper left corner coordinate is determined according to the first scale factor and the first direction vector; the second center coordinate is determined according to the second scale factor and the second direction vector; the second lower right corner coordinate is determined according to the third scale factor and the third direction vector.

[0028] The first upper left corner coordinate is the coordinate of the point closest to the upper left corner of the road defect in the image, the first center coordinate is the coordinate of the point closest to the center of the road defect in the image, and the first lower right corner coordinate is the coordinate of the point closest to the lower right corner of the road defect in the image. The second upper left corner coordinate, the third upper left corner coordinate, and the fourth upper left corner coordinate are all the first upper left corner coordinates of different coordinate systems. The first center coordinate is similar to the first lower right corner coordinate.

[0029] Intrinsic parameter matrix: usually expressed as K, including focal length (f x ,f y ), main point (c x ,c y ) and other parameters, K can be expressed as:

[0030]

[0031] Taking the first upper left corner coordinate as an example, multiply the inverse matrix of the intrinsic parameter matrix by the first upper left corner coordinate (u, v) to obtain the first direction vector (xn, yn, 1) in the camera coordinate system. The 1 in the first upper left corner coordinate is the depth of field coordinate value added during calculation. The calculation is as follows:

[0032]

[0033] Determine the second upper left corner coordinate p according to the first scale factor λ and the first direction vector d c , p c =λd.

[0034] Furthermore, the second position coordinates are converted into third position coordinates in the world coordinate system according to the extrinsic parameter matrix, including: the extrinsic parameter matrix includes a rotation matrix and a translation vector, and the third position coordinates include a third upper left corner coordinate, a third center coordinate and a third lower right corner coordinate; the inverse matrix of the rotation matrix is ​​calculated; and according to the inverse matrix of the rotation matrix and the translation vector, the second upper left corner coordinate, the second center coordinate and the second lower right corner coordinate are converted into the third upper left corner coordinate, the third center coordinate and the third lower right corner coordinate respectively.

[0035] The second upper left corner coordinate p c For example, the coordinates of the third upper left corner are marked as p w , according to the following formula, p c Convert to p w :

[0036] p w =R -1 (p c -t);

[0037] p w =R -1 (λd-t);

[0038] Where R is the rotation matrix and t is the translation vector. The rotation matrix R is a 3x3 orthogonal matrix that represents the rotation of the camera coordinate system relative to the world coordinate system. The translation vector t is a 3x1 vector that represents the position of the origin of the camera coordinate system in the world coordinate system. The translation vector t describes the translation of the camera in the world coordinate system. The rotation matrix and the translation vector describe the position and attitude of the camera in the world coordinate system.

[0039] Furthermore, the fourth position coordinates of the road surface defect on the road surface are calculated based on the plane equation and the third position coordinates, including: the fourth position coordinates include a fourth upper left corner coordinate, a fourth center coordinate and a fourth lower right corner coordinate; the fourth upper left corner coordinates of the road surface defect on the road surface are calculated based on the plane equation and the third upper left corner coordinates; the fourth center coordinates of the road surface defect on the road surface are calculated based on the plane equation and the third center coordinates; the fourth lower right corner coordinates of the road surface defect on the road surface are calculated based on the plane equation and the third lower right corner coordinates.

[0040] Furthermore, the fourth position coordinate of the road surface defect on the road surface is calculated based on the plane equation and the third position coordinate, including: taking the fourth center coordinate as the center position of the road surface defect on the road surface, and taking the distance between the fourth upper left corner coordinate and the fourth lower right corner coordinate as the length and width of the road surface defect on the road surface.

[0041] Finally, the center position, length and width of the road surface defect on the road surface are retained.

[0042] Furthermore, the fourth upper left corner coordinate of the road surface defect on the road surface is calculated based on the plane equation and the third upper left corner coordinate, including: solving the first proportional factor based on the plane equation and the third upper left corner coordinate; substituting the first proportional factor into the third upper left corner coordinate to obtain the fourth upper left corner coordinate.

[0043] p w =R -1 (λd-t), p w Contains three values ​​of x, y, and z. Substitute the three values ​​of x, y, and z into the plane equation Ax+By+Cz+D=0 to obtain the first scale factor λ. Then substitute the first scale factor into the third upper left corner coordinate to obtain the fourth upper left corner coordinate.

[0044] Furthermore, the fourth center coordinate of the road surface defect on the road surface is calculated based on the plane equation and the third center coordinate, including: solving the second scale factor based on the plane equation and the third center coordinate; substituting the second scale factor into the third center coordinate to obtain the fourth center coordinate.

[0045] Furthermore, the fourth lower right corner coordinate of the road surface defect on the road surface is calculated based on the plane equation and the third lower right corner coordinate, including: solving the third proportional factor based on the plane equation and the third lower right corner coordinate; substituting the third proportional factor into the third lower right corner coordinate to obtain the fourth lower right corner coordinate.

[0046] Furthermore, after calculating the fourth position coordinates of the road surface defect on the road surface based on the plane equation and the third position coordinates, the method also includes: obtaining the speed and vehicle position coordinates of the target vehicle; determining the distance between the target vehicle and the road surface defect based on the fourth position coordinates and the vehicle position coordinates; generating a control strategy for the target vehicle based on the speed and distance, and controlling the target vehicle according to the control strategy.

[0047] The control strategy includes acceleration, deceleration, and adjusting the vehicle's driving direction. For example, when the distance is greater than the preset distance, the vehicle speed can be maintained unchanged and the vehicle changes lanes to the right to avoid road defects; when the distance is less than or equal to the preset distance, the vehicle speed can be reduced and the vehicle changes lanes to the right to avoid road defects.

[0048] Furthermore, a control strategy for the target vehicle is generated based on the speed, the type of the road defect, and the distance, and the target vehicle is controlled according to the control strategy.

[0049] Different obstacle avoidance levels can be set for different types of road defects. Some types of road defects do not need to be avoided, while others do. Road defects that need to be avoided have different obstacle avoidance priorities. For example, if the priority of pothole obstacle avoidance is level one, the vehicle must avoid the pothole at a preset distance. The control strategy is to reduce the vehicle speed to a certain level, change lanes to the right, and avoid the road defect at a preset distance from the pothole. If the priority of rut obstacle avoidance is the lowest level, the vehicle does not need to be avoided, and the control strategy is to maintain the original driving state of the vehicle.

[0050] Figure 2 FIG. 1 is a flow chart of another method for determining the location of a road surface defect provided in an embodiment of the present application. Figure 2 As shown, the method includes:

[0051] S201, multiplying the inverse matrix of the intrinsic parameter matrix by the first upper left corner coordinate to obtain a first direction vector in the camera coordinate system;

[0052] S202, determining the coordinates of the second upper left corner according to the first scale factor and the first direction vector;

[0053] S203, converting the second upper left corner coordinates into the third upper left corner coordinates according to the inverse matrix of the rotation matrix and the translation vector;

[0054] S204, calculating the fourth upper left corner coordinate of the road surface defect on the road surface based on the plane equation and the third upper left corner coordinate.

[0055] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0056] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0057] Figure 3 Schematic diagram of a device for determining the location of a road surface defect provided in an embodiment of the present application. Figure 3 As shown, the device for determining the position of a road surface defect comprises:

[0058] The acquisition module 301 is configured to acquire an image of a road surface defect through a camera, acquire point cloud data of a road surface where the road surface defect is located through a radar device, and acquire an internal parameter matrix and an external parameter matrix of the camera;

[0059] The identification module 302 is configured to identify the first position coordinates of the road surface defect in the image through the defect detection model;

[0060] A first conversion module 303 is configured to convert the first position coordinates into second position coordinates in a shooting device coordinate system according to an intrinsic parameter matrix;

[0061] A second conversion module 304 is configured to convert the second position coordinates into third position coordinates in the world coordinate system according to the external parameter matrix;

[0062] The analysis module 305 is configured to perform principal component analysis on the point cloud data to calculate the plane equation of the road surface;

[0063] The calculation module 306 is configured to calculate a fourth position coordinate of the road surface defect on the road surface based on the plane equation and the third position coordinate.

[0064] According to the technical solution provided in the embodiment of the present application, an image of a road surface defect is obtained through a shooting device, point cloud data of the road surface where the road surface defect is located is obtained through a radar device, and the internal parameter matrix and external parameter matrix of the shooting device are obtained; the first position coordinate of the road surface defect in the image is identified through a defect detection model; the first position coordinate is converted into the second position coordinate in the coordinate system of the shooting device according to the internal parameter matrix; the second position coordinate is converted into the third position coordinate in the world coordinate system according to the external parameter matrix; principal component analysis is performed on the point cloud data to calculate the plane equation of the road surface; and the fourth position coordinate of the road surface defect on the road surface is calculated based on the plane equation and the third position coordinate. The above technical means can solve the problem of low accuracy of road surface defect position detection in the prior art, thereby improving the accuracy of defect position detection and ensuring traffic safety.

[0065] In some embodiments, the first conversion module 303 is also configured such that the first position coordinates include a first upper left corner coordinate, a first center coordinate and a first lower right corner coordinate, and the second position coordinates include a second upper left corner coordinate, a second center coordinate and a second lower right corner coordinate; calculate the inverse matrix of the intrinsic parameter matrix; multiply the inverse matrix of the intrinsic parameter matrix by the first upper left corner coordinate, the first center coordinate and the first lower right corner coordinate, respectively, to obtain a first direction vector, a second direction vector and a third direction vector in the coordinate system of the shooting device; determine the second upper left corner coordinate according to the first scale factor and the first direction vector; determine the second center coordinate according to the second scale factor and the second direction vector; determine the second lower right corner coordinate according to the third scale factor and the third direction vector.

[0066] The first upper left corner coordinate is the coordinate of the point closest to the upper left corner of the road defect in the image, the first center coordinate is the coordinate of the point closest to the center of the road defect in the image, and the first lower right corner coordinate is the coordinate of the point closest to the lower right corner of the road defect in the image. The second upper left corner coordinate, the third upper left corner coordinate, and the fourth upper left corner coordinate are all the first upper left corner coordinates of different coordinate systems. The first center coordinate is similar to the first lower right corner coordinate.

[0067] Intrinsic parameter matrix: usually expressed as K, including focal length (f x ,f y), main point (c x ,c y ) and other parameters, K can be expressed as:

[0068]

[0069] Taking the first upper left corner coordinate as an example, multiply the inverse matrix of the intrinsic parameter matrix by the first upper left corner coordinate (u, v) to obtain the first direction vector (xn, yn, 1) in the camera coordinate system. The 1 in the first upper left corner coordinate is the depth of field coordinate value added during calculation. The calculation is as follows:

[0070]

[0071] Determine the second upper left corner coordinate p according to the first scale factor λ and the first direction vector d c , p c =λd.

[0072] In some embodiments, the second transformation module 304 is also configured such that the external parameter matrix includes a rotation matrix and a translation vector, and the third position coordinates include a third upper left corner coordinate, a third center coordinate, and a third lower right corner coordinate; the inverse matrix of the rotation matrix is ​​calculated; and according to the inverse matrix of the rotation matrix and the translation vector, the second upper left corner coordinate, the second center coordinate, and the second lower right corner coordinate are respectively converted into the third upper left corner coordinate, the third center coordinate, and the third lower right corner coordinate.

[0073] The second upper left corner coordinate p c For example, the coordinates of the third upper left corner are marked as p w , according to the following formula, p c Convert to p w :

[0074] p w =R -1 (p c -t);

[0075] p w =R -1 (λd-t);

[0076] Where R is the rotation matrix and t is the translation vector. The rotation matrix R is a 3x3 orthogonal matrix that represents the rotation of the camera coordinate system relative to the world coordinate system. The translation vector t is a 3x1 vector that represents the position of the origin of the camera coordinate system in the world coordinate system. The translation vector t describes the translation of the camera in the world coordinate system. The rotation matrix and the translation vector describe the position and attitude of the camera in the world coordinate system.

[0077] In some embodiments, the calculation module 306 is also configured such that the fourth position coordinate includes a fourth upper left corner coordinate, a fourth center coordinate and a fourth lower right corner coordinate; calculates the fourth upper left corner coordinate of the road surface defect on the road surface based on the plane equation and the third upper left corner coordinate; calculates the fourth center coordinate of the road surface defect on the road surface based on the plane equation and the third center coordinate; and calculates the fourth lower right corner coordinate of the road surface defect on the road surface based on the plane equation and the third lower right corner coordinate.

[0078] In some embodiments, the calculation module 306 is further configured to use the fourth center coordinate as the center position of the road surface defect on the road surface, and use the distance between the fourth upper left corner coordinate and the fourth lower right corner coordinate as the length and width of the road surface defect on the road surface.

[0079] Finally, the center position, length and width of the road surface defect on the road surface are retained.

[0080] In some embodiments, the calculation module 306 is further configured to solve a first scale factor based on the plane equation and the third upper left corner coordinate; and substitute the first scale factor into the third upper left corner coordinate to obtain a fourth upper left corner coordinate.

[0081] p w =R -1 (λd-t), p w Contains three values ​​of x, y, and z. Substitute the three values ​​of x, y, and z into the plane equation Ax+By+Cz+D=0 to obtain the first scale factor λ. Then substitute the first scale factor into the third upper left corner coordinate to obtain the fourth upper left corner coordinate.

[0082] In some embodiments, the calculation module 306 is further configured to solve a second scale factor based on the plane equation and the third center coordinate; and substitute the second scale factor into the third center coordinate to obtain a fourth center coordinate.

[0083] In some embodiments, the calculation module 306 is further configured to solve a third scale factor based on the plane equation and the third lower right corner coordinate; and substitute the third scale factor into the third lower right corner coordinate to obtain a fourth lower right corner coordinate.

[0084] In some embodiments, the calculation module 306 is also configured to obtain the speed and vehicle position coordinates of the target vehicle; determine the distance between the target vehicle and the road surface defect based on the fourth position coordinates and the vehicle position coordinates; generate a control strategy for the target vehicle based on the speed and distance, and control the target vehicle according to the control strategy.

[0085] The control strategy includes acceleration, deceleration, and adjusting the vehicle's driving direction. For example, when the distance is greater than the preset distance, the vehicle speed can be maintained unchanged and the vehicle changes lanes to the right to avoid road defects; when the distance is less than or equal to the preset distance, the vehicle speed can be reduced and the vehicle changes lanes to the right to avoid road defects.

[0086] In some embodiments, the calculation module 306 is further configured to generate a control strategy for the target vehicle based on the speed, the type of the road defect, and the distance, and control the target vehicle according to the control strategy.

[0087] Different obstacle avoidance levels can be set for different types of road defects. Some types of road defects do not need to be avoided, while others do. Road defects that need to be avoided have different obstacle avoidance priorities. For example, if the priority of pothole obstacle avoidance is level one, the vehicle must avoid the pothole at a preset distance. The control strategy is to reduce the vehicle speed to a certain level, change lanes to the right, and avoid the road defect at a preset distance from the pothole. If the priority of rut obstacle avoidance is the lowest level, the vehicle does not need to be avoided, and the control strategy is to maintain the original driving state of the vehicle.

[0088] In some embodiments, the calculation module 306 is also configured to multiply the inverse matrix of the intrinsic parameter matrix by the first upper left corner coordinate to obtain a first direction vector in the coordinate system of the shooting device; determine the second upper left corner coordinate based on the first scale factor and the first direction vector; convert the second upper left corner coordinate into a third upper left corner coordinate based on the inverse matrix of the rotation matrix and the translation vector; and calculate the fourth upper left corner coordinate of the road surface defect on the road surface based on the plane equation and the third upper left corner coordinate.

[0089] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0090] Figure 4 Schematic diagram of an electronic device 4 provided in an embodiment of the present application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 401 executes the computer program 403, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0091] The electronic device 4 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 4 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 The electronic device 4 is merely an example and does not limit the electronic device 4 , and may include more or less components than those shown in the figure, or different components.

[0092] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0093] The memory 402 may be an internal storage unit of the electronic device 4, for example, a hard disk or memory of the electronic device 4. The memory 402 may also be an external storage device of the electronic device 4, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. The memory 402 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0094] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0095] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0096] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for determining the location of road surface defects, characterized in that: include: Acquire an image of a road surface defect by means of a photographing device, acquire point cloud data of a road surface where the road surface defect is located by means of a radar device, and acquire an internal parameter matrix and an external parameter matrix of the photographing device; Identifying the first position coordinates of the road surface defect in the image by a defect detection model; Converting the first position coordinates into second position coordinates in the coordinate system of the photographing device according to the intrinsic parameter matrix; Converting the second position coordinates into third position coordinates in a world coordinate system according to the external parameter matrix; Performing principal component analysis on the point cloud data to calculate the plane equation of the road surface; The fourth position coordinates of the road surface defect on the road surface are calculated based on the plane equation and the third position coordinates.

2. The method according to claim 1, characterized in that: Converting the first position coordinates into second position coordinates in the coordinate system of the shooting device according to the intrinsic parameter matrix includes: The first position coordinates include a first upper left corner coordinate, a first center coordinate, and a first lower right corner coordinate, and the second position coordinates include a second upper left corner coordinate, a second center coordinate, and a second lower right corner coordinate; Calculating the inverse matrix of the internal parameter matrix; Multiplying the inverse matrix of the intrinsic parameter matrix by the first upper left corner coordinate, the first center coordinate, and the first lower right corner coordinate respectively to obtain a first direction vector, a second direction vector, and a third direction vector in the coordinate system of the shooting device; Determine the second upper left corner coordinate according to the first scale factor and the first direction vector; determining the second center coordinates according to a second scale factor and a second direction vector; The second lower right corner coordinate is determined according to the third scale factor and the third direction vector.

3. The method according to claim 2, characterized in that Converting the second position coordinates into third position coordinates in a world coordinate system according to the external parameter matrix includes: The external parameter matrix includes a rotation matrix and a translation vector, and the third position coordinates include a third upper left corner coordinate, a third center coordinate and a third lower right corner coordinate; Calculating the inverse matrix of the rotation matrix; According to the inverse matrix of the rotation matrix and the translation vector, the second upper-left corner coordinate, the second center coordinate and the second lower-right corner coordinate are respectively converted into the third upper-left corner coordinate, the third center coordinate and the third lower-right corner coordinate.

4. The method according to claim 3, characterized in that: Calculating a fourth position coordinate of the road surface defect on the road surface based on the plane equation and the third position coordinate includes: The fourth position coordinates include a fourth upper left corner coordinate, a fourth center coordinate and a fourth lower right corner coordinate; Calculate the fourth upper left corner coordinate of the road surface defect on the road surface based on the plane equation and the third upper left corner coordinate; Calculating the fourth center coordinate of the road surface defect on the road surface based on the plane equation and the third center coordinate; The fourth lower right corner coordinate of the road surface defect on the road surface is calculated based on the plane equation and the third lower right corner coordinate.

5. The method according to claim 4, characterized in that Calculating a fourth position coordinate of the road surface defect on the road surface based on the plane equation and the third position coordinate includes: The fourth center coordinate is used as the center position of the road surface defect on the road surface, and the distance between the fourth upper left corner coordinate and the fourth lower right corner coordinate is used as the length and width of the road surface defect on the road surface.

6. The method according to claim 4, characterized in that Calculating the fourth upper left corner coordinate of the road surface defect on the road surface based on the plane equation and the third upper left corner coordinate includes: Solving the first scale factor based on the plane equation and the third upper left corner coordinate; Substituting the first scale factor into the third upper left corner coordinate, the fourth upper left corner coordinate is obtained.

7. The method according to claim 1, characterized in that After calculating the fourth position coordinates of the road surface defect on the road surface based on the plane equation and the third position coordinates, the method further includes: Get the speed and position coordinates of the target vehicle; determining a distance between the target vehicle and the road surface defect based on the fourth position coordinate and the vehicle position coordinate; A control strategy for the target vehicle is generated based on the speed and the distance, and the target vehicle is controlled according to the control strategy.

8. A device for determining the position of a road surface defect, characterized in that: include: an acquisition module, configured to acquire an image of a road surface defect through a photographing device, acquire point cloud data of a road surface where the road surface defect is located through a radar device, and acquire an internal parameter matrix and an external parameter matrix of the photographing device; an identification module, configured to identify a first position coordinate of the road surface defect in the image through a defect detection model; A first conversion module is configured to convert the first position coordinates into second position coordinates in the coordinate system of the shooting device according to the intrinsic parameter matrix; A second conversion module is configured to convert the second position coordinates into a third position coordinate in a world coordinate system according to the external parameter matrix; an analysis module, configured to perform principal component analysis on the point cloud data to calculate a plane equation of the road surface; A calculation module is configured to calculate a fourth position coordinate of the road surface defect on the road surface based on the plane equation and the third position coordinate.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images