Target detection method, medium, product, equipment and vehicle

Abstract

The invention relates to a target detection method, a medium, a product, equipment and a vehicle, and the method comprises the steps: converting a first image into a second image, carrying out the mapping conversion of a target detection result of the second image, and obtaining the position information of a detection target under a projection model of the first image; wherein the first image is an ultra-wide-angle image, and the second image is a perspective image. According to the method, an acquired original fisheye image is converted into an image similar to a public data set, and a 3D coordinate under a perspective projection model is converted into a 3D coordinate under an equidistant projection model by utilizing a corresponding relation of two imaging models, so that the 3D coordinate under the perspective projection model is converted into the 3D coordinate under the equidistant projection model under the condition of not additionally acquiring data and retraining the model; and based on the existing 3D target detection model, the effect of sensing obstacles around the vehicle is achieved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a target detection method, medium, product, device, and vehicle. Background Technology

[0002] When intelligent vehicles use assisted driving or autonomous driving, they need to detect targets in the surrounding environment using devices such as cameras, fisheye lenses, and radar to ensure that subsequent assisted driving functions can be completed accurately. In related technologies, target detection is typically achieved by labeling data from specific sensors (such as LiDAR sensors). This makes it difficult to directly reuse data from other sensors, increasing the time and manpower costs of data collection and labeling. Summary of the Invention

[0003] This application provides a target detection method, medium, product, equipment, and vehicle to solve the above-mentioned problems.

[0004] To achieve the above objectives, according to a first aspect of this application, a target detection method is provided, the method comprising:

[0005] The first image is transformed into the second image, and the target detection result of the second image is mapped and transformed to obtain the position information of the detected target under the projection model of the first image;

[0006] The first image is an ultra-wide-angle image, and the second image is a perspective image.

[0007] Optionally, converting the first image into a second image includes:

[0008] Based on the shooting angle corresponding to the first image, the first image is corrected to obtain the corrected image;

[0009] The second image is obtained based on the corrected image.

[0010] Optionally, the shooting angle is determined based on at least one of the pitch angle, roll angle, and yaw angle of the image acquisition device corresponding to the first image.

[0011] Optionally, the step of correcting the first image based on the shooting angle corresponding to the first image to obtain a corrected image includes:

[0012] Based on the shooting angle corresponding to the first image, determine the transformation matrix;

[0013] Based on the transformation matrix, the first image is corrected to obtain the corrected image.

[0014] Optionally, the transformation matrix represents the transformation relationship between the image acquisition device and the virtual camera.

[0015] Optionally, determining the transformation matrix based on the shooting angle corresponding to the first image includes:

[0016] Based on the shooting angle corresponding to the first image, the target component in the first rotation matrix of the image acquisition device corresponding to the shooting angle is adjusted to obtain the adjusted second rotation matrix corresponding to the virtual camera.

[0017] The transformation matrix is ​​obtained based on the first rotation matrix and the second rotation matrix.

[0018] Optionally, the step of correcting the first image based on the transformation matrix to obtain the corrected image includes:

[0019] Based on the distortion parameters of the image acquisition device corresponding to the first image, the first image is parameter-corrected to obtain the parameter-corrected image.

[0020] Based on the transformation matrix, pose correction is performed on the parameter-corrected image to obtain a pose-corrected image, which is used as the corrected image.

[0021] Optionally, the step of performing parameter correction on the first image based on the distortion parameters of the image acquisition device corresponding to the first image to obtain a parameter-corrected image includes:

[0022] Based on the distortion parameters, the first image is projected onto the camera coordinate system of the image acquisition device to obtain an image after parameter correction.

[0023] Optionally, the step of performing angle correction on the parameter-corrected image based on the transformation matrix to obtain the pose-corrected image includes:

[0024] Based on the transformation matrix, the parameter-corrected image is projected from the camera coordinate system of the image acquisition device to the camera coordinate system of the virtual camera to obtain the pose-corrected image.

[0025] Optionally, obtaining the second image based on the corrected image includes:

[0026] The corrected image is cylindrically unfolded to obtain the second image.

[0027] Optionally, the target detection result of the second image is obtained in the following way:

[0028] The second image is processed based on the object detection model to obtain the object detection result of the second image.

[0029] Optionally, the target detection result is the detection result under the perspective model, and the mapping transformation of the target detection result of the second image to obtain the position information of the detected target under the projection model of the first image includes:

[0030] The target detection result is mapped from the perspective model to the projection model to obtain the position information of the detected target under the projection model of the first image.

[0031] Optionally, transforming the first image into the second image includes:

[0032] Based on the first image and a preset mapping relationship, an associated image is obtained that is related to the first image, thereby determining the second image.

[0033] The preset mapping relationship is used to indicate the associated image corresponding to each of the multiple preset images.

[0034] Optionally, the method is applied to a vehicle, and the method further includes:

[0035] The location information of the detected target is converted to the vehicle coordinate system in order to control the vehicle.

[0036] Optionally, the first image is obtained by taking a fisheye camera.

[0037] According to a second aspect of this application, embodiments of this application also provide an electronic device, comprising:

[0038] A memory on which computer programs are stored;

[0039] A processor is configured to execute the computer program in the memory to implement the steps of any of the methods provided in the embodiments of this application.

[0040] According to a third aspect of this application, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods provided in the embodiments of this application.

[0041] According to a fourth aspect of this application, embodiments of this application also provide a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of any of the methods provided in embodiments of this application.

[0042] According to a fifth aspect of this application, embodiments of this application also provide a vehicle including the aforementioned target detection device, or, as described, an electronic device, or, performing the steps of any of the methods provided in embodiments of this application.

[0043] Some embodiments of this specification include at least the following beneficial effects: by converting ultra-wide-angle images into perspective images, existing target detection models can be reused, avoiding the complexity and high cost of designing and training models separately for ultra-wide-angle images, which not only saves development time but also improves development efficiency; by accurately mapping the target detection results on the perspective image back to the spatial coordinate system of the original first image, high-precision position information is ensured, which is helpful for applications in precise positioning scenarios (such as obstacle localization in autonomous driving).

[0044] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0047] Figure 1 These are application scenario diagrams of the target detection method shown in some embodiments of this specification;

[0048] Figure 2 This is an exemplary flowchart of a target detection method according to some embodiments of this specification;

[0049] Figure 3 This is an exemplary schematic diagram of the vehicle coordinate system and the camera coordinate system of the image acquisition device according to some embodiments of this specification;

[0050] Figure 4 This is an exemplary schematic diagram showing the transformation relationship between the vehicle coordinate system and the coordinate system of the image acquisition device according to some embodiments of this specification;

[0051] Figure 5 This is an exemplary schematic diagram showing the transformation relationship between the coordinate systems of an image acquisition device and a virtual camera according to some embodiments of this specification;

[0052] Figure 6 This is an exemplary flowchart illustrating the transformation between the coordinate systems of an image acquisition device and a virtual camera according to some embodiments of this specification;

[0053] Figure 7 This is an exemplary schematic diagram showing the unfolding of a cylindrical surface according to some embodiments of this specification;

[0054] Figure 8 This is an exemplary schematic diagram showing the correspondence between the equidistant projection model and the pinhole perspective imaging model according to some embodiments of this specification;

[0055] Figure 9 This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification;

[0056] Figure 10 This is an exemplary schematic diagram of a vehicle according to some embodiments of this specification. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0058] To facilitate understanding of the implementation schemes provided in this application, the relevant application background of the target detection method provided in this application will be explained first.

[0059] With the development of autonomous driving technology and intelligent driving assistance systems, current blind spot detection (BSD) systems typically use millimeter-wave radar and ultrasonic radar to detect targets and provide warnings for vehicles. However, they cannot perceive the types of obstacles in the surrounding environment. Fisheye cameras, on the other hand, can provide image data with a wide field of view and no blind spots. Due to their low cost, fisheye cameras are widely deployed in various vehicle models and are an important perception sensor in low- and mid-range vehicles.

[0060] Related patents propose using a vehicle-mounted surround-view fisheye camera for target detection to obtain the 3D coordinates, length, width, height, and corresponding yaw angle of obstacles. However, the installation angle of fisheye cameras varies between different vehicles, resulting in significant differences in the images of the targets to be detected, making it impossible to directly reuse the 3D target detection model. Furthermore, due to the lack of publicly available datasets containing data from various sensors, the 3D target detection model requires annotation using point cloud data collected by LiDAR sensors, and the detection box needs to be transformed into the camera coordinate system corresponding to the fisheye camera using calibrated external parameters. This requires manual data collection and annotation, which not only increases the time and manpower costs of data collection and annotation but also enhances the complexity and resource consumption of data preparation.

[0061] In view of this, some embodiments of this specification provide a target detection method that, based on the isometric projection model of fisheye imaging and the perspective projection model of pinhole perspective imaging, determines the correspondence between the target detection boxes in the two imaging models, enabling direct reuse of publicly available perspective image datasets, while also enabling the reuse of existing 3D target detection models; furthermore, a horizontal virtual camera is constructed and the calibrated image is cylindrically unfolded to transform the acquired original fisheye image into an image similar to the publicly available dataset. Utilizing the correspondence between the two imaging models, the 3D coordinates under the perspective projection model are converted into 3D coordinates under the isometric projection model, thereby achieving the effect of vehicle perimeter obstacle perception based on an existing 3D target detection model without additional data acquisition and model retraining.

[0062] Figure 1 These are application scenario diagrams of the target detection method shown in some embodiments of this specification.

[0063] like Figure 1 As shown, application scenarios for target detection methods can include image acquisition systems and terminal devices. Image acquisition systems and terminal devices can communicate via wired or wireless means.

[0064] An image acquisition system is used to acquire images of the environment surrounding a vehicle.

[0065] The terminal device processes data and / or information obtained from other devices or systems. The control device can execute program instructions based on this data, information, and / or processing results to perform one or more functions described in this application. In some embodiments, the control device may include one or more sub-processing devices (e.g., a single-core processing device or a multi-core multi-chip processing device). As an example only, a Central Processing Unit (CPU), an Application-Specific Integrated Circuit (ASIC), an Application-Specific Instruction-set Processor (ASIP), a Graphics Processing Unit (GPU), or any combination thereof may be used.

[0066] An image acquisition system can consist of multiple image acquisition devices. These devices can be configured to acquire images from multiple preset directions. These preset directions can be set according to actual needs. For example, the preset directions could be the front of the vehicle, the rear of the vehicle, and the left and right sides.

[0067] In some embodiments, the aforementioned plurality of image acquisition devices may be cameras of the same specification. For example, an image acquisition system may consist of multiple fisheye cameras.

[0068] In some embodiments, the plurality of cameras may be cameras of different specifications.

[0069] For example, an image acquisition system can consist of one or more fisheye cameras and one or more pinhole cameras.

[0070] In some embodiments, the image acquisition system may be mounted on a vehicle. The vehicle may be a land vehicle or a non-land vehicle.

[0071] The aforementioned land vehicles may include passenger cars, full-size sport utility vehicles (SUVs), vans, trucks, minivans, buses, motorcycles, bicycles, scooters, trains, snowmobiles, wheeled vehicles, tracked vehicles, or rail vehicles.

[0072] The aforementioned non-land vehicles may include drones, airplanes, hovercraft, spacecraft, ships, and sailboats.

[0073] In some embodiments, the image acquisition system may include four fisheye cameras arranged around the vehicle body.

[0074] The aforementioned terminal devices can be terminals or servers. Terminals can be in-vehicle terminals, smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, etc., but are not limited to these. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services.

[0075] Image acquisition systems and terminal devices can communicate via wired or wireless means.

[0076] It should be noted that the above wireless methods can achieve communication through a communication network, which can be a local area network (LAN), a wide area network (WAN) relayed through a relay device, or a combination of LAN and WAN.

[0077] The target detection method provided in this application can be applied to vehicle driving scenarios, as well as other application scenarios, such as vehicle parking.

[0078] It is important to note that the application scenarios of the object detection method are provided for illustrative purposes only and are not intended to limit the scope of this specification. Those skilled in the art can make various changes and modifications based on the description in this specification. For example, application scenarios may also include databases, information sources, etc. Furthermore, application scenarios may be implemented on other devices to achieve similar or different functions. However, these changes and modifications will not depart from the scope of this specification.

[0079] Figure 2 This is an exemplary flowchart of a target detection method according to some embodiments of this specification. In some embodiments, process 200 may be executed based on a terminal device. Figure 2 As shown, process 200 includes the following steps.

[0080] Step 210: Transform the first image into the second image to perform a mapping transformation on the target detection result of the second image, and obtain the position information of the detected target under the projection model of the first image.

[0081] The first image is an ultra-wide-angle image, and the second image is a perspective image.

[0082] Ultra-wide-angle images refer to photos or videos taken using ultra-wide-angle lenses with a large angle of view (such as 120 degrees or 180 degrees). However, due to their wide angle of view, ultra-wide-angle images taken with ultra-wide-angle lenses exhibit some distortion, especially at the edges, where stretching or distortion may occur. Fisheye lenses are a type of ultra-wide-angle lens that can provide a field of view exceeding 180 degrees, but they also produce noticeable image distortion.

[0083] In some embodiments, the projection model of the first image may include an equidistant projection model, an equal solid angle projection model, a stereoscopic projection model, etc.

[0084] A perspective image is an image generated based on a perspective projection model. In some embodiments, the second image may be a simplified or approximate perspective image.

[0085] In some embodiments, the first image may be an image taken with a fisheye camera without distortion correction.

[0086] In some embodiments, the second image is obtained by performing a perspective transformation on the first image to simulate the perspective effect of the human eye or a regular camera. The second image can be used to correct image distortion caused by mounting position or lens distortion.

[0087] Due to its unique imaging principle, the first image exhibits significant radial and tangential distortion. Distortion correction methods, such as those based on the intrinsic parameters and distortion parameters of the fisheye camera, are needed to correct the distorted fisheye image into an approximately distortion-free image.

[0088] In some embodiments, the first image may need to be transformed by perspective to convert it into a second image. Perspective transformation can be achieved using a perspective transformation matrix, which represents the mapping relationship between points in the first image and points in the second image.

[0089] In some embodiments of this specification, an ultra-wide-angle image can capture a wider field of view. After converting the ultra-wide-angle image into a perspective image, it helps to obtain information about the vehicle's surroundings and reduce blind spots. By performing target detection on the second image, obstacles around the vehicle, such as pedestrians, vehicles, and road signs, can be detected and located, which helps in the intelligent control of the vehicle.

[0090] In some embodiments, converting the first image into a second image includes:

[0091] Based on the shooting angle corresponding to the first image, the first image is corrected to obtain the corrected image;

[0092] A second image is obtained based on the corrected image.

[0093] The shooting angle refers to the angle between the overall pose of the image acquisition device that captures the first image and the reference plane. Different shooting angles corresponding to the first image will cause distortion of objects in the captured image (such as objects near the tilt direction being elongated).

[0094] Pose includes the position (coordinates in three-dimensional space) and orientation (rotation angle, which can be represented by Euler angles or quaternions) of the image acquisition device.

[0095] The reference plane can be a horizontal plane, a vertical plane, etc.

[0096] In some embodiments, the shooting angle is determined based on at least one of the pitch angle, roll angle, and yaw angle of the image acquisition device corresponding to the first image.

[0097] The pitch angle indicates the vertical tilt angle of the image acquisition device (e.g., the image acquisition device is tilted up or down).

[0098] The roll angle indicates the angle at which the image acquisition device rotates around the optical axis (e.g., the image acquisition device tilts left or right).

[0099] Yaw angle indicates the angle of horizontal rotation of the image acquisition device (e.g., the image acquisition device turns left or right).

[0100] In some embodiments, the image acquisition device can be calibrated to obtain calibration parameters, and the tilt matrix corresponding to the first image can be determined based on the calibration parameters.

[0101] Calibration parameters can include any one or a combination of intrinsic parameter matrices, extrinsic parameter matrices, distortion coefficients, etc. The intrinsic parameter matrix describes the internal parameters of the image acquisition device (such as focal length, principal point coordinates, etc.).

[0102] The extrinsic parameter matrix is ​​used to describe the orientation (rotation and translation) of the image acquisition device relative to a reference coordinate system (e.g., the world coordinate system).

[0103] Distortion coefficients are used to describe the radial and tangential distortion of an image acquisition device.

[0104] In some embodiments, the extrinsic parameter matrix may include a rotation matrix R and a translation vector t, wherein the rotation matrix R reflects the shooting angle of the image acquisition device at the time of installation.

[0105] The corrected image refers to the image obtained after correcting the first image.

[0106] In some embodiments, the first image is transformed and mapped to a virtual coordinate system (e.g., the virtual coordinate system of a virtual camera) to eliminate the effects of pose deviation of the image acquisition device.

[0107] In some embodiments, a second image is obtained based on the corrected image and the relationship between the isometric projection model of fisheye imaging and the perspective projection model of pinhole imaging.

[0108] In some embodiments of this specification, by correcting the shooting angle, additional distortion caused by improper installation of the image acquisition device or lens can be effectively reduced, making the image closer to the perspective of the real world; in the corrected image, the proportion and shape of objects in the image are more in line with the actual situation, which helps to improve the accuracy of subsequent image processing tasks (such as target detection, recognition, etc.).

[0109] In some embodiments, the first image is corrected based on the shooting angle corresponding to the first image to obtain a corrected image, including:

[0110] The transformation matrix is ​​determined based on the shooting angle corresponding to the first image;

[0111] Based on the transformation matrix, the first image is corrected to obtain the corrected image.

[0112] The transformation matrix is ​​a parameter used to correct distortion in the first image. It maps pixel coordinates in the first image to the corrected image coordinates.

[0113] In some embodiments, the corrected image coordinates can be the virtual coordinate system of the virtual camera or the coordinate system under the perspective projection model, wherein the virtual camera has a zero shooting angle relative to the image acquisition device.

[0114] In some embodiments, the transformation matrix represents the transformation relationship between the image acquisition device and the virtual camera.

[0115] In some embodiments, the transformation matrix may include a rotation matrix and a translation matrix. For example, the rotation and translation matrices may be based on the relative attitude differences (i.e., pitch, roll, and yaw angles) between the image acquisition device and the virtual camera.

[0116] In some embodiments, calibration parameters of the image acquisition device are obtained through calibration. For example, this can be accomplished by photographing a known geometric pattern (such as a checkerboard).

[0117] In some embodiments, the calibration parameters of the virtual camera can be predefined, such as the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of the virtual camera. The calibration parameters of the virtual camera can be designed according to application requirements.

[0118] In some embodiments, a transformation matrix can be calculated based on the calibration parameters of the image acquisition device and the virtual camera, and the first image can be converted into a corrected image using the transformation matrix.

[0119] In some embodiments of this specification, the rotation, translation, and projection relationships between the fisheye camera and the target coordinate system (such as a virtual camera) are determined by defining the transformation matrix (extrinsic parameters, intrinsic parameters, etc.), thus avoiding errors caused by empirical correction. By combining the projection model of the fisheye camera (such as isometric or stereoscopic projection) and the transformation matrix, geometric deformation caused by lens distortion or installation tilt can be accurately eliminated. In the vehicle surround view system, the images of multiple fisheye cameras are mapped to a unified coordinate system by the transformation matrix, which helps to eliminate splicing misalignment.

[0120] In some embodiments, determining the transformation matrix based on the shooting angle corresponding to the first image includes:

[0121] Based on the shooting angle corresponding to the first image, the target component corresponding to the shooting angle in the first rotation matrix of the image acquisition device is adjusted to obtain the adjusted second rotation matrix corresponding to the virtual camera.

[0122] The transformation matrix is ​​obtained based on the first and second rotation matrices.

[0123] The first rotation matrix of the image acquisition device is used to describe the rotation of the camera coordinate system of the image acquisition device relative to the reference coordinate system (e.g., the world coordinate system).

[0124] The adjusted second rotation matrix is ​​used to describe the rotation of the virtual camera's camera coordinate system relative to a reference coordinate system (e.g., the world coordinate system).

[0125] A rotation matrix can be represented by three rotation angles (Euler angles): the rotation angle about the x-axis, the rotation angle about the y-axis, and the rotation angle about the z-axis. Depending on the rotation order, there are multiple representations, such as ZYX order, XYZ order, etc.

[0126] Figure 3 This is an exemplary schematic diagram of the vehicle coordinate system and the camera coordinate system of the image acquisition device according to some embodiments of this specification.

[0127] In some embodiments, such as Figure 3 As shown, the image acquisition device is a fisheye camera mounted on the vehicle, and the reference coordinate system is the Vehicle Coordinate System (VCS), which describes the vehicle's position and orientation in three-dimensional space. For example, the VCS is defined as follows: the X-axis points forward of the vehicle, consistent with its direction of travel; the Y-axis is perpendicular to the X-axis and points to the left of the vehicle; the Z-axis is perpendicular to both the X and Y axes, pointing upwards and perpendicular to the ground; the origin of the VCS (0, 0, 0) is defined at the center point of the rear axle of the vehicle. The fisheye camera can be mounted on the front / rear bumpers and the left / right rearview mirrors. In autonomous driving and advanced driver assistance systems (ADAS), surround-view fisheye cameras are crucial for achieving functions such as automatic parking, obstacle detection, and blind spot monitoring.

[0128] It should be noted that fisheye cameras have a wider field of view than traditional cameras, typically reaching 180 degrees or more. During installation, the shooting angle of the fisheye camera should be considered to cover blind spots around the vehicle. The calibration process determines the intrinsic parameter matrix (e.g., focal length, principal point coordinates) and extrinsic parameter matrix (e.g., the position and rotation of the image acquisition device relative to the vehicle coordinate system). For example, the camera coordinate system of the image acquisition device (e.g.,...) Figure 3 The camera coordinate system shown is defined as follows: the X-axis points to the right side of the image acquisition device plane; the Y-axis is perpendicular to the X-axis and points below the image acquisition device plane; the Z-axis is perpendicular to both the X-axis and Y-axis and points in front of the image acquisition device, aligned with the optical axis of the image acquisition device; the origin (0, 0, 0) of the camera coordinate system of the image acquisition device is defined at the center point of the image acquisition device.

[0129] Figure 4 This is an exemplary schematic diagram showing the transformation relationship between the vehicle coordinate system and the coordinate system of the image acquisition device according to some embodiments of this specification.

[0130] In some embodiments, such as Figure 4As shown, after installation, the image acquisition device needs to be calibrated to determine its intrinsic and extrinsic parameter matrices. For example, the intrinsic parameter calibration process is as follows: acquire a preset checkerboard or dot grid calibration pattern; capture multiple images from different angles and positions; detect the corner points of the checkerboard or dot grid from the multiple images; determine the three-dimensional coordinates of the corresponding points; use an optimization algorithm to ensure the reprojection error is within acceptable limits, and solve for the intrinsic and extrinsic parameter matrices of the image acquisition device. Extrinsic parameter calibration requires placing a calibration plate on the ground around the vehicle to determine the three-dimensional coordinates of the corner points in the VCS coordinate system and the two-dimensional coordinates in the corresponding camera images. An optimization algorithm is used to solve for the extrinsic parameters and verify whether the error is within acceptable limits. The rotation matrix in the calibrated extrinsic parameter matrix of the image acquisition device is then set to Rreal.

[0131] Figure 5 This is an exemplary schematic diagram illustrating the transformation relationship between the coordinate systems of an image acquisition device and a virtual camera according to some embodiments of this specification.

[0132] In some embodiments, such as Figure 5 As shown, the image acquisition device needs to be installed to ensure there are no blind spots around the vehicle. The device is tilted downwards with a certain pitch angle, and the resulting image distortion needs to be eliminated. A virtual camera needs to be constructed, rotated a certain angle around the camera's X-axis, with its Y-axis pointing downwards and perpendicular to the ground, and its Z-axis pointing forward and parallel to the ground. This ensures that the imaging angle of the image acquisition device matches the imaging angle of the camera in the publicly available dataset. The rotation matrix in the extrinsic parameter matrix of the virtual camera relative to the vehicle coordinate system is Rvirtual.

[0133] Figure 6 This is an exemplary flowchart illustrating the transformation between the coordinate systems of an image acquisition device and a virtual camera according to some embodiments of this specification.

[0134] In some embodiments, such as Figure 6 As shown, the rotation matrix Rreal from the image acquisition device to the vehicle coordinate system is obtained (as in step 610). This rotation matrix is ​​a 3-DOF rotation vector.

[0135] In some embodiments, the rotation matrix Rreal is represented using Euler angles, with the rotation order being Z×Z (as in step 620). Three rotation angles are obtained, denoted as α, β, and γ, respectively, where the rotation matrix Rreal is... The 3-DOF rotation vector is decomposed into the product of 3 rotation matrices, where R X (β) represents the rotation matrix corresponding to the pitch angle.

[0136] In some embodiments, the Euler angle β corresponding to the rotation around the X-axis obtained by solving is set to 0, resulting in a new rotation matrix Rvirtual (as in step 630). With the virtual camera's Y-axis pointing downwards, the Euler angle form is converted back to a rotation matrix, resulting in a horizontal virtual camera rotation matrix Rvirtual. The virtual camera is a virtual camera with a pitch angle of 0, thereby eliminating the influence of the camera's installation pitch angle.

[0137] In some embodiments, the transformation relationship between the horizontal virtual camera and the image acquisition device is calculated (as in step 640). The transformation matrix Rvirtual2real is obtained. virtual2real =R virtual .inv()*R real The transformation matrix is ​​used to transform three-dimensional coordinates between the coordinate systems of the two virtual cameras and the image acquisition device.

[0138] In some embodiments of this specification, by measuring the orientation of the camera coordinate system of the image acquisition virtual camera relative to a reference coordinate system (e.g., the world coordinate system), the transformation relationship between the image acquisition device and the camera coordinate system of the virtual camera can be accurately obtained, which helps to correct the first image.

[0139] In some embodiments, the first image is corrected based on the transformation matrix to obtain a corrected image, including:

[0140] Based on the distortion parameters of the image acquisition device corresponding to the first image, parameter correction is performed on the first image to obtain the parameter-corrected image;

[0141] Based on the transformation matrix, pose correction is performed on the parameter-corrected image to obtain the pose-corrected image, which is then used as the corrected image.

[0142] Distortion parameters are parameters used to describe the imaging distortion characteristics of an image acquisition device. For example, distortion parameters may include radial distortion parameters.

[0143] In some embodiments, pixels in a distorted image can be mapped to corresponding positions in a distortion-free image based on distortion parameters and a distortion model.

[0144] In some embodiments, an equidistant projection model is established based on the distortion coefficient and the principle of equidistant projection; each pixel in the acquired first image is corrected according to the equidistant projection model to obtain a parameter-corrected image.

[0145] The equidistant projection model is based on the distortion coefficients of a fisheye camera and the principle of equidistant projection. The image acquisition device is a camera using a fisheye lens. Fisheye lenses are used to capture a large field of view, which can be 180-270 degrees. This manual does not specify a particular viewing angle for the image acquisition device.

[0146] Because errors occur during lens manufacturing and camera installation in different vehicle image acquisition devices, the intrinsic and extrinsic parameters and distortion coefficients of different cameras vary. To obtain the distortion coefficients of the image acquisition device, the vehicle's camera can be calibrated.

[0147] The principle of equidistant projection can be expressed as shown in formula (1):

[0148] rd = fθ; (1)

[0149] Where rd represents the distance from the midpoint of the image to the center of distortion, i.e., the imaging height; f is the focal length of the image acquisition device; and θ is the angle between the incident ray and the optical axis of the image acquisition device, i.e., the incident angle.

[0150] Considering the distortion characteristics of light entering the image acquisition device, such as refraction and non-linear propagation, and the fact that actual lenses may not precisely conform to any mathematical projection model due to various reasons, such as lens manufacturing errors, we consider taking the first five terms of the Taylor expansion of rd with respect to θ to approximate the actual projection function of the image acquisition device, as shown in formula (2):

[0151] rd=θ(1+k1θ 2 +k2θ 4 +k3θ 6 +k4θ 8 (2)

[0152] Where k1, k2, k3, and k4 are the distortion coefficients corresponding to the quadratic, quartic, hexatic, and octogentic terms in the Taylor expansion of θ, respectively.

[0153] In some embodiments of this specification, an equidistant projection model is established based on the distortion coefficient and the principle of equidistant projection; each pixel in the first image acquired by the image acquisition device is corrected according to the equidistant projection model, thereby effectively correcting the distorted fisheye image and obtaining the first image after parameter correction, which facilitates the identification and processing of subsequent pixels.

[0154] In some embodiments, parameter correction is performed on the first image based on the distortion parameters of the image acquisition device corresponding to the first image to obtain a parameter-corrected image, including:

[0155] Based on the distortion parameters, the first image is projected onto the camera coordinate system of the image acquisition device to obtain the parameter-corrected image.

[0156] In some embodiments, the parameter-corrected image is an image in the camera coordinate system of the image acquisition device.

[0157] For example, a system of equations can be established based on formula (3):

[0158]

[0159] By solving the system of equations, the three-dimensional coordinates of the pixel position (u′, v′) in the first image in the camera coordinate system of the corresponding image acquisition device can be obtained as (X... c Y c Z c ), where X′, Y′ are normalized coordinates, ψ is the incident ray angle, and ψ′ is the distorted incident angle.

[0160] In some embodiments, angle correction is performed on the parameter-corrected image based on the transformation matrix to obtain a pose-corrected image, including:

[0161] Based on the transformation matrix, the parameter-corrected image is projected from the camera coordinate system of the image acquisition device to the camera coordinate system of the virtual camera to obtain the pose-corrected image.

[0162] In some embodiments, a transformation matrix is ​​used to map each pixel in the parameter-corrected image to its corresponding position in the camera coordinate system of the virtual camera.

[0163] For example, the pose-corrected image can be obtained using the following formula (4):

[0164]

[0165] Where H is the transformation matrix, (X c Y c Z c (X) represents the three-dimensional coordinates of the image acquisition device's camera coordinate system. c ′, Y c ′, Z c ′) represents the three-dimensional coordinates of the virtual camera in the camera coordinate system.

[0166] In some embodiments of this specification, distortion parameter correction can correct the imaging distortion of the image acquisition device, which helps to improve the quality of the first image and enhance the accuracy of target detection and recognition; transformation matrix can help adapt to image acquisition devices with different installation positions and angles, and improve the applicability of the target detection model.

[0167] In some embodiments, a second image is obtained based on the corrected image, including:

[0168] The corrected image is then subjected to cylindrical unfolding to obtain the second image.

[0169] Cylindrical unrolling refers to mapping each pixel in the second image onto a cylindrical surface to obtain a cylindrical image.

[0170] Figure 7 This is an exemplary schematic diagram showing the unfolding of a cylindrical surface according to some embodiments of this specification;

[0171] like Figure 7 As shown, by projecting a 3D scene image onto a cylinder, vertical objects in the scene can remain vertical in the cylindrical image. Objects in the cylindrical image are similar to those in the perspective image. When an object moves along the vertical and azimuth axes, if the object has a constant radial distance (i.e., its distance from the center of the cylinder remains constant), then in the cylindrical projection, the object will be projected onto an image with a constant shape. A vertical line at any position in the 3D scene image is projected as a vertical line in the cylindrical image; a horizontal plane at any position in the 3D scene image is projected as a straight line across the center of the cylindrical image (all other horizontal lines become curves). Cylindrical images can be created by a perspective camera, which captures pixels in the scene column by column by rotating the perspective camera around an axis (e.g., the vertical axis), thus generating a cylindrical projection image.

[0172] For example, for each coordinate point (u,v) on the cylindrical image, the transformation relationship between this coordinate point and the three-dimensional coordinate point in the camera coordinate system of the virtual camera is shown in formula (5):

[0173]

[0174] in, Let f be the angle between the line connecting the optical center of the virtual camera to the 3D coordinate point and the optical axis in the horizontal direction, and f = f x =f y c is the focal length in pixels. x ,c y This represents the position of the origin of the image coordinate system for the cylindrical image in the pixel coordinate system.

[0175] In some embodiments of this specification, simplified or similar quasi-perspective images are obtained by cylindrical unfolding, which helps to reuse existing object detection models, facilitates the use of publicly available perspective image datasets, eliminates the need for data collection and annotation training, saves the data annotation and training process, and greatly shortens the development process.

[0176] In some embodiments, the target detection results of the second image are obtained in the following ways:

[0177] The second image is processed based on the object detection model to obtain the object detection result of the second image.

[0178] An object detection model is an algorithm or model used to detect objects.

[0179] In some embodiments, the object detection model is a machine learning model. For example, the object detection model may include any one or a combination of Convolutional Neural Networks (CNN) models, Neural Networks (NN) models, or other custom model structures.

[0180] In some embodiments, the input to the object detection model includes a second image, and the output may include object detection results.

[0181] In some embodiments, the object detection model can be trained using a large number of labeled training samples through various feasible methods. For example, parameters can be updated using gradient descent. An exemplary training process includes: inputting multiple labeled training samples into an initial model; constructing a loss function using the labels and the results of the initial object detection model; and iteratively updating the parameters of the initial object detection model based on the loss function using gradient descent or other methods. The model training is complete when preset conditions are met, resulting in a trained object detection model. These preset conditions may include loss function convergence, the number of iterations reaching a threshold, etc.

[0182] In some embodiments, the training samples include at least sample perspective images. The training samples may be obtained based on historical data.

[0183] In some embodiments, the labels may include the actual target detection results corresponding to the training samples. Labels can be obtained automatically or manually.

[0184] In some embodiments, the target objects (such as vehicles, pedestrians, buildings, etc.) in the image can be located and identified based on the target detection model, and multiple detection boxes of the target objects can be generated as the target detection results.

[0185] In some embodiments, the object detection model can output relevant information about the detection boxes. For example, the location information of the detection boxes in the second image can be represented by a set of coordinates, such as the coordinates of the top-left and bottom-right corners of the detection box, or by using the coordinates of the center point of the detection box along with its width and height. The size of the detection box, i.e., its width and height, reflects the actual size of the detected target in the second image. Another example is the category label associated with each detection box, indicating the category to which the detected target contained within the box belongs (e.g., "pedestrian," "animal," etc.). Yet another example is a confidence score, a value between 0 and 1, representing the degree of confidence in the detection result. A high score indicates a high probability that the detection result is correct, while a low score indicates a low probability.

[0186] In some embodiments of this specification, existing target detection models are used to obtain actual detection boxes under equidistant projection. This enables seamless insertion of existing target detection models without the need for retraining the model using self-collected data. This helps improve the technical effect of perceiving obstacles around vehicles, eliminates the data annotation and training process, and greatly shortens the development process.

[0187] In some embodiments, the target detection result is the detection result under the perspective model. A mapping transformation is performed on the target detection result of the second image to obtain the position information of the detected target under the projection model of the first image, including:

[0188] The target detection results are mapped from the perspective model to the projection model to obtain the position information of the detected target under the projection model of the first image.

[0189] A perspective model is the imaging model corresponding to a perspective image, such as a perspective projection model. A projection model is the imaging model used by an image acquisition device, such as an isometric projection model.

[0190] Figure 8 This is an exemplary schematic diagram showing the correspondence between the isometric projection model and the pinhole perspective imaging model according to some embodiments of this specification.

[0191] In some embodiments, such as Figure 8 As shown, θ is the angle between the line connecting the optical center to the three-dimensional coordinate point and the optical axis in the horizontal direction under the isometric projection model of the image acquisition device (e.g., a fisheye camera). In the perspective projection model of pinhole imaging, the angle between the line connecting the optical center to the three-dimensional coordinate point and the optical axis in the horizontal direction is given.

[0192] The actual coordinates of the vehicle in the camera coordinate system of the image acquisition device (e.g., a fisheye camera) are [X... C Y C Z C ] TThe target detection result obtained by the target detection model inference includes the detection box of the detected target, which can be represented as a three-dimensional coordinate point [X′]. CC Y′ CC Z′ CC The target detection model outputs detection boxes based on perspective images, requiring the inference results to be converted to the isometric projection model of the image acquisition device (e.g., a fisheye camera).

[0193] In some embodiments, the image acquisition device uses an isometric projection model R = fθ, and the perspective lens uses a perspective projection model R = f tanθ. Based on the transformation relationship between the two projection models, the relationship between the three-dimensional coordinates of the detected target under the isometric projection model and the three-dimensional coordinates inferred by the target detection model is obtained, as shown in formula (6):

[0194]

[0195] in

[0196] In some embodiments of this specification, by using an existing target detection model, the output of the target detection model is transformed into virtual and real coordinates to obtain the actual detection box under equidistant projection. In target tracking and localization tasks, the actual detection box under equidistant projection can provide more accurate target location information.

[0197] In some embodiments, transforming the first image into the second image includes:

[0198] Based on the first image and a preset mapping relationship, associated images related to the first image are obtained to determine the second image.

[0199] Among them, the preset mapping relationship is used to indicate the associated image corresponding to each of the multiple preset images.

[0200] The preset mapping relationship describes the correspondence between points in the first image and points in the second image. The preset mapping relationship can be obtained by pre-calibration.

[0201] In some embodiments, a mapping relationship from the coordinates of the first image to the coordinates of the second image can be established based on the imaging model of the image acquisition device (such as an isometric projection model, an isometric projection model, etc.).

[0202] In some embodiments, feature points can be extracted from the first image and the second image to determine the correspondence between the two feature points and establish a preset mapping relationship.

[0203] In some embodiments, each pixel in the first image can be mapped to a corresponding position in the second image according to a preset mapping relationship.

[0204] In some embodiments, the transformation matrix can be pre-calibrated and stored for direct use during real-time processing, avoiding redundant calculations. By modifying the parameters of the transformation matrix (such as rotation angle), different mounting postures or virtual camera configurations can be quickly simulated without recalibrating the hardware.

[0205] In some embodiments of this specification, a preset mapping relationship can transform the coordinates in the first image to the second image that the object detection model can process, thereby improving the model's generalization ability to different perspectives and scenes.

[0206] In some embodiments, the method is applied to a vehicle, and the method further includes:

[0207] The location information of the detected target is converted into the vehicle coordinate system in order to control the vehicle.

[0208] In some embodiments, the position information of the detected target in the vehicle coordinate system can be obtained based on the coordinates under the isometric projection model and through the first rotation matrix in the extrinsic parameter matrix of the image acquisition device.

[0209] In some embodiments of this specification, the position of the detected target can be transformed from the isometric projection model to the vehicle's actual three-dimensional spatial coordinate system through virtual-real coordinate transformation. This helps to achieve accurate vehicle positioning and tracking, and facilitates autonomous driving.

[0210] In some embodiments, the first image is obtained by taking a fisheye camera.

[0211] In some embodiments, the first image can be any picture taken by a fisheye camera when the vehicle is in motion or stationary. As an example, after the fisheye camera captures the first image, it sends the first image to the terminal device; correspondingly, the device receives the first image from the fisheye camera, thereby acquiring the first image.

[0212] In some embodiments, it can be communicatively connected to an image acquisition device to dynamically acquire a first image. For example, during the vehicle's movement, images of the target detection object can be periodically acquired to obtain a first image, etc.

[0213] The first image can be an image directly acquired by the image acquisition device, or it can be an image acquired by the image acquisition device through methods such as frame extraction.

[0214] In some embodiments, each frame of the first image can be a real-time image, for example, an image captured by an image acquisition device can be acquired in real time (e.g., every 0.02 seconds, 0.01 seconds, etc.). In some embodiments, each frame of the first image can also be an image stored in a storage device.

[0215] In some embodiments, image videos acquired by the image acquisition device can be acquired periodically. The image video is a sequence of multiple first images arranged in chronological order of scanning time. In some embodiments, each first image in the image video can be temporally continuous.

[0216] In some embodiments, the target detection method shown in some embodiments of this specification can be performed sequentially on each frame of the first image in the image video in chronological order.

[0217] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0218] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0219] Figure 9 This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification. For example... Figure 9 As shown, the electronic device 900 may include a processor 901 and a memory 902. The electronic device 900 may also include one or more of a multimedia component 903, an input / output (I / O) component 904, and a communication component 905. In this embodiment, the electronic device 900 may be a device for implementing the target detection method provided in this embodiment.

[0220] The processor 901 controls the overall operation of the electronic device 900 to complete all or part of the steps in the target detection method described above. The memory 902 stores various types of data to support the operation of the electronic device 900. This data may include, for example, instructions for any application or method operating on the electronic device 900, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 902 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 903 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 902 or transmitted via communication component 905. The audio component also includes at least one speaker for outputting audio signals. I / O component 904 provides an interface between processor 901 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 905 is used for wired or wireless communication between the electronic device 900 and other devices. Wireless communication includes, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, Narrow Band Internet of Things (NB-IoT), Enhanced Machine Type Communication (eMTC), or other 5G technologies, or combinations thereof, without limitation. Therefore, the corresponding communication component 905 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0221] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the target detection method described above.

[0222] In another exemplary embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the program instructions are executed by a processor, the program implements the steps of the target detection method described above. For example, the computer-readable storage medium may be the memory 902 containing the program instructions described above. The program instructions may be executed by the processor 901 of the electronic device 900 to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application.

[0223] Alternatively, when the instructions are executed by a computer, they may be used to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application.

[0224] In another exemplary embodiment, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the steps of the target detection method described above. For example, the computer program product may be the memory 902 including the computer program, which may be executed by the processor 901 of the electronic device 900 to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application.

[0225] Alternatively, when the instructions are executed by a computer, they may be used to implement or execute the methods, steps, and logic diagrams disclosed in the embodiments of this application.

[0226] Figure 10 This is an exemplary schematic diagram of a vehicle according to some embodiments of this specification.

[0227] like Figure 10 As shown, this application also provides a vehicle equipped with the electronic equipment provided in any of the above embodiments. The electronic equipment is used to execute the target detection method provided in any of the above embodiments. The vehicle may be a gasoline-powered vehicle, a plug-in hybrid electric vehicle, or a new energy vehicle, etc., and this specification does not specifically limit it.

[0228] In one embodiment, the vehicle can be configured for fully or partially autonomous driving. For example, the vehicle can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human intervention, determine the possible behaviors of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of that other vehicle performing a possible behavior, and control the vehicle based on the determined information. When the vehicle is in autonomous driving mode, it can be configured to operate without human interaction.

[0229] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0230] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0231] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although the descriptions of each embodiment in this application have different focuses, and the parts not described in detail in a certain embodiment can be referred to the relevant embodiments of other embodiments, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of this application without departing from the content of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A target detection method characterized by, The method comprises: transforming the first image into a second image to map and convert a target detection result of the second image, to obtain position information of a detection target under a projection model of the first image; wherein the first image is an ultra-wide-angle image, and the second image is a perspective image.

2. The method of claim 1, wherein, The transformation of the first image into the second image comprises: correcting the first image based on a shooting angle corresponding to the first image to obtain a corrected image; obtaining the second image based on the corrected image.

3. The method of claim 2, wherein, The shooting angle is determined based on at least one of a pitch angle, a roll angle, and a yaw angle of an image acquisition device corresponding to the first image.

4. The method of claim 2, wherein, The correction of the first image based on the shooting angle corresponding to the first image to obtain the corrected image comprises: determining a transformation matrix based on the shooting angle corresponding to the first image; correcting the first image based on the transformation matrix to obtain the corrected image.

5. The method of claim 4, wherein, The transformation matrix represents a transformation relationship between the image acquisition device and a virtual camera.

6. The method of claim 4, wherein, The determination of the transformation matrix based on the shooting angle corresponding to the first image comprises: adjusting a target component corresponding to the shooting angle in a first rotation matrix of the image acquisition device based on the shooting angle corresponding to the first image to obtain an adjusted second rotation matrix corresponding to the virtual camera; obtaining the transformation matrix based on the first rotation matrix and the second rotation matrix.

7. The method of claim 4, wherein, The correction of the first image based on the transformation matrix to obtain the corrected image comprises: performing parameter correction on the first image based on distortion parameters of the image acquisition device corresponding to the first image to obtain a parameter-corrected image; performing pose correction on the parameter-corrected image based on the transformation matrix to obtain a pose-corrected image as the corrected image.

8. The method of claim 7, wherein, The parameter correction on the first image based on the distortion parameters of the image acquisition device corresponding to the first image to obtain the parameter-corrected image comprises: projecting the first image into a camera coordinate system of the image acquisition device based on the distortion parameters to obtain the parameter-corrected image.

9. The method of claim 7, wherein, The angle correction on the parameter-corrected image based on the transformation matrix to obtain the pose-corrected image comprises: projecting the parameter-corrected image from a camera coordinate system of the image acquisition device into a camera coordinate system of the virtual camera based on the transformation matrix to obtain the pose-corrected image.

10. The method of claim 2, wherein, The obtaining of the second image based on the corrected image comprises: cylindrical expansion of the corrected image to obtain the second image.

11. The method of claim 1, wherein, The target detection result of the second image is obtained by: processing the second image based on a target detection model to obtain the target detection result of the second image.

12. The method of claim 1, wherein, The target detection result is a detection result under a perspective model, and the mapping and conversion of the target detection result of the second image to obtain the position information of the detection target under the projection model of the first image comprises: Map the target detection result from the perspective model to the projection model to obtain position information of the detection target in the projection model of the first image.

13. The method of claim 1, wherein, The first image is transformed into the second image, comprising: Based on the first image and a preset mapping relationship, an associated image associated with the first image is obtained to determine the second image, Wherein, the preset mapping relationship is used to indicate the respective corresponding associated images of a plurality of preset images.

14. The method according to any one of claims 1 to 13, characterized in that, The method is applied to a vehicle, and the method further comprises: Converting the position information of the detection target to a vehicle coordinate system to control the vehicle.

15. The method according to any one of claims 1 to 13, characterized in that, The first image is obtained by an fisheye camera.

16. An electronic device, comprising: Comprise: A memory having a computer program stored thereon; A processor for executing the computer program in the memory to implement the steps of the method of any one of claims 1 to 15.

17. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 15.

18. A computer program product, characterised in that, Comprise computer programs or instructions, which are executed by the processor to implement the steps of the method of any one of claims 1 to 15.

19. A vehicle characterized by comprising: Comprise the electronic device of claim 16, or for executing the steps of the method of any one of claims 1 to 15.

Images