Target spot area positioning method and device and electronic equipment

Abstract

The invention discloses a target spot area positioning method and device and electronic equipment, and relates to the technical field of image vision and vehicles. The method comprises the steps that a feature point set, vehicle poses and camera parameters corresponding to a calibration target in a calibration scene of the vehicle surround view system are acquired, and the feature point set is used for representing spatial geometric features of a target spot area on the calibration target; tolerance space sampling is carried out based on the vehicle poses, and multiple sets of candidate poses are obtained; projecting and mapping the feature point set to a camera image plane according to camera parameters and the multiple groups of candidate poses to obtain a projection point set corresponding to the feature point set; and carrying out boundary positioning calculation on the projection point set to obtain region boundary data corresponding to the target region. According to the invention, the technical problem of poor dynamic adaptability and environment robustness of a target spot area positioning method in the prior art is solved.

Description

Technical Field

[0001] This application relates to the fields of image vision technology and vehicle technology, and more specifically, to a target area localization method, device and electronic device. Background Technology

[0002] In the process of calibrating a vehicle surround view system, the target area serves as an important reference for visual calibration and is crucial for the accuracy and reliability of the calibration system.

[0003] Related technologies typically rely on two methods to calibrate vehicle surround-view systems. Method 1: This method relies on manually marking target areas of fixed size and location in the image. However, this method cannot adapt to dynamic changes in vehicle pose, resulting in poor dynamic adaptability and low efficiency in target area localization. Method 2: This method uses automatic localization based on 2D image processing. However, this method is sensitive to lighting and occlusion, leading to poor environmental robustness in target area localization.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a target region localization method, apparatus, and electronic device to at least solve the technical problem of poor dynamic adaptability and environmental robustness of target region localization methods in related technologies.

[0006] According to one aspect of the embodiments of this application, a target region localization method is provided, comprising: acquiring a feature point set, vehicle pose, and camera parameters corresponding to a calibration target in a vehicle surround view system calibration scene, wherein the feature point set is used to characterize the spatial geometric features of the target region on the calibration target; performing tolerance space sampling based on the vehicle pose to obtain multiple sets of candidate poses; projecting and mapping the feature point set onto the camera image plane according to the camera parameters and the multiple sets of candidate poses to obtain a projection point set corresponding to the feature point set; and performing boundary localization calculation on the projection point set to obtain the region boundary data corresponding to the target region.

[0007] Optionally, obtaining the feature point set corresponding to the calibration target in the vehicle surround view system calibration scenario includes: obtaining the spatial coordinates of the target point corresponding to the target point region on the calibration target; constructing a feature point set based on the spatial coordinates and the region template, wherein the region template takes the target point as the center point, and the feature point set includes the target point and multiple neighboring points.

[0008] Optionally, the tolerance space sampling based on the vehicle pose to obtain multiple candidate poses includes: using the vehicle pose and a preset tolerance range to perform spatial interpolation sampling to obtain multiple candidate poses, wherein the vehicle pose is used to characterize the vehicle's calibration pose relative to the calibration target, and the multiple candidate poses are characterized by multiple pose parameter pairs, each pose parameter pair including displacement parameters and rotation parameters.

[0009] Optionally, the camera parameters include intrinsic parameters, omnidirectional model parameters, and distortion parameters. Based on the camera parameters and multiple candidate poses, the feature point set is projected onto the camera image plane to obtain the projected point set. This includes: constructing a calibration projection model of the vehicle surround view system based on the intrinsic parameters, omnidirectional model parameters, and distortion parameters; and using the calibration projection model, projecting the feature point set in the calibration target coordinate system onto the image plane corresponding to each candidate pose to obtain the projected point set.

[0010] Optionally, the boundary localization calculation of the projection point set to obtain the regional boundary data corresponding to the target area includes: filtering and optimizing the projection point set to obtain an optimized point set; calculating the convex hull boundary of the optimized point set to obtain the coordinates of the convex hull vertices, wherein the coordinates of the convex hull vertices are used to locate the regional convex hull boundary corresponding to the target area; and validating the boundary validity of the convex hull vertices to generate regional boundary data.

[0011] Optionally, the projection point set is filtered and optimized to obtain an optimized point set, including: filtering the projection point set based on the corner point distribution of the regional template in the target area to obtain a corner point set; and removing outliers from the corner point set based on the median of the projection points corresponding to the projection point set and a preset outlier threshold to obtain an optimized point set.

[0012] Optionally, performing convex hull boundary calculation on the optimized point set to obtain the convex hull vertex coordinates includes: normalizing the coordinates of the optimized point set to eliminate the coordinate difference between the image height dimension and the image width dimension to obtain the processed point set; performing triangulation calculation on the processed point set to determine the convex hull input points; and using the convex hull construction algorithm and the convex hull input points to perform convex hull boundary calculation to obtain the convex hull vertex coordinates.

[0013] Optionally, performing boundary validity verification on the convex hull vertex coordinates to generate region boundary data includes: performing boundary validity verification on the convex hull vertex coordinates using image boundary constraints and area threshold constraints to obtain verification results, wherein the image boundary constraints are constructed based on the pixel size of the camera image, and the area threshold constraints are constructed based on the area threshold of the target region; in response to the verification results satisfying the target conditions, generating region boundary data based on the convex hull vertex coordinates.

[0014] According to another aspect of the embodiments of this application, a target region localization device is also provided, comprising: an acquisition module, configured to acquire a feature point set, vehicle pose, and camera parameters corresponding to a calibration target in a vehicle surround view system calibration scene, wherein the feature point set is used to characterize the spatial geometric features of the target region on the calibration target; a sampling module, configured to perform tolerance space sampling based on the vehicle pose to obtain multiple sets of candidate poses; a projection module, configured to project and map the feature point set onto the camera image plane according to the camera parameters and multiple sets of candidate poses to obtain a projection point set corresponding to the feature point set; and a localization module, configured to perform boundary localization calculation on the projection point set to obtain region boundary data corresponding to the target region.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the target region localization method of any of the above-mentioned methods when it runs.

[0016] In this embodiment, by acquiring the feature point set, vehicle pose, and camera parameters corresponding to the calibration target in the vehicle surround view system calibration scenario, a data foundation is provided for subsequent calibration of the vehicle surround view system. Furthermore, tolerance space sampling is performed based on the vehicle pose to obtain multiple candidate poses, which helps to more comprehensively evaluate the impact of vehicle pose changes on surround view system calibration. Further, by combining the feature point set, camera parameters, and multiple candidate poses for projection mapping calculation, the feature point set can be transformed into projection point data on the camera image plane corresponding to each of the multiple candidate poses, resulting in a more comprehensive projection point set. Furthermore, boundary localization calculation is performed on the projection point set, which improves the coverage completeness of the regional boundary data corresponding to the target area, enhances the dynamic adaptability and environmental robustness of the target area localization method, and thus allows for the acquisition of more reliable calibration parameters using this regional boundary data. In addition, all the above steps in this application are completed automatically, without relying on manual annotation of the target area, thereby improving the efficiency of the target area localization method. Therefore, this application achieves the goal of efficiently and accurately locating the target area, thereby improving the dynamic adaptability and environmental robustness of the target area localization method, and solving the technical problem of poor dynamic adaptability and environmental robustness of the target area localization method in related technologies. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and provide related descriptions of those embodiments to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of an optional computing terminal for implementing a target region localization method according to an embodiment of this application;

[0019] Figure 2 This is a flowchart of a target region localization method according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of an optional target region localization method according to an embodiment of this application;

[0021] Figure 4 This is a structural block diagram of a target area localization device according to an embodiment of this application. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments only include a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] First, the operating environment of the above method embodiments will be described by way of example. Figure 1 This is a hardware structure block diagram of an optional computing terminal for implementing a target region localization method according to an embodiment of this application, such as... Figure 1As shown, the computing terminal 10 (e.g., a computer terminal, a mobile smart terminal, a vehicle terminal, or a cloud computing virtual terminal, etc.) may include: one or more processors 102 (e.g., processors 102a, 102b, ..., 102n), a memory 104 for storing data, and a transmission device 106 for implementing communication functions. The processor 102 may include, but is not limited to, processing components such as microprocessors (MCUs) or field-programmable gate arrays (FPGAs).

[0025] The aforementioned computing terminal 10 may further include: a display device 110, an input / output interface 108, a Universal Serial Bus (USB) port (which can be used as one of the ports of a computer bus, not shown in the figure), a network interface (not shown in the figure), a power supply (not shown in the figure), and a camera (not shown in the figure).

[0026] It should be noted that one or more processors 102 and / or other data processing circuits in the aforementioned computing terminal 10 may be wholly or partially embodied in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be wholly or partially integrated into any other element in the computing terminal 10 (or mobile device).

[0027] The memory 104 can be used to store software programs and modules of application software, such as the program instructions and data storage devices corresponding to the target area localization method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned target area localization method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the vehicle terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the vehicle terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC) and a network interface, which can be connected to other network devices via a base station to communicate with the Internet. The transmission device 106 can use wired and / or wireless network connections for data communication. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] The input / output interface can be connected to the corresponding input / output device of the computing terminal 10 to realize input / output functions. This input / output device may include, but is not limited to, a cursor control device, a keyboard, and a display. The aforementioned input / output device may be built into the computing terminal 10 or be an external device connected to the computing terminal 10.

[0030] Those skilled in the art will understand that Figure 1 The structure of the computing terminal 10 shown is for illustrative purposes only and does not impose strict limitations on the structure of the computing terminal 10 described above. For example, the computing terminal 10 may also include components that are larger than... Figure 1 The more or fewer components shown, or the computing terminal 10 may have the same Figure 1 The components are shown in different categories.

[0031] This application provides a target area localization method. This method can be used to provide target area localization functionality for preset application scenarios. These preset application scenarios may include scenarios such as surround-view system calibration scenarios and camera-radar fusion calibration scenarios.

[0032] For example, in the context of surround view system calibration, during vehicle manufacturing and maintenance, the technical solution of this application embodiment can be used to synchronize and correct multi-angle cameras in the surround view system, thereby improving the dynamic adaptability of the surround view system calibration.

[0033] For example, in a camera and radar fusion calibration scenario, by combining the technical solution of this application embodiment with radar data, the camera can be automatically calibrated to achieve accurate matching of visual and radar data.

[0034] The vehicle target area localization method provided in this application embodiment can be applied to, but is not limited to, the application scenarios listed above. With the continuous evolution of technology, the above method can also be applied to a wider range of scenarios, such as multi-camera joint calibration application scenarios, thereby providing higher performance target area localization function.

[0035] Under the above operating environment, the embodiments of this application provide the following: Figure 2 The target area localization method shown is as follows: Figure 2 This is a flowchart of a target region localization method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following implementation steps.

[0036] Step S201: Obtain the feature point set, vehicle pose, and camera parameters corresponding to the calibration target in the vehicle surround view system calibration scene. The feature point set is used to characterize the spatial geometric features of the target area on the calibration target.

[0037] During vehicle manufacturing, maintenance, or functional upgrades, in order to ensure that the vehicle surround view system (i.e., the onboard camera system) can accurately perceive the surrounding environment, it is necessary to calibrate the vehicle surround view system installed in the vehicle under a specific environment. This specific environment can be regarded as the vehicle surround view system calibration scenario.

[0038] The aforementioned calibration target is a special tool used in the calibration of a vehicle surround view system. This calibration target possesses well-defined spatial characteristics (e.g., regular patterns, clear contrast, and known target dimensions). In practical applications, the form of the calibration target can be selected according to actual needs, and may include, but is not limited to, checkerboard patterns, circular arrays, and random dot patterns. Specifically, in this application, the dimensions of the calibration target can be 2 meters long and 2 meters wide.

[0039] The aforementioned target area can refer to a specific region on the calibration target used for the calibration of the surround-view system. This target area can be adjusted accordingly based on the form of the selected calibration target. For example, when the selected calibration target is a checkerboard pattern, each intersection point (i.e., the corner point of the checkerboard pattern) on the calibration target can be regarded as a feature point, which can be used to calculate the camera's internal parameters. Therefore, the area corresponding to the corner point of the checkerboard pattern can be determined as the aforementioned target area.

[0040] The aforementioned feature point set can refer to a series of points with unique spatial geometric features selected on the calibration target. During the calibration of a vehicle surround-view system, this feature point set can be used to locate and calculate the target area. This feature point set is used to characterize the spatial geometric features of the target area on the calibration target (e.g., the shape, size, position, and orientation of the object in three-dimensional space).

[0041] The vehicle pose described above can be the vehicle's position and orientation information in three-dimensional space. This vehicle pose can be used to characterize the vehicle's current state. It is understandable that the vehicle pose directly affects the viewing angle and positional relationship of the camera relative to the ground or calibration target in the vehicle surround view system (especially in dynamic environments); therefore, the vehicle pose is crucial for the calibration of the vehicle surround view system.

[0042] The vehicle's pose can be acquired using several sensor systems: an inertial measurement unit, a visual positioning system (which uses a camera and feature matching algorithm to calculate the pose by recognizing features of the surrounding environment), and lidar. Specifically, in this application, the vehicle's pose can be obtained using a pose sensor.

[0043] The aforementioned camera parameters can be both internal and external settings of the camera in the vehicle surround view system. These camera parameters characterize the camera's optical properties and its mounting position and orientation. For example, these camera parameters may include, but are not limited to: focal length, image sensor principal point coordinates, lens distortion coefficient, and camera pose relative to the vehicle coordinate system. These camera parameters can be obtained from factory preset manual data of the camera in the vehicle surround view system, or they can be calculated using image processing technology.

[0044] It is easy to understand that in this application, by obtaining the feature point set, vehicle pose and camera parameters corresponding to the calibration target in the vehicle surround view system calibration scene, a data foundation is provided for the subsequent calibration of the vehicle surround view system.

[0045] Step S202: Based on the vehicle pose, perform tolerance space sampling to obtain multiple sets of candidate poses.

[0046] The aforementioned candidate poses can be used to simulate the vehicle's behavior under different positions and attitudes. These candidate poses can include rotation vectors and displacement vectors. The rotation vector represents the angle of rotation of the vehicle around the three coordinate axes (X-axis, Y-axis, and Z-axis), and can be represented by quaternions or Euler angles. The displacement vector describes the vehicle's positional offset in three-dimensional space, and can consist of coordinate changes in the X, Y, and Z axes.

[0047] The candidate pose can be used to characterize the dynamic changes of the vehicle in the surround view system calibration scenario. Based on the vehicle pose, tolerance space sampling (i.e., sampling in this space based on the known vehicle pose and the error tolerance in actual operation) is performed to obtain multiple sets of candidate poses. In the subsequent projection mapping calculation process, the mapping effect of the target area of ​​the feature point set under different pose conditions is checked, which can more comprehensively evaluate the impact of vehicle pose changes on surround view system calibration, and thus obtain more reliable calibration parameters.

[0048] The aforementioned tolerance space refers to a multi-dimensional space constructed based on the known vehicle pose, taking into account the error tolerance (including rotational and displacement error ranges) of pose measurement in actual operation. This error tolerance can be determined through error analysis of the accuracy of the vehicle pose measurement system. During the tolerance space sampling process, random or regular sampling methods can be used. For example, Monte Carlo methods (with adaptive adjustments to the number of Monte Carlo samplings based on actual application requirements) or uniform sampling can be used to generate multiple sets of candidate poses. These candidate poses represent various possible states of the vehicle within the error tolerance. It is understandable that in practical applications, multiple sampling methods can be combined to obtain a more diverse range of candidate poses.

[0049] It is easy to understand that by sampling the tolerance space based on the vehicle pose, multiple candidate poses are obtained, which helps to determine the projection point set corresponding to the feature point set under multiple candidate poses. This allows for a more comprehensive evaluation of the impact of vehicle pose changes on the surround view system calibration, thereby improving the dynamic adaptability and environmental robustness of the target area localization and obtaining more reliable calibration parameters.

[0050] Step S203: Based on the camera parameters and multiple candidate poses, the feature point set is projected onto the camera image plane to obtain the projection point set corresponding to the feature point set.

[0051] The aforementioned camera image plane can refer to the two-dimensional plane of the image captured by the camera sensor in the vehicle surround view system. The aforementioned projection point set can refer to the set of points formed by mapping the feature point set of the calibration target from three-dimensional space onto the camera image plane during the calibration process of the vehicle surround view system. This projection point set directly corresponds to the projection of the feature point set onto the image plane; that is, each feature point is transformed into one or more projection points through projection mapping, thereby establishing a one-to-one or one-to-many relationship between the feature point set and the projection point set. This projection point set can include the pixel coordinate information of the feature points of the calibration target in the image coordinate system, and this pixel coordinate information can include pixel coordinates in the horizontal direction and pixel coordinates in the vertical direction.

[0052] The aforementioned projection mapping calculation methods may include, but are not limited to: mapping calculation methods based on deep learning projection models, mapping calculation methods based on stereo matching algorithms, and mapping calculation methods based on visual synchronous localization. It is understood that in practical applications, these calculation methods can be used individually or in combination to optimize the projection mapping calculation of feature points on the camera image plane of the surround-view system, thereby improving the accuracy of the projection point set.

[0053] The above projection mapping calculation process can be summarized as follows: using the spatial three-dimensional coordinates of the feature point set, combined with the vehicle's current pose information and camera parameters, multiple projection mapping calculations are performed using perspective projection or omnidirectional projection models to obtain the projection point set on the camera image plane. It can be understood that in each projection mapping calculation, a set of candidate poses is used to obtain multiple two-dimensional projection point data corresponding to the feature point set under that set of candidate poses. All two-dimensional projection point data are collected to form the projection point set.

[0054] Furthermore, by projecting the feature point set onto the camera image plane, this embodiment uses the feature point set as an input source during the generation of the projection point set. The projection point set is generated using a projection model under multiple candidate poses, ensuring the geometric correlation and data consistency between the projection point set and the feature point set. The projection point set not only reflects the projection of the feature point set under the ideal pose but also covers all possible projection positions within the pose tolerance range, thereby enhancing the robustness of the data.

[0055] It is easy to understand that in this application, the feature point set is projected onto the camera image plane according to the camera parameters and multiple candidate poses. This can transform the feature point set into projection point data on the camera image plane corresponding to each of the multiple candidate poses, thereby obtaining a more comprehensive projection point set and improving the coverage completeness of the regional boundary data corresponding to the target area.

[0056] Step S204: Perform boundary positioning calculation on the projection point set to obtain the region boundary data corresponding to the target point region.

[0057] The aforementioned area boundary data is used to calculate the calibration parameters of the vehicle surround view system.

[0058] The aforementioned region boundary data defines the extent of the target's projected region on the camera image plane. This region boundary data may include, but is not limited to: a list of boundary point coordinates (which can represent the smallest closed contour of the target region on the image, typically composed of a series of two-dimensional coordinate points), boundary shape data (which can describe the shape of the projected target region, such as a polygon composed of straight lines, curves, or polylines), and boundary validity labels (which can indicate whether the boundary points meet the calibration requirements under specific conditions). The methods for calculating the aforementioned boundary localization may include, but are not limited to: convex hull algorithms, closed contour detection algorithms, deep learning methods, statistical clustering algorithms, and geometric fitting algorithms.

[0059] The aforementioned calibration parameters can be used to describe the geometric relationship between the camera, the vehicle, and the surrounding environment in a vehicle surround view system. After obtaining the boundary data corresponding to the target area, the calibration parameters of the vehicle surround view system can be calculated through the following steps: First, match boundary points with real positions. Match the boundary points in the boundary data obtained from the boundary localization calculation with the real positions in the world coordinate system to form corresponding point pairs. Second, establish model equations. Based on the corresponding point pairs and the known camera model, establish a set of model equations for the camera's intrinsic and extrinsic parameters. Third, optimize the solution. Use the least squares method, Kalman filtering, or other nonlinear optimization methods to solve the set of model equations for the camera's intrinsic and extrinsic parameters to obtain the optimal estimate of the camera's intrinsic and extrinsic parameters. Fourth, verify and adjust the parameters. Verify the calculated calibration parameters, for example, by reprojecting the boundary points to check consistency.

[0060] The aforementioned regional boundary data plays a crucial role in various application scenarios. Specifically, this regional boundary data can be used in scenarios including, but not limited to, the following.

[0061] The first scenario is visual system calibration. Region boundary data can be used as input data to calculate calibration parameters for vehicle surround view monitors (AVM), forward-looking systems, or other multi-sensor fusion systems. By accurately locating the effective region of the calibration target point in the image, the convergence speed of the calibration algorithm and the accuracy of parameter estimation can be significantly improved.

[0062] The second scenario involves perception and detection tasks. In the real-time perception process of autonomous driving or robot vision, region boundary data can provide accurate and adaptive dynamic regions of interest (ROIs) for subsequent vision algorithms (such as target detection, object recognition, and feature matching). This effectively reduces the processing area, eliminates image background interference, and thus improves computational efficiency and algorithm robustness, making it particularly suitable for embedded platforms that are sensitive to computational resources.

[0063] The third scenario involves data association and fusion. In multi-camera systems or sensor fusion applications, regional boundary data provides spatial constraints for the association of observation data from different perspectives and at different times. For example, it can be used to determine the correspondence of the same physical target point in images from different cameras, or as spatial prior information in temporal filtering.

[0064] The fourth scenario is system diagnosis and quality assessment. The quality of the region boundary data (such as area, shape regularity, and relative position to the image boundary) can be directly used to assess whether the calibration scene settings are reasonable, whether the camera status is normal, or the effectiveness of the calibration process itself, thereby enabling the system to self-monitor and diagnose.

[0065] As can be seen from the above, the region boundary data generated through the embodiments of this application is a general type of computer vision data used to accurately define the spatial location of specific objects in an image. Therefore, the application scope of this application extends from specific AVM calibration to all generalized computer vision tasks that require automated, high-precision localization and analysis of specific target regions in images, demonstrating outstanding technical versatility and application flexibility.

[0066] It is easy to understand that in this application, by performing boundary positioning calculations on the projection point set, the regional boundary data corresponding to the target area can be obtained more accurately, which enhances the dynamic adaptability and environmental robustness of the target area positioning, thereby obtaining more reliable calibration parameters.

[0067] In this embodiment, by acquiring the feature point set, vehicle pose, and camera parameters corresponding to the calibration target in the vehicle surround view system calibration scenario, a data foundation is provided for subsequent calibration of the vehicle surround view system. Furthermore, tolerance space sampling is performed based on the vehicle pose to obtain multiple candidate poses, which helps to more comprehensively evaluate the impact of vehicle pose changes on surround view system calibration. Further, by combining the feature point set, camera parameters, and multiple candidate poses for projection mapping calculation, the feature point set can be transformed into projection point data on the camera image plane corresponding to each of the multiple candidate poses, resulting in a more comprehensive projection point set. Furthermore, boundary localization calculation is performed on the projection point set, which improves the coverage completeness of the regional boundary data corresponding to the target area, enhances the dynamic adaptability and environmental robustness of the target area localization method, and thus allows for the acquisition of more reliable calibration parameters using this regional boundary data. In addition, all the above steps in this application are completed automatically, without relying on manual annotation of the target area, thereby improving the efficiency of the target area localization method. Therefore, this application achieves the goal of efficiently and accurately locating the target area, thereby improving the dynamic adaptability and environmental robustness of the target area localization method, and solving the technical problem of poor dynamic adaptability and environmental robustness of the target area localization method in related technologies.

[0068] Optionally, in step S201 above, obtaining the feature point set corresponding to the calibration target in the vehicle surround view system calibration scenario includes the following implementation steps:

[0069] Step S211: Obtain the spatial coordinates of the target point corresponding to the target point area on the calibration target;

[0070] Step S212: Based on spatial coordinates and a region template, construct a feature point set, wherein the region template takes the target point as the center point, and the feature point set includes the target point and multiple neighboring points.

[0071] The spatial coordinates of the target point mentioned above refer to the positional information of a specific target point on the calibration target in three-dimensional space. These spatial coordinates can use the world coordinate system, the calibration target coordinate system, or the vehicle coordinate system as a reference coordinate system, and can be specifically determined according to the actual application. These spatial coordinates can include coordinate values ​​in three dimensions: the X-axis, the Y-axis, and the Z-axis, representing the positional information of the target point in the horizontal, vertical, and depth directions, respectively.

[0072] The spatial coordinates of the target points can be obtained through the following methods: 1. Using measuring instruments (e.g., laser rangefinders) to measure the three-dimensional coordinates of the target points before calibration begins; 2. If the geometric dimensions and layout of the calibration target are known, the spatial coordinates of the target points corresponding to the target point area on the calibration target can be directly read from the design drawing data; 3. Using sensor information collected by multiple sensors such as vehicle positioning systems, inertial sensors, and lidar, sensor fusion processing is performed to calculate the three-dimensional position of the target points in real time.

[0073] The aforementioned region template can refer to a preset structure used to define the target point and its surrounding set of neighboring points. Specifically, in application, to improve the positioning accuracy and stability of the subsequently obtained projection point set, the region template can be optimized by incorporating machine learning algorithms. This region template may include a center point (specifically, the aforementioned target point can be used as this center point) and template geometric parameters (which can be used to describe the maximum distance between neighboring points and the center point).

[0074] In one exemplary application scenario, such as Figure 3 As shown, the target point mentioned above can be a target corner point. The spatial coordinates of the target point corresponding to the target point area on the calibration target are obtained (denoted as...). Using the aforementioned target point as the center point of the region template, the corresponding template geometric dimension parameters are denoted as follows: The template's geometric dimensions can be dynamically adjusted based on the pose estimation variance. Based on this spatial coordinates and the region template, a feature point set (denoted as ) can be constructed. In particular, the feature point set can be constructed using equation (1).

[0075] Equation (1)

[0076] In equation (1), This represents the local coordinate system rotation matrix.

[0077] It should be noted that in equation (1), "in " represents the Cartesian product, then express The range of possible values, i.e. The offset range in the first dimension (x-axis) is The offset range in the second dimension (y-axis) is also... The offset range in the third dimension (z-axis) is 0, that is, There was no change in the third dimension. Specifically, The first dimension (x-axis) can take the value of ,0, In the second dimension (y-axis), it can take the value of ,0, The value in the third dimension (z-axis) is fixed at 0. Thus, the eight neighborhood points corresponding to the center point can be obtained. The center point and the eight neighborhood points corresponding to the center point are used as the feature point set. Thus, it is possible to generate a multi-scale template point set based on the spatial coordinates of the target point corresponding to the target point area on the calibration target.

[0078] It is easy to understand that, through the above steps S211 to S212, in this embodiment of the application, a feature point set is constructed based on spatial coordinates and a region template. This feature point set includes the target point and multiple neighboring points. The multiple neighboring points can provide additional geometric feature information. Even when the target point is occluded or there is insufficient lighting, the geometric feature information of the neighboring points can be used to ensure the stability and accuracy of the target point region positioning, thereby enhancing the environmental robustness of the target point region positioning method.

[0079] Optionally, in step S202 above, sampling the tolerance space based on the vehicle pose to obtain multiple sets of candidate poses includes the following implementation steps:

[0080] Step S221: Spatial interpolation sampling is performed using the vehicle pose and a preset tolerance range to obtain multiple candidate poses. The vehicle pose is used to characterize the vehicle's calibration pose relative to the calibration target. The multiple candidate poses are characterized by multiple pose parameter pairs, each of which includes a displacement parameter and a rotation parameter.

[0081] The aforementioned preset tolerance range refers to the allowable deviation range set for vehicle pose estimation during the vehicle surround view system calibration process. This preset tolerance range can be used to simulate changes in the vehicle's position and attitude during actual use. The preset tolerance range may include displacement parameter tolerances (which may include the tolerance ranges for the vehicle in the x-axis direction, the y-axis direction, and the z-axis direction) and rotation parameter tolerances (which may include the tolerance ranges for the vehicle's rotation angles around the x-axis, y-axis, and z-axis, respectively).

[0082] In one exemplary application scenario, it is still as follows Figure 3As shown, tolerance space sampling is performed based on the vehicle pose (i.e., spatial interpolation sampling is performed using the vehicle pose and a preset tolerance range) to obtain multiple sets of candidate poses. Specifically, a preset tolerance range is determined based on the accuracy of the vehicle pose estimation system; using the determined preset tolerance range, a multi-dimensional space containing all pose deviations is constructed (denoted as ). This allows for Monte Carlo sampling within the multidimensional space to obtain multiple candidate poses. Specifically, the preset tolerance range can include displacement parameter tolerances (including the vehicle's tolerance range in the x-axis direction, denoted as...). The tolerance range for the vehicle in the x-axis direction can be taken as 0.3 meters; the tolerance range for the vehicle in the y-axis direction is denoted as... The tolerance range of the vehicle in the x-axis direction can be taken as 0.3 meters) and the rotation parameter tolerance (including the tolerance range of the vehicle's rotation angle around the x-axis, denoted as...) The vehicle's rotation angle tolerance around the x-axis can be set to 5°. The vehicle's pose is acquired in real-time using a pose sensor as its calibration pose relative to the target. During spatial interpolation sampling using the vehicle pose and the preset tolerance range, a Gaussian or uniform distribution is used to ensure sampling is within the displacement parameter tolerance. Multiple sets of displacement vectors (i.e., displacement parameters) are randomly selected, and a spherical linear interpolation algorithm is used to uniformly sample within the rotation parameter tolerance, generating multiple rotation vectors (i.e., rotation parameters). The sampled rotation and translation vectors are combined to form multiple pose parameter pairs (denoted as...). Each pose parameter pair can represent a set of candidate poses, and each set of candidate poses represents a pose state of the vehicle in the calibration scenario. In particular, in some application scenarios, the rotation parameter tolerance can also be decomposed into independent sampling intervals of pitch, yaw, and roll, and quaternion interpolation can be used to generate rotation vectors.

[0083] It is readily understood that, through the aforementioned step S221, in this embodiment of the application, spatial interpolation sampling is performed within the tolerance range of the calibration pose to simulate the position and attitude changes of a vehicle in a real-world application scenario, thereby improving the adaptability and robustness of the target area localization method to the dynamic pose of the vehicle. Specifically, a combination of spherical linear interpolation and Gaussian distribution sampling methods can be used for spatial interpolation sampling. Using spherical linear interpolation to sample rotation parameters ensures the uniformity of rotation sampling, while using Gaussian distribution sampling to sample displacement parameters more realistically simulates the displacement characteristics of the vehicle.

[0084] Optionally, the camera parameters include intrinsic parameters, omnidirectional model parameters, and distortion parameters. In step S203 above, the feature point set is projected onto the camera image plane based on the camera parameters and multiple candidate poses to obtain the projected point set, which includes the following implementation steps:

[0085] Step S231: Based on the internal parameters, omnidirectional model parameters, and distortion parameters, construct the calibration projection model of the vehicle surround view system;

[0086] Step S232: Using the calibration projection model, the feature point set in the calibration target coordinate system is projected onto the image plane corresponding to each candidate pose to obtain the projection point set.

[0087] The aforementioned intrinsic parameters can be a set of parameters describing the camera's optical characteristics and imaging geometry. These intrinsic parameters can be used to characterize the camera's inherent properties. These intrinsic parameters may include, but are not limited to, focal length, pixel size, and principal point. Specifically, these intrinsic parameters can be represented in matrix form. By utilizing these intrinsic parameters to construct a calibration projection model of the vehicle surround-view system, three-dimensional coordinates can be mapped to coordinates corresponding to the image plane through the camera's optical properties, ensuring that the calibration projection model can correctly understand and process the projection transformation from real space to image space.

[0088] The aforementioned omnidirectional model parameters can be used to describe the degree to which light bends with varying distances from the lens center. By constructing a calibration projection model using these omnidirectional model parameters, accurate projection mapping can be ensured even with large field-of-view angles, guaranteeing correct object identification and positioning in complex driving environments. Specifically, the camera in the aforementioned vehicle surround-view system can be a fisheye camera with a field of view (i.e., the maximum angle of the scene that the camera can capture) of 190°.

[0089] The aforementioned distortion parameters can be used to characterize image distortion. These parameters can include radial distortion parameters and tangential distortion parameters. The radial distortion parameter describes the magnification difference between the image's central and edge regions. Positive distortion (barrel distortion) and negative distortion (pincushion distortion) can be corrected by adjusting the radial distortion parameter. By constructing a calibration projection model using these distortion parameters, it is possible to correct image geometric distortion caused by the camera lens, ensuring that the actual position of the projected point set more closely matches the ideal position, thus facilitating more accurate target point location. The aforementioned calibration projection model can refer to a mathematical model used to map a set of feature points in three-dimensional space to a two-dimensional image plane.

[0090] The process of constructing a calibration projection model for a vehicle surround view system can be as follows: First, a standard pinhole camera model is constructed using internal parameters; second, omnidirectional model parameters are introduced to reflect the nonlinear projection characteristics of wide-angle or fisheye lenses, and the pinhole model is corrected at large angles to improve the accuracy of the model; finally, distortion compensation is performed on the model output by combining distortion parameters to reduce the impact of image distortion. Thus, the fusion of the omnidirectional camera model and the polynomial distortion model can be achieved.

[0091] The aforementioned calibration target coordinate system can be a reference coordinate system defined in physical space, which can be used to provide a reference for describing the position of feature points on the calibration target. This calibration target coordinate system can use a corner point or center point of the calibration target as the origin of the coordinate system, and establish the x-axis and y-axis along its inherent geometric features (such as long and short sides), while the z-axis can be perpendicular to the calibration target plane and point towards the observer.

[0092] In one exemplary application scenario, it is still as follows Figure 3 As shown, the calibration projection model of the vehicle surround view system is constructed based on the internal parameters, omnidirectional model parameters and distortion parameters, as shown in Equation (2).

[0093] Equation (2)

[0094] In equation (2), This represents the first component of the pixel coordinates in the image coordinate system corresponding to the image plane. Represents a nonlinear projection function. This represents the intrinsic parameter matrix corresponding to the intrinsic parameters. Represents a set of feature points. This represents the pose parameter pair corresponding to the candidate pose. Represents the parameters of the omnidirectional model. This represents the distortion parameter.

[0095] In the same application scenario, equation (2) can be transformed to obtain equation (3).

[0096] Equation (3)

[0097] In equation (3), This represents the distortion parameter.

[0098] Continuing in the aforementioned application scenario, this calibration projection model is used to project the feature point set in the calibration target coordinate system onto the image plane corresponding to each candidate pose (i.e., multi-view projection), obtaining the projected point set. Specifically, from the multiple sampled candidate poses, each candidate pose is selected as the currently selected candidate pose for projection calculation. Each candidate pose is characterized by a pose parameter pair, each including rotation and displacement parameters. Using the calibration projection model, the feature point set in the calibration target coordinate system is projected onto the image plane corresponding to the currently selected candidate pose. That is, the three-dimensional coordinates corresponding to each feature point in the feature point set are converted into two-dimensional image coordinates on the image plane corresponding to the currently selected candidate pose, obtaining the initial projected point set of the feature point set corresponding to the currently selected candidate pose. These initial projected point sets are then combined to obtain the final projected point set. In some application scenarios, a weighted fusion strategy can also be used to weight and fuse the multi-view projection results.

[0099] In some applications, the projection mapping calculation process also includes a distortion compensation step, which converts distorted image coordinates into distortion-free coordinates (denoted as ). As shown in equation (4).

[0100] Equation (4)

[0101] In equation (4), Indicates the distortion radius. Indicates the first coefficient. This indicates the second coefficient.

[0102] Still in the above application scenario, the above calibration projection model needs to satisfy equation (5), specifically, equation (5) is as follows.

[0103] Equation (5)

[0104] In equation (5), ,in, This represents the z-axis coordinate of a feature point in the feature point set within the calibration target coordinate system.

[0105] It is easy to understand that, through the above steps S231 to S232, in this embodiment of the application, the calibration projection model of the vehicle surround view system is constructed by combining internal parameters, omnidirectional model parameters and distortion parameters, so that the calibration projection model can adapt to various shooting conditions (in particular, it can adapt to the influence of a wide range of viewing angles and distortion effects), thereby improving the accuracy of the projection point set and enhancing the reliability of subsequent target area positioning.

[0106] Optionally, in step S204 above, the boundary localization calculation of the projection point set to obtain the region boundary data corresponding to the target point region includes the following implementation steps:

[0107] Step S241: Filter and optimize the projection point set to obtain an optimized point set;

[0108] Step S242: Calculate the convex hull boundary of the optimized point set to obtain the coordinates of the convex hull vertices. The coordinates of the convex hull vertices are used to locate the convex hull boundary of the region corresponding to the target point region.

[0109] Step S243: Perform boundary validity verification on the convex hull vertex coordinates to generate region boundary data.

[0110] The aforementioned set of optimized points can be obtained using a selection optimization method. This selection optimization method may include, but is not limited to: methods based on a pre-defined selection strategy, methods based on deep learning algorithms (e.g., convolutional neural networks, object detection networks), selection methods based on Hough transform, image segmentation methods, and least squares fitting methods. The process of selection optimization based on deep learning algorithms can be as follows: pre-training the convolutional neural network model to enable it to learn complex image features, thereby identifying and classifying optimized points in the projection point set and improving the accuracy of selecting optimized points.

[0111] It is understandable that each of the screening and optimization methods listed above has its applicable scenarios and advantages and disadvantages. When selecting a screening and optimization method, the characteristics of the projection point set, the limitations of computational resources, and the specific application requirements must be considered. For example, deep learning methods perform excellently in handling complex environments and irregular shapes, but require a large amount of training data and computational resources; while least squares fitting methods are computationally efficient, but may be more sensitive to noise and occlusion. In practical applications, the most suitable method or a combination strategy can be adopted according to the specific situation to achieve better screening and optimization results.

[0112] The convex hull vertex coordinates mentioned above can be obtained through several methods: a combination of Andrew's monotone chain algorithm and triangulation, Graham's scan method, the wrapping algorithm, and divide-and-conquer algorithms (e.g., dividing the optimization point set into left and right parts, recursively calculating the convex hulls of the left and right subsets, and finally merging the two sub-convex hulls to obtain the final convex hull boundary, thus obtaining the convex hull vertex coordinates). These convex hull vertex coordinates can be used to locate the convex hull boundary of the region corresponding to the target point area.

[0113] The above boundary validity verification can be implemented through the following methods: image mask-based verification, bounding box-based verification, and geometric verification combined with machine learning prediction. Specifically, the image mask-based verification process can be as follows: create a binary mask matching the image size, where the inside of the image boundary is marked as 1 and the outside as 0; map the convex hull vertex coordinates onto the binary mask, and check if all vertices are located within the region marked as 1; if all vertices are inside the mask, the boundary is considered valid; otherwise, adjust or recalculate the convex hull. The bounding box-based verification process can be as follows: calculate the minimum bounding box of the convex hull vertex coordinates; check if the bounding box is completely within the image boundary; if the bounding box contains all vertices and is located within the image boundary, the convex hull boundary is considered valid. The geometric verification combined with machine learning prediction process can be as follows: use a machine learning model to predict the convex hull vertex coordinates within the image boundary and area range; combine geometric verification constraints to comprehensively judge the validity of the convex hull boundary, obtaining the geometric verification result; perform a weighted calculation on the prediction result and the geometric verification result to obtain the final verification result.

[0114] Optionally, in step S241 above, the process of filtering and optimizing the projection point set to obtain an optimized point set includes the following steps: using a first filtering strategy to perform corner point filtering on the projection point set to obtain a corner point set, wherein the first filtering strategy is determined based on the region template of the target area; using a second filtering strategy to remove outliers from the corner point set to obtain an optimized point set, wherein the second filtering strategy is determined based on the median of the projection points and the outlier threshold.

[0115] The first screening strategy described above can be used to filter out the corner points corresponding to the target region from the set of projection points. It is understood that, due to differences in the size parameters and geometric features of the region templates, the first screening strategy needs to be determined based on the region template of the target region to ensure that the first screening strategy matches the region template.

[0116] By utilizing the first filtering strategy, corner point filtering can be performed on the projection point set to obtain a corner point set. Non-corner points or irrelevant points in the projection point set can be removed, thus reducing the amount of data in subsequent processing steps while retaining key feature points (i.e., corner points).

[0117] It is understandable that some outliers may exist in the corner point set due to noise, occlusion, or other anomalies, deviating from the normal distribution. The second screening strategy described above can be used to remove these outliers. This second screening strategy is based on the median of the projected points and an outlier threshold. The median of the projected points can be determined as follows: sort the coordinates of the points in the projected point set by value, and determine the median value in the middle of the sorted result. In a two-dimensional image coordinate system, the median values ​​of the projected points in the x and y coordinate dimensions can be calculated separately. The outlier threshold can be set based on statistical principles; specifically, it can be set as a multiple of the median of the projected points.

[0118] Optionally, in step S241 above, the process of filtering and optimizing the projection point set to obtain an optimized point set includes the following steps:

[0119] Step S2411: Based on the distribution of corner points in the regional template of the target area, perform corner point filtering on the projection point set to obtain the corner point set;

[0120] Step S2412: Based on the median of the projected points corresponding to the projected point set and the preset outlier threshold, outlier points are removed from the corner point set to obtain an optimized point set.

[0121] The aforementioned set of projection points may include multiple projection points. The median of these projection points represents the coordinates of the center position corresponding to the multiple projection points, which can be calculated based on the projection coordinates of the multiple projection points. For example, the center position can be calculated based on the coordinates of the multiple projection points along the X and Y axes in the image coordinate system. Specifically, the calculation method includes taking the median of the X-axis coordinates of all projection points as the X-coordinate of the center point and the median of the Y-axis coordinates of all projection points as the Y-coordinate of the center point, thus obtaining the coordinates (X, Y) of the center position, which can then be used as the median.

[0122] The aforementioned preset outlier threshold can be a threshold pre-set for the projected point set. This outlier threshold is used to characterize the allowable deviation of each corner point in the corner point set from the median (i.e., the coordinate value of the center position). In application scenarios, the Euclidean distances from all corner points in the corner point set to the median can be calculated first, and the outlier threshold can be set based on the statistical distribution of these distances (e.g., the standard deviation or median of the distances). When the Euclidean distance from a corner point to the median exceeds the preset outlier threshold, the corner point can be considered an outlier and should be removed from the corner point set.

[0123] It should be noted that the aforementioned outlier threshold, besides being a fixed or experience-based preset value, can also be adaptively set according to the dynamic needs or data characteristics of the actual application scenario in practice. Specifically, the outlier threshold can be a threshold dynamically calculated based on the statistical characteristics of the current or historical projection point set (or corner point set), image quality assessment parameters, or real-time system performance indicators.

[0124] In step S2411, the specific process of corner point filtering is as follows: identifying and extracting projection points corresponding to the predefined corner point positions in the region template. Specifically, the region template contains multiple feature points (e.g., a nine-square grid centered on the target point) during construction. In this region template, the corner points at the four corner positions are predefined as key corner points. After performing projection mapping, the projection point set retains the same index order as the template point set. Therefore, corner point filtering extracts the projection points with index numbers 0, 2, 6, and 8 from the complete projection point set through direct index mapping, and these projection points together constitute the corner point set. The above processing utilizes the inherent geometric structure of the region template, which can efficiently filter non-key points and retain key corner points used to define the boundary of the target area, thereby reducing the amount of data required for subsequent calculations.

[0125] Further, in step S2412, the outlier removal process is as follows: Based on a statistical distribution model, abnormal deviation points in the corner point set are identified and removed. Specifically, firstly, the median coordinates of all points in the corner point set along the X and Y axes of the image coordinate system are calculated, and a robust center estimation based on the median coordinates is performed. Next, the Euclidean distance from each corner point to the median point is calculated. Then, the scale parameter of the Euclidean distance (e.g., the median or standard deviation of all distances) is compared with a preset outlier threshold (e.g., 3 times the standard deviation). Finally, all points whose distance exceeds the preset threshold are identified as outliers and removed from the corner point set, and the remaining projected points constitute the final optimized point set. The above processing effectively overcomes outliers caused by projection errors, instantaneous occlusion, or image noise, ensuring the accuracy and robustness of subsequent convex hull boundary calculations.

[0126] In an exemplary application scenario, during the construction of a feature point set, index values ​​can be set for the target point and multiple neighboring points contained in the feature point set. Following a left-to-right, top-to-bottom order, the index values ​​of the nine feature points in the feature point set are set. Thus, the index values ​​of the four corner points (top left, top right, bottom left, and bottom right) in the feature point set correspond to 0, 2, 6, and 8, respectively. A first filtering strategy is determined based on the region template of the target point area. This first filtering strategy is used for index value matching. When the index value corresponding to a projection point in the projection point set is any one of 0, 2, 6, or 8, the projection point is considered a corner point and is retained. Conversely, when the index value corresponding to a projection point in the projection point set is other than 0, the projection point is considered not a corner point and is removed. This completes the corner point filtering process for the projection point set, resulting in a corner point set.

[0127] Continuing in the aforementioned application scenario, after determining the second filtering strategy based on the median of the projected points and the outlier threshold, the corner point set is traversed. For any corner point, if any dimension of its corresponding coordinates deviates from the median of the projected points by more than the outlier threshold, the corner point is considered an outlier and needs to be removed. After traversing the corner point set, the optimized point set described above is obtained. By eliminating outliers affected by noise and interference, a higher-quality optimized point set can be obtained, reducing the bias in the subsequent generation of region boundary data. Specifically, the standard deviation of the point coordinate values ​​corresponding to the projected point set (denoted as ) can be used to determine the outlier threshold. Multiples of ) are used as outlier thresholds (e.g., When the median value of the projection point corresponding to a corner point in the corner point set is greater than 100%, If the corner point is not an outlier, then it can be treated as an outlier and removed to obtain an optimized point set.

[0128] Understandably, in practical applications, other methods can also be used to determine the second screening strategy to remove outliers from the corner point set. For example, the Mahalanobis distance corresponding to the corner point set can be calculated, and the first distribution (e.g., The distribution threshold is used as the outlier threshold.

[0129] It is easy to understand that, through the above steps S2411 to S2412, in this embodiment of the application, the first screening strategy is used to screen the corner points of the projection point set, which can remove non-corner points or irrelevant points in the projection point set to obtain a corner point set. While retaining key feature points (i.e., corner points), the amount of data in subsequent processing steps is reduced. Furthermore, the second screening strategy is used to remove outliers from the corner point set, removing abnormal points in the corner point set to obtain a better quality optimized point set, providing a better data foundation for subsequent convex hull boundary calculation.

[0130] Optionally, in step S242 above, calculating the convex hull boundary of the optimized point set to obtain the coordinates of the convex hull vertices includes the following implementation steps:

[0131] Step S2421: Perform coordinate normalization on the optimized point set to eliminate the coordinate difference between the image height dimension and the image width dimension, and obtain the processed point set;

[0132] Step S2422: Perform triangulation calculation on the processing point set to determine the convex hull input point;

[0133] Step S2423: Calculate the convex hull boundary using the convex hull construction algorithm and the convex hull input points to obtain the coordinates of the convex hull vertices.

[0134] The aforementioned coordinate differences refer to discrepancies caused by inconsistencies in the ratio or scale between the image height and width dimensions. When coordinate differences exist between the image height and width dimensions, it leads to geometric distortion in different directions within the image, affecting the accuracy of subsequent processing steps. By performing coordinate normalization on the optimized point set, the coordinate differences between the image height and width dimensions can be eliminated, ensuring consistency in their ratio or scale. The aforementioned processing point set can be obtained using normalization methods (e.g., Z-score normalization).

[0135] The specific process of the coordinate normalization process described above can be as follows: Read the image metadata (including focal length, image sensor size, etc.) as the basis for coordinate normalization; use the affine transformation matrix and image metadata to adjust and optimize the coordinate values ​​corresponding to each point in the point set, so that the ratio of the image height dimension coordinates and the image width dimension coordinates tends to be consistent, so as to eliminate the coordinate difference between the image height dimension and the image width dimension; further, scale the image height dimension coordinates and the image width dimension coordinates according to the actual pixel ratio of the image to obtain the processed point set.

[0136] The aforementioned convex hull input points can refer to a set of points used to construct the convex hull boundary. Specifically, the vertices of the triangles formed by triangulation can be selected as the aforementioned convex hull input points. The aforementioned convex hull vertex coordinates can refer to the coordinate positions of the vertices of the smallest convex polygon enclosing the projection point set in the image coordinate system.

[0137] The aforementioned convex hull construction algorithm can refer to an algorithm used to calculate the minimum convex hull formed by a set of points in space. This convex hull construction algorithm may include, but is not limited to: Graham's scan method, Andrew's monotone chain algorithm (specifically, it constructs a monotone chain in two steps, scanning the point set upwards and downwards to generate the convex hull boundary), and wrapping algorithm (specifically, it starts from an external point and gradually selects the farthest point to expand the convex hull boundary).

[0138] In one exemplary application scenario, it is still as follows Figure 3 As shown, the specific process of convex hull boundary calculation can be as follows: The Z-score normalization method is used to normalize the coordinates of the optimized point set, normalizing all point coordinates in the optimized point set to between 0 and 1, thus obtaining the processing point set; further, a triangulation method (e.g., Delaunay triangulation method) is used to perform triangulation calculation on the coordinate-normalized processing point set, converting the processing point set into a set of interconnected triangles, and using the vertices of the formed triangles as the convex hull input points for convex hull boundary calculation; further, a base point (e.g., the point with the smallest or largest x-coordinate among the convex hull input points) is determined as the starting point in the convex hull input points, and the convex hull construction algorithm is used to traverse the convex hull input points, selecting points that can form the current outermost boundary each time, gradually constructing the upper and lower monotonic chains of the convex hull, merging the two monotonic chains, removing duplicate vertices, forming a closed convex hull boundary, and recording the vertex coordinates that form the convex hull boundary (i.e., the convex hull vertex coordinates).

[0139] In some application scenarios, after using the convex hull construction algorithm and the convex hull input point to calculate the convex hull boundary and obtain the coordinates of the convex hull vertex, the coordinates of the convex hull vertex can also be integerized, as shown in Equation (6).

[0140] Equation (6)

[0141] In equation (6), This represents the coordinates of the convex hull vertices before integerization. Represents the coordinates of the convex hull vertices after integerization.

[0142] Still in the above application scenario, by integerizing the coordinates of the convex hull vertices (i.e., anti-aliasing), sub-pixel level correction of the convex hull vertex coordinates can be achieved. The correction amount (denoted as...) As shown in equation (7).

[0143] Equation (7)

[0144] In equation (7), This represents the coordinates of the convex hull vertices before correction. This indicates taking the coordinates of the convex hull vertices before correction. The decimal part.

[0145] It is readily understood that, through the aforementioned steps S2421 to S2423, in this embodiment of the application, by performing coordinate normalization processing on the optimized point set, the coordinate difference between the image height dimension and the image width dimension can be eliminated, ensuring that the coordinates of all points can be compared and analyzed at a uniform scale during subsequent convex hull boundary calculation; furthermore, triangulation calculation is performed on the processed point set to determine the convex hull input points, providing structured data points for convex hull boundary calculation; furthermore, by using the convex hull construction algorithm and the convex hull input points to perform convex hull boundary calculation, more accurate convex hull vertex coordinates can be obtained.

[0146] Optionally, in step S243 above, the boundary validity verification of the convex hull vertex coordinates and the generation of region boundary data include the following implementation steps:

[0147] Step S2431: The boundary validity of the convex hull vertex coordinates is verified using image boundary constraints and area threshold constraints to obtain the verification result. The image boundary constraints are constructed based on the pixel size of the camera image, and the area threshold constraints are constructed based on the area threshold of the target region.

[0148] Step S2432: In response to the verification result satisfying the target condition, generate region boundary data based on the coordinates of the convex hull vertex.

[0149] The image boundary constraint described above can be used to limit the effective image range. This image boundary constraint is constructed based on the pixel size of the camera image. In this image boundary constraint, the effective range of the convex hull vertex coordinates on the x-axis and y-axis can be defined. The effective image range can be determined based on the effective range of the convex hull vertex coordinates on the x-axis and y-axis.

[0150] In one application scenario, image boundary constraints can be obtained by reading the width and height parameters of the image from camera parameters or image metadata; further, establishing coordinate constraints based on the width and height parameters to construct image boundary constraints.

[0151] The area threshold constraint described above can be used to determine whether the area of ​​the region enclosed by the coordinates of the convex hull vertices is within a preset reasonable range. A minimum area threshold and a maximum area threshold can be defined within this area threshold constraint.

[0152] In one application scenario, the area threshold constraint can be obtained in the following way: based on the previous calibration and experimental data, the average area and standard deviation of the target projection area are statistically analyzed, and the minimum area threshold and maximum area threshold are set to construct the area threshold constraint.

[0153] The above verification results can be used to determine whether the convex hull vertex coordinates satisfy the image boundary constraints and area threshold constraints. Specifically, the target conditions can be set as follows: if the verification result indicates that the convex hull vertex coordinates satisfy both the image boundary constraints and the area threshold constraints, then when the system determines that the verification result indicates that the convex hull vertex coordinates satisfy both the image boundary constraints and the area threshold constraints, the verification result is considered to satisfy the target conditions. At this time, region boundary data is generated based on the convex hull vertex coordinates.

[0154] In one exemplary application scenario, it is still as follows Figure 3 As shown, the above image boundary constraints can be: ,in, Represents the coordinates of the convex hull vertices. This represents the region enclosed by the coordinates of the convex hull vertices. This represents the coordinates of the convex hull vertex on the x-axis. The parameter representing the width of the image. This represents the coordinates of the convex hull vertex on the y-axis. This represents the height parameter of the image. The area threshold constraint mentioned above can be: ,in, This represents the area of ​​the region enclosed by the coordinates of the convex hull vertices. This represents the minimum area threshold. Indicates the threshold coefficient. This represents the maximum area threshold. The specific process of boundary validity verification can be as follows: Implement a dual verification mechanism, using image boundary constraints and area threshold constraints to verify the boundary validity of the convex hull vertex coordinates, and obtain the verification result; when the verification result meets the target condition, generate region boundary data based on the convex hull vertex coordinates. It can be understood that when the verification result indicates that the convex hull vertex coordinates meet both the image boundary constraints and the area threshold constraints, the verification result can be considered to meet the target condition. Based on the verified convex hull vertex coordinates, generate polygon boundary data (i.e., region boundary data) corresponding to the target region. In particular, this region boundary data can be a sequence of polygon vertices in the target region.

[0155] It is easy to understand that, through the above steps S2431 to S2432, in this embodiment of the application, by utilizing image boundary constraints and area threshold constraints, the coordinates of the convex hull vertex are double-checked, which can alleviate the problem of inaccurate ROI boundaries caused by calculation errors or abnormal situations, and improve the accuracy of target area localization.

[0156] In one exemplary application scenario, this application also provides a pose space sampling density and target region coverage (denoted as...). The quantitative relationship model of ) is expressed as follows: Therefore, the target area coverage of the technical solution of this application can be calculated to reach 98.5%, which is a significant improvement compared to the 82% target area coverage of the traditional method. Furthermore, by utilizing the technical solution of this application, real-time processing of high-resolution (e.g., 4K resolution) images can be supported with a latency of less than 100ms / frame. Specifically, the processing latency of the technical solution of this application is 65ms / frame, which is a significant improvement compared to the 120ms / frame processing latency of the traditional method. The robustness of the technical solution of this application under extreme lighting conditions can reach 89%, which is a significant improvement compared to the 56% robustness of the traditional method under extreme lighting conditions.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0158] It should be noted that, for the sake of simplicity, the technical solutions in the above method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the order of actions in the described action combination, because according to this application, some of the above steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this application specification are preferred embodiments, and the actions and modules involved are not necessarily essential for implementing the technical solutions of this application.

[0159] According to an embodiment of this application, a device embodiment for a target region localization device is also provided. This target region localization device is used to implement the above-described method embodiment and various optional implementations of the method embodiment. The technical content already described above will not be repeated in the device embodiment. It should be noted that, in the following related descriptions of the device embodiment, "module" can be software, hardware, or a combination of software and hardware used to implement a specified function.

[0160] Please refer to Figure 4The aforementioned target region localization device includes an acquisition module 401, used to acquire the feature point set, vehicle pose, and camera parameters corresponding to the calibration target in the vehicle surround view system calibration scene, wherein the feature point set is used to characterize the spatial geometric features of the target region on the calibration target; a sampling module 402, used to perform tolerance space sampling based on the vehicle pose to obtain multiple sets of candidate poses; a projection module 403, used to project and map the feature point set onto the camera image plane according to the camera parameters and multiple sets of candidate poses to obtain the projection point set corresponding to the feature point set; and a localization module 404, used to perform boundary localization calculation on the projection point set to obtain the region boundary data corresponding to the target region.

[0161] Optionally, the acquisition module 401 is further configured to: acquire the spatial coordinates of the target point corresponding to the target point region on the calibration target; and construct a feature point set based on the spatial coordinates and the region template, wherein the region template takes the target point as the center point, and the feature point set includes the target point and multiple neighboring points.

[0162] Optionally, the sampling module 402 is further configured to: perform spatial interpolation sampling using the vehicle pose and a preset tolerance range to obtain multiple candidate poses, wherein the vehicle pose is used to characterize the vehicle's calibration pose relative to the calibration target, and the multiple candidate poses are characterized by multiple pose parameter pairs, each pose parameter pair including displacement parameters and rotation parameters.

[0163] Optionally, the camera parameters include intrinsic parameters, omnidirectional model parameters, and distortion parameters. The projection module 403 is also used to: construct a calibration projection model of the vehicle surround view system based on the intrinsic parameters, omnidirectional model parameters, and distortion parameters; and use the calibration projection model to project the feature point set in the calibration target coordinate system onto the image plane corresponding to each candidate pose to obtain the projection point set.

[0164] Optionally, the positioning module 404 is further configured to: perform filtering and optimization processing on the projection point set to obtain an optimized point set; perform convex hull boundary calculation on the optimized point set to obtain the coordinates of the convex hull vertices, wherein the coordinates of the convex hull vertices are used to locate the convex hull boundary of the region corresponding to the target point region; and perform boundary validity verification on the convex hull vertices to generate region boundary data.

[0165] Optionally, the positioning module 404 is further configured to: perform corner point filtering on the projection point set based on the corner point distribution of the regional template of the target area to obtain a corner point set; and remove outliers from the corner point set based on the median of the projection points corresponding to the projection point set and a preset outlier threshold to obtain an optimized point set.

[0166] Optionally, the aforementioned positioning module 404 is further configured to: perform coordinate normalization processing on the optimized point set to eliminate the coordinate difference between the image height dimension and the image width dimension, thereby obtaining a processed point set; perform triangulation calculation on the processed point set to determine the convex hull input point; and use the convex hull construction algorithm and the convex hull input point to perform convex hull boundary calculation to obtain the convex hull vertex coordinates.

[0167] Optionally, the positioning module 404 is further configured to: perform boundary validity verification on the coordinates of the convex hull using image boundary constraints and area threshold constraints, and obtain verification results, wherein the image boundary constraints are constructed based on the pixel size of the camera image, and the area threshold constraints are constructed based on the area threshold of the target region; in response to the verification results satisfying the target conditions, generate region boundary data based on the coordinates of the convex hull vertex.

[0168] It should be noted that the above-mentioned acquisition module 401, sampling module 402, projection module 403 and positioning module 404 correspond to steps S201 to S204 in the method embodiment. The four modules are the same as the corresponding steps in terms of implementation instances and application scenarios, but are not limited to the content disclosed in the above method embodiment.

[0169] It should be noted that the modules mentioned in the above device embodiments can be implemented by software, hardware, or a combination of both. For example, when the modules are implemented by hardware, they can be placed in the same processor, or they can be placed in different processors in any combination. As another example, the modules can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). These modules can also operate as part of the device in environments such as... Figure 1 In the computing terminal 10 shown.

[0170] According to an embodiment of this application, an embodiment of an electronic device is also provided. The electronic device includes a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to implement the above-described target area localization method.

[0171] According to an embodiment of this application, a computer-readable storage medium is also provided, which includes a stored program, wherein the program controls the device where the computer-readable storage medium is located to implement the above-described target area localization method when it is running.

[0172] Optionally, the aforementioned computer storage media may include, but are not limited to: hard disk drives (HDDs), solid state drives (SSDs), USB flash drives, optical discs, memory cards, cloud storage media, and network-attached storage (NAS) devices.

[0173] Optionally, the aforementioned computer-readable storage medium may be configured to store a computer program for performing the following steps: acquiring a feature point set, vehicle pose, and camera parameters corresponding to a calibration target in a vehicle surround view system calibration scene, wherein the feature point set is used to characterize the spatial geometric features of the target area on the calibration target; performing tolerance space sampling based on the vehicle pose to obtain multiple sets of candidate poses; projecting the feature point set onto the camera image plane according to the camera parameters and multiple sets of candidate poses to obtain a projection point set corresponding to the feature point set; and performing boundary localization calculation on the projection point set to obtain the region boundary data corresponding to the target area.

[0174] According to an embodiment of this application, a computer program product is also provided. This computer program product includes a computer program that, when executed by a processor, can implement the aforementioned target region localization method.

[0175] Optionally, the aforementioned computer program product can provide target area localization services based on the aforementioned target area localization method.

[0176] Optionally, in this embodiment, the computer program product can be a set of instructions and code pre-written according to the target area localization method described above. This computer program product can run on various computer platforms, including personal computers, servers, mobile devices, etc.

[0177] Optionally, in this embodiment, the instructions and code corresponding to the computer program product are used to implement the following method steps: obtaining the feature point set, vehicle pose, and camera parameters corresponding to the calibration target in the vehicle surround view system calibration scene, wherein the feature point set is used to characterize the spatial geometric features of the target area on the calibration target; performing tolerance space sampling based on the vehicle pose to obtain multiple sets of candidate poses; projecting and mapping the feature point set onto the camera image plane according to the camera parameters and multiple sets of candidate poses to obtain the projection point set corresponding to the feature point set; performing boundary localization calculation on the projection point set to obtain the area boundary data corresponding to the target area.

[0178] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0179] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of multiple modules can be a logical functional division, and in actual implementation in application scenarios, there can be any other possible division methods. Furthermore, multiple modules (or units or components within modules) can be combined with each other and integrated into another system. For example, some features in the method embodiments described above can be ignored or skipped during execution.

[0180] It should be noted that in the above embodiments, the modules, components, or units described as separate parts can be physically separated or physically integrated. The components shown as modules or units can be physical modules or units, or virtual modules or units. That is, multiple modules or units can be located in the same position or distributed across multiple positions or spaces. In application scenarios, depending on the actual needs of the scenario, some or all of the multiple modules or units can be selected to implement the technical solutions of the embodiments of this application, thereby achieving the corresponding technical objectives.

[0181] Specifically, for integrated functional modules or units, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0182] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for target region localization, characterized in that, include: The feature point set, vehicle pose, and camera parameters corresponding to the calibration target in the vehicle surround view system calibration scene are obtained, wherein the feature point set is used to characterize the spatial geometric features of the target point region on the calibration target. Based on the vehicle pose, tolerance space sampling is performed to obtain multiple sets of candidate poses; Based on the camera parameters and the multiple candidate poses, the feature point set is projected onto the camera image plane to obtain the projection point set corresponding to the feature point set; Boundary localization calculations are performed on the projection point set to obtain the region boundary data corresponding to the target point region.

2. The target region localization method according to claim 1, characterized in that, Obtaining the feature point set corresponding to the calibration target in the vehicle surround view system calibration scene includes: Obtain the spatial coordinates of the target point corresponding to the target point region on the calibration target; Based on the spatial coordinates and the region template, the feature point set is constructed, wherein the region template takes the target point as the center point, and the feature point set includes the target point and multiple neighboring points.

3. The target region localization method according to claim 1, characterized in that, Based on the vehicle pose, tolerance space sampling is performed to obtain the multiple sets of candidate poses, including: Spatial interpolation sampling is performed using the vehicle pose and a preset tolerance range to obtain the multiple sets of candidate poses. The vehicle pose is used to characterize the vehicle's calibration pose relative to the calibration target. The multiple sets of candidate poses are characterized by multiple pose parameter pairs, each of which includes a displacement parameter and a rotation parameter.

4. The target region localization method according to claim 1, characterized in that, The camera parameters include intrinsic parameters, omnidirectional model parameters, and distortion parameters. Based on the camera parameters and the multiple sets of candidate poses, the feature point set is projected onto the camera image plane to obtain the projected point set, which includes: Based on the internal parameters, the omnidirectional model parameters, and the distortion parameters, a calibration projection model of the vehicle surround view system is constructed. Using the calibration projection model, the feature point set in the calibration target coordinate system is projected onto the image plane corresponding to each candidate pose to obtain the projection point set.

5. The target region localization method according to claim 1, characterized in that, Boundary localization calculations are performed on the projection point set to obtain the region boundary data corresponding to the target point region, including: The projection point set is filtered and optimized to obtain an optimized point set; The convex hull boundary is calculated on the optimized point set to obtain the coordinates of the convex hull vertex, wherein the coordinates of the convex hull vertex are used to locate the convex hull boundary of the region corresponding to the target point region; The boundary validity of the convex hull vertex coordinates is validated to generate the region boundary data.

6. The target region localization method according to claim 5, characterized in that, The optimized point set is obtained by filtering and optimizing the projected point set, which includes: Based on the corner point distribution of the regional template in the target area, the corner point set is filtered to obtain a corner point set. Based on the median of the projected points corresponding to the projected point set and the preset outlier threshold, outlier points are removed from the corner point set to obtain the optimized point set.

7. The target region localization method according to claim 5, characterized in that, Perform convex hull boundary calculation on the optimized point set to obtain the coordinates of the convex hull vertices, including: The optimized point set is subjected to coordinate normalization to eliminate the coordinate difference between the image height dimension and the image width dimension, thus obtaining the processed point set; Triangulation calculations are performed on the set of processing points to determine the convex hull input points; The convex hull boundary is calculated using the convex hull construction algorithm and the convex hull input point to obtain the coordinates of the convex hull vertex.

8. The target region localization method according to claim 5, characterized in that, Perform boundary validity verification on the coordinates of the convex hull vertices to generate the region boundary data, including: The boundary validity of the convex hull vertex coordinates is verified using image boundary constraints and area threshold constraints to obtain the verification result. The image boundary constraints are constructed based on the pixel size of the camera image, and the area threshold constraints are constructed based on the area threshold of the target point region. In response to the verification result satisfying the target condition, the region boundary data is generated based on the coordinates of the convex hull vertex.

9. A target area localization device, characterized in that, include: The acquisition module is used to acquire the feature point set, vehicle pose and camera parameters corresponding to the calibration target in the calibration scene of the vehicle surround view system, wherein the feature point set is used to characterize the spatial geometric features of the target point area on the calibration target. The sampling module is used to perform tolerance space sampling based on the vehicle pose to obtain multiple sets of candidate poses; The projection module is used to project and map the feature point set onto the camera image plane according to the camera parameters and the multiple sets of candidate poses, so as to obtain the projection point set corresponding to the feature point set; The positioning module is used to perform boundary positioning calculations on the projection point set to obtain the region boundary data corresponding to the target point region.

10. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program executes the target region localization method according to any one of claims 1 to 8 when it runs.

Images