Correcting method, device, equipment, readable storage medium and computer program product

Abstract

Embodiments of the present application provide a correction method and device, equipment, computer readable storage medium and computer program product, relating to the fields of image recognition. The method comprises: obtaining a visible contour region of a target object in a target surround view, the target object being at a predetermined height from the ground, the visible contour region of the target object being used to represent a contour region of the target object on the target surround view; determining a predicted contour region of the target object based on the visible contour region of the target object, the predicted contour region of the target object being used to represent a predicted contour region of the target object on the ground in the target surround view; correcting the predicted contour region of the target object based on a preset mapping table to determine a corrected contour region; in this way, the position (predicted contour region) of the target object with the predetermined height is corrected, thereby obtaining a more accurate target object position (corrected contour region).

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a correction method, apparatus, device, readable storage medium, and computer program product. Background Technology

[0002] In current technologies for automated parking perception, a surround view is typically used as the perception input to detect obstacles (target objects) at a predetermined height within the surround view. Existing technologies usually segment the visible outline region of the obstacle, treating this outline region as the actual area of ​​the obstacle. However, due to image distortion caused by the stitching process of the surround view, the visible outline region of the obstacle cannot accurately represent its actual area. Therefore, using the visible outline region as the obstacle's location (actual area) will result in significant errors, leading to low accuracy in obstacle position detection. Summary of the Invention

[0003] This application addresses the shortcomings of existing methods by proposing a correction method, apparatus, device, computer-readable storage medium, and computer program product to solve the problem of how to improve the accuracy of obstacle position detection.

[0004] Firstly, this application provides a corrective method, including:

[0005] Obtain the visible outline region of the target object in the target toroidal view. The target object is at a predetermined height above the ground. The visible outline region of the target object is used to characterize the outline region of the target object in the target toroidal view.

[0006] Based on the visible contour region of the target object, the predicted contour region of the target object is determined. The predicted contour region of the target object is used to characterize the predicted contour region of the target object on the ground in the target surround view.

[0007] Based on a preset mapping table, the predicted contour region of the target object is corrected to determine the corrected contour region.

[0008] In one embodiment, determining the predicted contour region of the target object based on its visible contour region includes:

[0009] Input the target panoramic view into the preset detection model, perform detection processing on the visible contour area of ​​the target object, and obtain the predicted contour area of ​​the target object.

[0010] In one embodiment, based on a preset mapping table, the predicted contour region of the target object is corrected to determine the corrected contour region, including:

[0011] Based on a preset mapping table, the coordinates of any first point on the boundary of the predicted contour region are corrected to obtain the coordinates of any second point on the boundary of the corrected contour region; the mapping table includes the mapping relationship between the coordinates of any first point and the coordinates of any second point.

[0012] In one embodiment, before correcting the predicted contour region of the target object based on a preset mapping table and determining the corrected contour region, the method further includes:

[0013] Construct a plane, which is at a certain height above the ground;

[0014] Based on the coordinates of any third point on the plane and the projection matrix corresponding to the camera in the vehicle, the third point is projected onto the image captured by the camera to obtain the coordinates of the fourth point. The coordinates of the fourth point are the coordinates of the point on the image. The target surround view is determined by the images captured by each camera in the vehicle.

[0015] Based on the coordinates of the fourth point and the homography matrix corresponding to the camera, the fourth point is projected onto the target ring view to obtain the coordinates of the fifth point, which are the coordinates of the points on the target ring view.

[0016] Based on the coordinates of each third point and the corresponding coordinates of each fifth point, a preset mapping table is determined, wherein the coordinates of each third point include the coordinates of any second point, and the coordinates of each fifth point include the coordinates of any first point.

[0017] In one embodiment, before projecting the third point onto the image captured by the camera based on the coordinates of any third point on the plane and the projection matrix corresponding to the camera in the vehicle, to obtain the coordinates of the fourth point, the method further includes:

[0018] The projection matrix corresponding to the camera is determined based on the camera's intrinsic parameters and its extrinsic parameters relative to the vehicle.

[0019] In one embodiment, before projecting the fourth point onto the target loop view based on the coordinates of the fourth point and the homography matrix corresponding to the camera to obtain the coordinates of the fifth point, the method further includes:

[0020] Based on the projection matrix corresponding to the camera, any number of points in the pixel region corresponding to the camera are projected onto the image captured by the camera, resulting in multiple projected points on the image captured by the camera; the pixel region corresponding to the camera is the pixel region corresponding to the camera in the target panoramic view.

[0021] Determine the homography matrix corresponding to the camera based on any number of points in the pixel region corresponding to the camera and the multiple points projected.

[0022] Secondly, this application provides a corrective device, comprising:

[0023] The first processing module is used to obtain the visible outline area of ​​the target object in the target panoramic view. The target object is at a predetermined height from the ground. The visible outline area of ​​the target object is used to characterize the outline area of ​​the target object in the target panoramic view.

[0024] The second processing module is used to determine the predicted contour region of the target object based on the visible contour region of the target object. The predicted contour region of the target object is used to characterize the predicted contour region of the target object on the ground in the target surround view.

[0025] The third processing module is used to correct the predicted contour region of the target object based on a preset mapping table and determine the corrected contour region.

[0026] Thirdly, this application provides an electronic device, including: a processor, a memory, and a bus;

[0027] A bus is used to connect the processor and memory;

[0028] Memory, used to store operation instructions;

[0029] A processor is used to execute the correction method of the first aspect of this application by invoking operation instructions.

[0030] Fourthly, this application provides a computer-readable storage medium storing a computer program that is used to perform the correction method of the first aspect of this application.

[0031] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the correction method in the first aspect of this application.

[0032] The technical solution provided in this application has at least the following beneficial effects:

[0033] The visible contour region of the target object in the target surround view is obtained. The target object is at a predetermined height above the ground. The visible contour region of the target object is used to represent the contour region of the target object in the target surround view. Based on the visible contour region of the target object, the predicted contour region of the target object is determined. The predicted contour region of the target object is used to represent the predicted contour region of the target object on the ground in the target surround view. Based on a preset mapping table, the predicted contour region of the target object is corrected to determine the corrected contour region. In this way, the position (predicted contour region) of the target object at a predetermined height is corrected to obtain a more accurate target object position (corrected contour region), thereby improving the detection accuracy of the target object position (obstacle position). Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0035] Figure 1 This is a schematic diagram of the architecture of the correction system provided in the embodiments of this application;

[0036] Figure 2 A flowchart illustrating a correction method provided in an embodiment of this application;

[0037] Figure 3 A schematic diagram of a surround view provided for an embodiment of this application;

[0038] Figure 4 A schematic diagram of a surround view provided for an embodiment of this application;

[0039] Figure 5 A flowchart illustrating a correction method provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of the structure of a correction device provided in an embodiment of this application;

[0041] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0042] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0043] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” indicates implementation as “A,” or implementation as “B,” or implementation as “A and B.”

[0044] It is understood that in the specific implementation of this application, data related to correction is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0046] This application provides a correction method for a correction system, which relates to fields such as image recognition.

[0047] To better understand and explain the solutions of the embodiments of this application, some technical terms involved in the embodiments of this application will be briefly explained below.

[0048] Automatic parking: Automatic parking is a function that uses sensors around the vehicle to detect information about the surrounding environment and available parking space, plan the parking path, and control the vehicle's steering and acceleration / deceleration to complete the parking operation semi-automatically or automatically.

[0049] Homography matrix: In computer vision, the homography of a plane is defined as the projection mapping from one plane to another. If the homography is set as a two-dimensional matrix M, then photo 1 multiplied by M is photo 2. Homography matrix has many practical applications, such as image correction, image alignment, and camera motion calculation between two images.

[0050] The solutions provided in this application relate to correction techniques. The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0051] To better understand the solution provided in the embodiments of this application, the solution will be described below in conjunction with a specific application scenario.

[0052] In one embodiment, Figure 1 The diagram shows an architecture schematic of a correction system applicable to embodiments of this application. It is understood that the correction method provided in the embodiments of this application can be applied to, but is not limited to, applications such as... Figure 1 In the application scenarios shown.

[0053] In this example, as Figure 1As shown, the architecture of the correction system in this example may include, but is not limited to, vehicle 10, server 20, and database 30. Vehicle 10, server 20, and database 30 can interact via network 40. Vehicle 10 is a vehicle used to collect relevant data of a target scene (e.g., a scene in a parking lot). Vehicle 10 takes pictures using cameras and sends the pictures to server 20. Server 20 determines a target surround view based on the pictures. Server 20 obtains the visible contour region of the target object in the target surround view. The target object is at a predetermined height above the ground. The visible contour region of the target object is used to represent the contour region of the target object in the target surround view. Based on the visible contour region of the target object, server 20 determines the predicted contour region of the target object. The predicted contour region of the target object is used to represent the predicted contour region of the target object on the ground in the target surround view. Server 20 corrects the predicted contour region of the target object based on a preset mapping table to determine the corrected contour region. Server 20 sends the corrected contour region to database 30 for storage.

[0054] It is understood that the above is only one example, and this embodiment is not limited here.

[0055] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0056] The aforementioned networks may include, but are not limited to, wired networks and wireless networks. Wired networks include local area networks (LANs), metropolitan area networks (MANs), and wide area networks (WANs). Wireless networks include Bluetooth, Wi-Fi, and other networks that enable wireless communication. Specific details can be determined based on actual application scenario requirements and are not limited here.

[0057] See Figure 2 , Figure 2 This illustration shows a flowchart of a correction method provided in an embodiment of this application. The method can be executed by any electronic device, such as a server. As an optional implementation, the method can be executed by a server. For ease of description, the following descriptions of some optional embodiments will use a server as the execution subject of the method. Figure 2 As shown, the correction method provided in this application includes the following steps:

[0058] S201, Obtain the visible outline region of the target object in the target panoramic view. The target object is at a predetermined height above the ground. The visible outline region of the target object is used to characterize the outline region of the target object in the target panoramic view.

[0059] Specifically, the target object is an obstacle, such as the chassis of other vehicles besides your own in a parking lot. For example, such as... Figure 3 or Figure 4 The vehicle shown is used to collect relevant data about the scene in the parking lot. The target object is at a predetermined height above the ground, for example, the chassis is 20 centimeters above the ground. For example, the vehicle drives in the parking lot, takes pictures using multiple cameras (cameras) on the vehicle, and sends the pictures to the server; the server determines a target surround view based on the pictures; the server obtains the visible outline area of ​​the chassis in the target surround view, with the chassis at a predetermined height of 20 centimeters above the ground. For example, if... Figure 3 Representing a target surround view, the visible outline area of ​​the vehicle chassis is as follows: Figure 3 The area enclosed by the trapezoidal frame.

[0060] S202, Based on the visible contour region of the target object, determine the predicted contour region of the target object. The predicted contour region of the target object is used to characterize the predicted contour region of the target object on the ground in the target surround view.

[0061] Specifically, the target object is an obstacle, such as the chassis of other vehicles besides your own in a parking lot. For example, if Figure 4 Representing the target surround view, the predicted contour area of ​​the vehicle chassis is as follows: Figure 4 The area enclosed by the rectangular frame.

[0062] S203, based on a preset mapping table, corrects the predicted contour region of the target object and determines the corrected contour region.

[0063] Specifically, a pre-defined mapping table is used to represent the mapping relationship between the coordinates of points in the predicted contour region and the coordinates of points in the corrected contour region. The target object is an obstacle, such as the chassis of other vehicles besides your own in a parking lot. For example, if Figure 4 Representing the target surround view, the predicted contour region and the corrected contour region of the vehicle chassis are respectively as follows: Figure 4 The area enclosed by different rectangles.

[0064] In this embodiment, the visible contour region of the target object in the target surround view is obtained. The target object is at a predetermined height above the ground. The visible contour region of the target object is used to characterize the contour region of the target object in the target surround view. Based on the visible contour region of the target object, a predicted contour region of the target object is determined. The predicted contour region of the target object is used to characterize the predicted contour region of the target object on the ground in the target surround view. Based on a preset mapping table, the predicted contour region of the target object is corrected to determine the corrected contour region. In this way, the position (predicted contour region) of the target object at a predetermined height is corrected, thereby obtaining a more accurate target object position (corrected contour region) and improving the detection accuracy of the target object position (obstacle position).

[0065] In one embodiment, determining the predicted contour region of the target object based on its visible contour region includes:

[0066] Input the target panoramic view into the preset detection model, perform detection processing on the visible contour area of ​​the target object, and obtain the predicted contour area of ​​the target object.

[0067] Specifically, the toroidal view sample is manually annotated to obtain the original outline regions of the manually annotated toroidal view sample. The manual annotation process, for example, involves jointly inferring from multiple toroidal view samples within a certain timeframe before and after a given toroidal view sample, and then annotating the original outline regions of the obstacles in that toroidal view sample; the obstacle in the toroidal view sample is, for example, a vehicle chassis. For example, if... Figure 3 Representing a surround view sample, the original outline area of ​​the vehicle chassis is as follows: Figure 3 The area enclosed by the rectangular frame.

[0068] The training process of the detection model is as follows: a surround view sample is input into the detection model, which detects the visible contour regions of obstacles (such as the chassis of a vehicle) in the surround view sample and outputs the predicted contour regions of the obstacles in the surround view sample. The detection model substitutes the predicted contour regions of the obstacles in the surround view sample and the original contour regions annotated manually into the loss function. If the value of the loss function is less than or equal to a preset loss threshold, the training of the detection model ends. If the value of the loss function is greater than the preset loss threshold, the network parameters of the detection model are updated, and the training of the detection model continues until the value of the loss function is less than or equal to the preset loss threshold.

[0069] For example, the prediction process of the detection model involves inputting a target surround view into the detection model, detecting the visible contour regions of obstacles (such as vehicle chassis) in the target surround view, and obtaining the predicted contour regions of the obstacles.

[0070] The detection model can be RetinaNet (a one-stage network), FCOS (Fully Convolutional One-Stage Object Detection), or other object detection models.

[0071] In one embodiment, based on a preset mapping table, the predicted contour region of the target object is corrected to determine the corrected contour region, including:

[0072] Based on a preset mapping table, the coordinates of any first point on the boundary of the predicted contour region are corrected to obtain the coordinates of any second point on the boundary of the corrected contour region; the mapping table includes the mapping relationship between the coordinates of any first point and the coordinates of any second point.

[0073] Specifically, the preset mapping table is used to represent the mapping relationship between the coordinates of points on the predicted contour region and the coordinates of points on the corrected contour region; for example, the coordinates of any first point on the predicted contour region are corrected to obtain the coordinates of any second point on the corrected contour region; the mapping table includes the mapping relationship between the coordinates of any first point and the coordinates of any second point.

[0074] For example, any first point is P2, and any second point is P1; based on a preset mapping table, the coordinates of point P2 on the boundary of the predicted contour region are corrected to obtain the coordinates of point P1 on the boundary of the corrected contour region; the mapping table includes the mapping relationship between the coordinates of point P2 and the coordinates of point P1.

[0075] In one embodiment, before correcting the predicted contour region of the target object based on a preset mapping table and determining the corrected contour region, steps A1-A4 are further included:

[0076] Step A1: Construct a plane at a certain height from the ground.

[0077] Specifically, for example, in the vehicle coordinate system, a plane is selected at a certain height above the ground, such as 20 centimeters.

[0078] Step A2: Based on the coordinates of any third point on the plane and the projection matrix corresponding to the camera in the vehicle, project the third point onto the image captured by the camera to obtain the coordinates of the fourth point. The coordinates of the fourth point are the coordinates of the point on the image. The target surround view is determined by the images captured by each camera in the vehicle.

[0079] Specifically, for example, let any third point be point P1; based on the coordinates of point P1 on the plane and the projection matrix corresponding to a camera P in the vehicle, project point P1 onto the image captured by the camera P to obtain the coordinates of the fourth point; calculate the coordinates of the fourth point as shown in formula (1):

[0080] The coordinates of the fourth point = projection matrix × coordinates of point P1 (1)

[0081] It should be noted that if P1 is in the common view of at least two cameras in the vehicle, then camera P is any one of the at least two cameras.

[0082] Step A3: Based on the coordinates of the fourth point and the homography matrix corresponding to the camera, project the fourth point onto the target ring view to obtain the coordinates of the fifth point, which are the coordinates of the point on the target ring view.

[0083] Specifically, for example, the fifth point is point P2; based on the coordinates of the fourth point and the homography matrix corresponding to the camera, the fourth point is projected onto the target ring view to obtain the coordinates of point P2; the coordinates of point P2 are calculated as shown in formula (2):

[0084] The coordinates of point P2 = homography matrix × coordinates of the fourth point (Formula 2)

[0085] Step A4: Based on the coordinates of each third point and the corresponding coordinates of each fifth point, determine a preset mapping table, where the coordinates of each third point include the coordinates of any second point, and the coordinates of each fifth point include the coordinates of any first point.

[0086] Specifically, for example, a mapping relationship is established between the coordinates of point P2 in the predicted contour region and the coordinates of point P1 in the corrected contour region, and the mapping table includes this mapping relationship.

[0087] In one embodiment, before projecting the third point onto the image captured by the camera based on the coordinates of any third point on the plane and the projection matrix corresponding to the camera in the vehicle, to obtain the coordinates of the fourth point, the method further includes:

[0088] The projection matrix corresponding to the camera is determined based on the camera's intrinsic parameters and its extrinsic parameters relative to the vehicle.

[0089] Specifically, the camera's intrinsic parameters (intrinsic parameter matrix) include, for example, the camera's focal length, optical center, and distortion coefficients. These intrinsic parameters describe the transformation from the image coordinate system to the camera coordinate system. The camera's extrinsic parameters (extrinsic parameter matrix) relative to the vehicle include, for example, the camera's position parameters relative to the vehicle. These extrinsic parameters describe the transformation relationship from the camera coordinate system to the vehicle coordinate system. The projection matrix corresponding to the camera is calculated as shown in formula (3):

[0090] Projection matrix = Intrinsic parameter matrix × Extrinsic parameter matrix (Formula 3)

[0091] In one embodiment, before projecting the fourth point onto the target ring view based on the coordinates of the fourth point and the homography matrix corresponding to the camera to obtain the coordinates of the fifth point, steps B1-B2 are further included:

[0092] Step B1: Based on the projection matrix corresponding to the camera, project any number of points in the pixel region corresponding to the camera onto the image captured by the camera, to obtain multiple projected points on the image captured by the camera; the pixel region corresponding to the camera is the pixel region corresponding to the camera in the target panoramic view.

[0093] Specifically, each camera in the vehicle has a corresponding pixel area in the surround view. For example, camera C has a corresponding pixel area D in the surround view. Four points can be arbitrarily selected in pixel area D (any number of points in the pixel area corresponding to the camera). Based on the projection matrix corresponding to the camera, these four points are projected onto the image captured by the camera to obtain the four projected points on the image captured by the camera.

[0094] Step B2: Determine the homography matrix corresponding to the camera based on any number of points in the pixel region corresponding to the camera and the number of projected points.

[0095] Specifically, for example, the homography matrix corresponding to the camera is determined based on four points in the pixel region D corresponding to the camera and four points projected onto the image captured by the camera.

[0096] Applying the embodiments of this application has at least the following beneficial effects:

[0097] For target objects with a predetermined height, the position (predicted contour area) is corrected to obtain a more accurate target object position (corrected contour area), thereby improving the detection accuracy of target object position (obstacle position).

[0098] To better understand the methods provided in the embodiments of this application, the solutions of the embodiments of this application will be further explained below with reference to specific application scenarios.

[0099] In a specific application scenario, such as obstacle location detection in a parking lot, see [link to example]. Figure 5 This illustrates the processing flow of a correction method, such as... Figure 5 As shown, the processing flow of the correction method provided in this application embodiment includes the following steps:

[0100] S501, the user manually annotates the surround view sample of the scene in the parking lot to obtain the original outline area of ​​the obstacle in the surround view sample.

[0101] Specifically, users can label obstacles in a round-view sample by jointly inferring from multiple round-view samples within a certain time period before and after the round-view sample, thus marking the original outline area of ​​the obstacles in the round-view sample.

[0102] S502, the server trains the detection model based on the surround view samples and the original contour regions of obstacles in the surround view samples, and obtains the trained detection model.

[0103] Specifically, the server inputs the surround view sample into the detection model. The detection model detects the visible contour regions of obstacles in the surround view sample and outputs the predicted contour regions of obstacles in the surround view sample. The detection model substitutes the predicted contour regions of obstacles in the surround view sample and the original contour regions of obstacles in the surround view sample into the loss function. If the value of the loss function is less than or equal to the preset loss threshold, the training of the detection model ends, and the trained detection model is obtained.

[0104] S503: The vehicle drives in the parking lot, takes pictures using multiple cameras on the vehicle, and sends the pictures to the server.

[0105] Specifically, a self-driving car is a vehicle used to collect relevant data about the scene in a parking lot.

[0106] S504: The server determines the target surround view through images captured by multiple cameras, and inputs the target surround view into the trained detection model to detect the visible contour regions of obstacles in the target surround view, thereby obtaining the predicted contour regions of the obstacles.

[0107] Specifically, trained detection models include object detection models such as RetinaNet and FCOS.

[0108] S505, the server determines the projection matrix and homography matrix corresponding to the camera.

[0109] Specifically, the server determines the projection matrix corresponding to the camera based on the camera's intrinsic parameters and extrinsic parameters relative to the vehicle. Based on this projection matrix, the server projects any number of points in the camera's corresponding pixel region onto the image captured by the camera, resulting in multiple projected points on the image. The camera's corresponding pixel region is the pixel region corresponding to the camera in the target panoramic view. The server then determines the homography matrix corresponding to the camera based on these points and the projected points.

[0110] In S506, the server constructs a mapping table based on the projection matrix and homography matrix corresponding to the camera.

[0111] Specifically, the server constructs a plane at a certain height above the ground. Based on the coordinates of any third point on the plane and the projection matrix corresponding to the camera in the vehicle, the server projects the third point onto the image captured by the camera to obtain the coordinates of the fourth point. The coordinates of the fourth point are the point coordinates on the image. The target surround view is determined by the images captured by each camera in the vehicle. Based on the coordinates of the fourth point and the homography matrix corresponding to the camera, the server projects the fourth point onto the target surround view to obtain the coordinates of the fifth point. The coordinates of the fifth point are the point coordinates on the target surround view. Based on the coordinates of each third point and the corresponding coordinates of each fifth point, the server determines a mapping table. The coordinates of each third point include the coordinates of any second point, and the coordinates of each fifth point include the coordinates of any first point.

[0112] S507, the server corrects the predicted contour region of obstacles in the target surround view based on the mapping table, and determines the corrected contour region.

[0113] Specifically, the server corrects the coordinates of any first point on the boundary of the predicted contour region based on the mapping table, and obtains the coordinates of any second point on the boundary of the corrected contour region; the mapping table includes the mapping relationship between the coordinates of any first point and the coordinates of any second point.

[0114] Applying the embodiments of this application has at least the following beneficial effects:

[0115] For obstacles with a predetermined height, the position (predicted contour area) is corrected to obtain a more accurate obstacle position (corrected contour area), thereby improving the accuracy of obstacle position detection.

[0116] This application also provides a correction device, the structural schematic diagram of which is shown below. Figure 6 As shown, the correction device 60 includes a first processing module 601, a second processing module 602, and a third processing module 603.

[0117] The first processing module 601 is used to obtain the visible outline area of ​​the target object in the target panoramic view. The target object is at a predetermined height from the ground. The visible outline area of ​​the target object is used to characterize the outline area of ​​the target object in the target panoramic view.

[0118] The second processing module 602 is used to determine the predicted contour region of the target object based on the visible contour region of the target object. The predicted contour region of the target object is used to characterize the predicted contour region of the target object on the ground in the target surround view.

[0119] The third processing module 603 is used to correct the predicted contour region of the target object based on a preset mapping table and determine the corrected contour region.

[0120] In one embodiment, the second processing module 602 is specifically used for:

[0121] Input the target panoramic view into the preset detection model, perform detection processing on the visible contour area of ​​the target object, and obtain the predicted contour area of ​​the target object.

[0122] In one embodiment, the third processing module 603 is specifically used for:

[0123] Based on a preset mapping table, the coordinates of any first point on the boundary of the predicted contour region are corrected to obtain the coordinates of any second point on the boundary of the corrected contour region; the mapping table includes the mapping relationship between the coordinates of any first point and the coordinates of any second point.

[0124] In one embodiment, the third processing module 603 is further configured to:

[0125] Construct a plane, which is at a certain height above the ground;

[0126] Based on the coordinates of any third point on the plane and the projection matrix corresponding to the camera in the vehicle, the third point is projected onto the image captured by the camera to obtain the coordinates of the fourth point. The coordinates of the fourth point are the coordinates of the point on the image. The target surround view is determined by the images captured by each camera in the vehicle.

[0127] Based on the coordinates of the fourth point and the homography matrix corresponding to the camera, the fourth point is projected onto the target ring view to obtain the coordinates of the fifth point, which are the coordinates of the points on the target ring view.

[0128] Based on the coordinates of each third point and the corresponding coordinates of each fifth point, a preset mapping table is determined, wherein the coordinates of each third point include the coordinates of any second point, and the coordinates of each fifth point include the coordinates of any first point.

[0129] In one embodiment, the third processing module 603 is further configured to:

[0130] The projection matrix corresponding to the camera is determined based on the camera's intrinsic parameters and its extrinsic parameters relative to the vehicle.

[0131] In one embodiment, the third processing module 603 is further configured to:

[0132] Based on the projection matrix corresponding to the camera, any number of points in the pixel region corresponding to the camera are projected onto the image captured by the camera, resulting in multiple projected points on the image captured by the camera; the pixel region corresponding to the camera is the pixel region corresponding to the camera in the target panoramic view.

[0133] Determine the homography matrix corresponding to the camera based on any number of points in the pixel region corresponding to the camera and the multiple points projected.

[0134] Applying the embodiments of this application has at least the following beneficial effects:

[0135] The visible contour region of the target object in the target surround view is obtained. The target object is at a predetermined height above the ground. The visible contour region of the target object is used to represent the contour region of the target object in the target surround view. Based on the visible contour region of the target object, the predicted contour region of the target object is determined. The predicted contour region of the target object is used to represent the predicted contour region of the target object on the ground in the target surround view. Based on a preset mapping table, the predicted contour region of the target object is corrected to determine the corrected contour region. In this way, the position (predicted contour region) of the target object at a predetermined height is corrected to obtain a more accurate target object position (corrected contour region), thereby improving the detection accuracy of the target object position (obstacle position).

[0136] This application also provides an electronic device, the structural schematic diagram of which is shown below. Figure 7 As shown, Figure 7 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0137] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0138] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0139] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0140] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0141] Electronic devices include, but are not limited to, servers.

[0142] Applying the embodiments of this application has at least the following beneficial effects:

[0143] The visible contour region of the target object in the target surround view is obtained. The target object is at a predetermined height above the ground. The visible contour region of the target object is used to represent the contour region of the target object in the target surround view. Based on the visible contour region of the target object, the predicted contour region of the target object is determined. The predicted contour region of the target object is used to represent the predicted contour region of the target object on the ground in the target surround view. Based on a preset mapping table, the predicted contour region of the target object is corrected to determine the corrected contour region. In this way, the position (predicted contour region) of the target object at a predetermined height is corrected to obtain a more accurate target object position (corrected contour region), thereby improving the detection accuracy of the target object position (obstacle position).

[0144] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0145] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0146] Based on the same principles as the methods provided in the embodiments of this application, the embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in any of the optional embodiments of this application described above.

[0147] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0148] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A method of correction, characterized in that, The method comprises: acquiring a visible contour area of a target object in a target ring view, the target object being at a predetermined height from the ground, the visible contour area of the target object being used to represent a contour area of the target object on the target ring view; determining a predicted contour area of the target object based on the visible contour area of the target object, the predicted contour area of the target object being used to represent a predicted contour area of the target object on the ground in the target ring view; correcting the predicted contour area of the target object based on a preset mapping table to determine a corrected contour area; the step of correcting the predicted contour area of the target object based on the preset mapping table to determine the corrected contour area comprises: correcting a coordinate of any first point on a boundary of the predicted contour area based on the preset mapping table to obtain a coordinate of any second point on a boundary of the corrected contour area; the mapping table comprises a mapping relationship between the coordinate of the any first point and the coordinate of the any second point; before the step of correcting the predicted contour area of the target object based on the preset mapping table to determine the corrected contour area, the method further comprises: constructing a plane, the plane being at a predetermined height from the ground; projecting any third point on the plane to a picture taken by a camera in a vehicle based on a coordinate of the third point and a projection matrix corresponding to the camera to obtain a coordinate of a fourth point, the coordinate of the fourth point being a point coordinate on the picture, the target ring view being determined based on pictures taken by each camera in the vehicle; projecting the fourth point to the target ring view based on the coordinate of the fourth point and a homography matrix corresponding to the camera to obtain a coordinate of a fifth point, the coordinate of the fifth point being a point coordinate on the target ring view; determining the preset mapping table based on the coordinates of the third points and the coordinates of the corresponding fifth points, the coordinates of the third points comprising the coordinate of the any second point, and the coordinates of the fifth points comprising the coordinate of the any first point.

2. The method of claim 1, wherein, the step of determining the predicted contour area of the target object based on the visible contour area of the target object comprises: inputting the target ring view into a preset detection model to perform detection processing on the visible contour area of the target object to obtain the predicted contour area of the target object.

3. The method of claim 1, wherein, before the step of projecting any third point on the plane to a picture taken by a camera in a vehicle based on a coordinate of the third point and a projection matrix corresponding to the camera to obtain a coordinate of a fourth point, the method further comprises: determining the projection matrix corresponding to the camera based on intrinsic parameters of the camera and extrinsic parameters of the camera relative to the vehicle.

4. The method of claim 1, wherein, before the step of projecting the fourth point to the target ring view based on the coordinate of the fourth point and a homography matrix corresponding to the camera to obtain a coordinate of a fifth point, the method further comprises: projecting any multiple points in the pixel region corresponding to the camera to a picture taken by the camera based on a projection matrix corresponding to the camera, to obtain multiple projected points on the picture taken by the camera; the pixel region corresponding to the camera is a pixel region corresponding to the camera on the target surround view; determining a homography matrix corresponding to the camera based on the multiple projected points and the multiple points in the pixel region corresponding to the camera.

5. An orthotic device characterized by, The method comprises: a first processing module configured to acquire a visible contour region of a target object in a target surround view, the target object being at a predetermined height from the ground, and the visible contour region of the target object being used to represent a contour region of the target object on the target surround view; a second processing module configured to determine a predicted contour region of the target object based on the visible contour region of the target object, the predicted contour region of the target object being used to represent a predicted contour region of the target object on the ground in the target surround view; a third processing module configured to correct the predicted contour region of the target object based on a preset mapping table, to obtain a corrected contour region. The third processing module is specifically configured to: correct a coordinate of any first point on a boundary of the predicted contour region based on the preset mapping table, to obtain a coordinate of any second point on a boundary of the corrected contour region; and the mapping table comprises a mapping relationship between the coordinate of the any first point and the coordinate of the any second point. The third processing module is further configured to: construct a plane, the plane being at a preset height from the ground; project the any third point on the plane to a picture taken by a camera in the ego vehicle based on a coordinate of the any third point and a projection matrix corresponding to the camera, to obtain a coordinate of a fourth point, the coordinate of the fourth point being a point coordinate on the picture, and the target surround view being determined based on pictures taken by cameras in the ego vehicle; project the fourth point to the target surround view based on the coordinate of the fourth point and a homography matrix corresponding to the camera, to obtain a coordinate of a fifth point, the coordinate of the fifth point being a point coordinate on the target surround view; determine the preset mapping table based on the coordinates of the third points and the coordinates of the fifth points corresponding to the third points, the coordinates of the third points comprising the coordinate of the any second point, and the coordinates of the fifth points comprising the coordinate of the any first point.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-5. The processor executes the computer program to implement the steps of the method of any one of claims 1-4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-4.

Images