Information processing methods, devices, electronic equipment and storage media

Abstract

The present application relates to an information processing method, device, electronic device and storage medium, which can be applied to the field of maps. The method includes: obtaining acquisition trajectory information of an image acquisition device, the acquisition trajectory information includes multiple first images and the positioning information of each of the multiple first images; based on the positioning information of each of the multiple first images, screening multiple second images from the multiple first images, the trajectory angle change information corresponding to the positioning information of the multiple second images is greater than the angle threshold; based on the multiple second images, obtaining the target point cloud data, initial posture information and initial internal reference information corresponding to the acquisition trajectory information; calibrating the initial internal reference information according to the target point cloud data and the initial posture information to obtain the target internal reference information of the image acquisition device. According to the technical solution of the present application, the accuracy of the target internal reference information can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an information processing method, apparatus, electronic device and storage medium. Background Technology

[0002] Currently, crowdsourced camera equipment varies in brand and is generally fisheye lens, resulting in significant distortion. To obtain more realistic images or perform accurate ranging and 3D reconstruction, camera intrinsics are calibrated. The calibrated intrinsics can then be used for image correction, ranging, and 3D reconstruction. However, related technologies typically use images collected by crowdsourced equipment directly to estimate camera intrinsics, leading to inaccurate estimations and consequently, suboptimal results in image correction and 3D reconstruction. Summary of the Invention

[0003] In view of the aforementioned technical problems, this application proposes an information processing method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of this application, an information processing method is provided, the method comprising:

[0005] Acquire acquisition trajectory information of an image acquisition device, wherein the acquisition trajectory information includes multiple first images and the positioning information of each of the multiple first images;

[0006] Based on the positioning information of each of the multiple first images, multiple second images are selected from the multiple first images, and the trajectory angle change information corresponding to the positioning information of the multiple second images is greater than the angle threshold.

[0007] Based on the multiple second images, acquire the target point cloud data, initial pose information, and initial intrinsic parameter information corresponding to the acquired trajectory information;

[0008] Based on the target point cloud data and the initial pose information, the initial intrinsic parameter information is calibrated to obtain the target intrinsic parameter information of the image acquisition device.

[0009] According to another aspect of this application, an information processing apparatus is provided, comprising:

[0010] The acquisition module is used to acquire acquisition trajectory information of the image acquisition device, the acquisition trajectory information including multiple first images and the positioning information of each of the multiple first images;

[0011] A filtering module is used to filter multiple second images from the multiple first images based on the positioning information of each of the multiple first images, wherein the trajectory angle change information corresponding to the positioning information of the multiple second images is greater than an angle threshold.

[0012] The three-dimensional recovery module is used to acquire target point cloud data, initial pose information and initial intrinsic parameter information corresponding to the acquisition trajectory information based on the multiple second images;

[0013] The target intrinsic parameter information acquisition module is used to calibrate the initial intrinsic parameter information based on the target point cloud data and the initial pose information to obtain the target intrinsic parameter information of the image acquisition device.

[0014] According to another aspect of this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the above-described method.

[0015] According to another aspect of this application, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0016] Multiple first images and their respective positioning information are acquired by an image acquisition device. From these first images, multiple second images with trajectory angle changes greater than an angle threshold are selected. Target point cloud data, initial pose information, and initial intrinsic parameter information are then obtained based on these second images. Furthermore, the initial intrinsic parameter information is calibrated based on the target point cloud data and initial pose information. Since the trajectory angle changes of the multiple second images are greater than the angle threshold, the intrinsic parameters can be effectively constrained, thereby improving the accuracy of intrinsic parameter calibration. In other words, the purpose of automatic and accurate intrinsic parameter calibration can be achieved.

[0017] Other features and aspects of this application will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0018] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this application together with the specification and serve to explain the principles of this application.

[0019] Figure 1 This diagram illustrates an application system provided according to an embodiment of the present application.

[0020] Figure 2 A flowchart is shown for an information processing method provided according to an embodiment of this application.

[0021] Figure 3 This illustration shows a flowchart of an information processing method according to an embodiment of the present application.

[0022] Figures 4a-4c This diagram illustrates the motion of an image acquisition device according to an embodiment of this application.

[0023] Figures 5a-5b This diagram illustrates a turning point according to an embodiment of the present application.

[0024] Figure 6 This diagram illustrates a three-dimensional reconstruction of multiple acquisition trajectories according to an embodiment of this application.

[0025] Figure 7 The diagram illustrates a method for selecting multiple second images from multiple first images based on the positioning information of each of the multiple first images, according to an embodiment of this application.

[0026] Figure 8 The flowchart illustrates a method for obtaining target intrinsic parameter information of an image acquisition device by calibrating initial intrinsic parameter information based on target point cloud data and initial pose information, according to an embodiment of this application.

[0027] Figure 9 This illustration shows a reprojection diagram according to an embodiment of the present application.

[0028] Figure 10 This diagram illustrates a block diagram of an information processing apparatus according to an embodiment of the present application.

[0029] Figure 11 This diagram illustrates an electronic device for information processing according to an embodiment of the present application. Detailed Implementation

[0030] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0031] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0032] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0033] Please see Figure 1 , Figure 1 This diagram illustrates an application system according to an embodiment of this application. The application system can be used in the information processing method of this application. Figure 1As shown, the application system may include at least server 01 and terminal 02.

[0034] In this embodiment of the application, the server 01 can be used for information processing. The server 01 may include an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0035] In this embodiment, the terminal 02 can be used to acquire images, such as images on a road. As an example, the terminal can be a crowdsourced device (a device for image acquisition), such as a vehicle camera or a dashcam; this application is not limited to this. The terminal 02 can also include physical devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices. Physical devices can also include software running on them, such as applications. In this embodiment, the operating system running on the terminal 02 can include, but is not limited to, Android, iOS, Linux, and Windows.

[0036] Terminal 02 can upload the collected images to the cloud (e.g., server 01), and the service platform can use these collected images to calibrate the intrinsic parameters of terminal 02. These collected images can carry location information, such as GPS (Global Positioning System) location information.

[0037] In the embodiments described in this specification, the terminal 02 and the server 01 can be directly or indirectly connected through wired or wireless communication, and this application does not limit this connection.

[0038] In a specific embodiment, when server 02 is a distributed system, this distributed system can be a blockchain system. When the distributed system is a blockchain system, it can be formed by multiple nodes (any form of computing device connected to the network, such as servers or user terminals). These nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). In a distributed system, any machine, such as a server or terminal, can join and become a node. A node includes a hardware layer, a middleware layer, an operating system layer, and an application layer. Specifically, the functions of each node in the blockchain system may include:

[0039] 1) Routing: A basic function of nodes used to support communication between nodes.

[0040] In addition to routing capabilities, nodes can also have the following functions:

[0041] 2) Applications are deployed in the blockchain to implement specific business needs. They record data related to the implementation of functions to form record data, carry digital signatures in the record data to indicate the source of the task data, and send the record data to other nodes in the blockchain system. When other nodes successfully verify the source and integrity of the record data, they add the record data to a temporary block.

[0042] It should be noted that in the specific embodiments of this application, data related to user information is involved. When the following embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0043] Figure 2 A flowchart illustrating an information processing method according to an embodiment of this application is shown. Figure 2 As shown, the method may include:

[0044] S201, Obtain the acquisition trajectory information of the image acquisition device. The acquisition trajectory information includes multiple first images and the positioning information of each of the multiple first images.

[0045] As an example, Figure 3 This is a flowchart illustrating the information processing process, such as... Figure 3As shown, the original data (multiple first images and their respective positioning information) can be cleaned to obtain cleaned data: turning point images. Based on these turning point images, 3D reconstruction can be performed to obtain target point cloud data, initial pose information, and initial intrinsic parameter information. Furthermore, the initial intrinsic parameters can be optimized to obtain target intrinsic parameter information (optimized result). Optionally, image correction, ranging, map annotation, and 3D reconstruction can be performed based on the target intrinsic parameter information. Figure 3 As shown, image correction can include image distortion correction. The specific information processing flow is described below.

[0046] In the embodiments of this specification, in order to calibrate the intrinsic parameters of the image acquisition device (crowdsourced device) so as to use the calibrated target intrinsic parameter information for image correction (such as distortion correction), ranging, or 3D reconstruction, accurate target intrinsic parameter information can improve the efficiency and success rate of 3D reconstruction. Based on this, the acquisition trajectory information of the image acquisition device can be obtained. This acquisition trajectory information can be multiple first images uploaded by the image acquisition device and arranged in sequence, carrying positioning information. That is, the acquisition trajectory information can include multiple first images and the positioning information of each of the multiple first images. Arranged in sequence can mean arranged according to the acquisition time order of the multiple first images.

[0047] S203, based on the positioning information of each of the multiple first images, select multiple second images from the multiple first images, wherein the trajectory angle change information corresponding to the positioning information of the multiple second images is greater than the angle threshold.

[0048] In the embodiments of this specification, the trajectory angle change information corresponding to the positioning information of multiple second images is greater than an angle threshold. That is, the multiple second images are images of turning points within the acquisition trajectory from among multiple first images. The selection of turning point images for subsequent 3D reconstruction and intrinsic parameter calibration is determined based on the discovery and analysis of various camera movements as shown in Figure 4. Specifically, as... Figure 4a As shown, when the camera (image acquisition device) moves parallel to the image, image Img1 moves to Img2, as... Figure 4a As shown, there is a common 3D spatial point P in the two images. When the focal length increases, the 3D spatial point p corresponding to the feature point u1 in Img1 will move along the ray from the camera center and the feature point u1 to a position p' farther away from the camera plane. The line connecting p' and the focal point of image Img2 still intersects the imaging plane of image Img2 at u2. Therefore, the change in focal length will cause the overall scaling of the spatial point. However, since the 3D reconstruction based purely on the image has no scale information, its constraints are insufficient to solve the focal length.

[0049] like Figure 4bAs shown, when the camera moves back and forth, that is, when the camera's direction of movement is perpendicular to the camera's imaging plane, the image Img1 moves to Img2, as shown. Figure 4b As shown, when the camera's focal length increases, the 3D spatial point p' corresponding to the feature point u2 in image Img2 is connected to the focal point of image Img1, and intersects with the imaging plane of Img1 as u1'. Its position deviates slightly from the position of feature point u1 before the focal length increases (i.e., residual). Therefore, a residual exists when the focal length changes, but the residual is too small. Image feature matching also has a certain deviation between pixels, so this residual is still insufficient to constrain the focal length.

[0050] like Figure 4c As shown, when the camera movement causes the center of the front and rear images to have a certain angle, when the focal length increases, the position u2' of the spatial point p' corresponding to the feature point u1 in image Img1 projected in image Img2 has a large residual with the feature point u2. Therefore, this motion method can effectively constrain all parameters and successfully solve the focal length.

[0051] In summary, for different camera movements, the residuals generated by backprojecting 3D points will vary depending on the camera focal length. Only when the camera movement has a certain angle can the constraint be sufficient to calculate the focal length and obtain the correct intrinsic parameters. Therefore, we select crowdsourced data containing turning or camera movement deflection for intrinsic parameter self-calibration to ensure the accuracy of the intrinsic parameter calibration.

[0052] In the embodiments of this specification, the location points corresponding to these positioning information can be discrete points, such as... Figure 5a and 5b As shown. When collecting trajectory information as a single trajectory, these discrete points can be fitted into a trajectory curve. This allows extraction of points from the trajectory curve where the trajectory angle change exceeds an angle threshold, i.e., turning points, such as... Figure 5a and 5b The figures 501 and 502 are shown in the diagram. The angle threshold can be set to 30°, but this application does not limit this setting. Furthermore, the first image corresponding to the turning point and the first image corresponding to points within a preset range from the turning point can be used as multiple second images. That is, the first image near the turning point is selected from the multiple first images as the second image. The preset range can be a preset distance, and this application does not limit the preset distance.

[0053] S205, based on multiple second images, acquires target point cloud data, initial pose information and initial intrinsic parameter information corresponding to the acquired trajectory information.

[0054] In one possible implementation, target point cloud data, initial pose information, and initial intrinsic parameter information can be constructed based on 3D reconstruction. As an example, the aforementioned acquisition trajectory information can be the information of a single acquisition trajectory, which can be constructed using the following steps:

[0055] Extract first corner feature information from multiple second images;

[0056] Based on the feature information of the first corner point, the trajectory is reconstructed in three dimensions to obtain the target point cloud data, initial pose information and initial intrinsic parameter information.

[0057] The target point cloud data can be point cloud data corresponding to multiple first images in space, and the initial position information and initial intrinsic parameter information can be the initial pose information and initial intrinsic parameter information of the image acquisition device.

[0058] In practical applications, corner points in the second image can be feature extracted to obtain first corner point feature information. This first corner point feature information can then be used for trajectory 3D reconstruction to obtain point cloud data in the corresponding 3D space. For example, pose information can be estimated based on the first corner point feature information of adjacent second images to obtain initial pose information; and initial intrinsic parameter information such as focal length can be obtained based on the positioning information of the first corner point feature information. Therefore, back-projection processing can be performed on the first corner point feature information based on the initial pose information and initial intrinsic parameter information to obtain point cloud data in the corresponding 3D space. This disclosure does not limit the specific process of 3D reconstruction.

[0059] Optionally, the aforementioned data collection trajectory information can be information on multiple data collection trajectories, where multiple data collection trajectories can refer to data collection trajectories corresponding to multiple data collection time periods. In this case, such as... Figure 6 As shown, turning point mining and 3D reconstruction can be performed separately for each acquisition trajectory to obtain point cloud data, pose information, and intrinsic parameter information corresponding to each acquisition trajectory. Based on this, the above S205 may include the following steps:

[0060] From multiple second images, determine the second image corresponding to each acquisition trajectory;

[0061] Corner feature extraction is performed on the second image corresponding to each acquisition trajectory to obtain the second corner feature information corresponding to each acquisition trajectory.

[0062] Based on the feature information of the second corner point of each acquisition trajectory, the trajectory 3D reconstruction process is performed to obtain multiple point cloud data, multiple pose information and multiple intrinsic parameter information corresponding to multiple acquisition trajectories;

[0063] Based on multiple point cloud data, multiple pose information and multiple intrinsic parameter information, the target point cloud data, initial pose information and initial intrinsic parameter information are obtained.

[0064] In the embodiments of this specification, information from multiple acquisition trajectories of the same image acquisition device can be used to improve the success rate of 3D reconstruction. Based on this, point cloud data, pose information, and intrinsic parameter information corresponding to each acquisition trajectory can be obtained, that is, multiple point cloud data, multiple pose information, and multiple intrinsic parameter information can be obtained. Thus, multiple point cloud data can be used as target point cloud data, multiple pose information can be used as initial pose information, and multiple intrinsic parameter information can be used as initial intrinsic parameter information.

[0065] In another possible implementation, visual odometry can be used to perform 3D spatial reconstruction processing on multiple second images to obtain the aforementioned target point cloud data, initial pose information, and initial intrinsic parameter information. For example, key information (such as corner information) can be captured in real time from the second images, thereby obtaining rotation and translation matrices based on key information between adjacent frames. Accumulating multiple frames in this way constitutes visual odometry, thereby obtaining initial pose information, initial intrinsic parameter information, and target point cloud data in the corresponding 3D space. This disclosure does not limit these aspects.

[0066] S207. Based on the target point cloud data and initial pose information, the initial intrinsic parameter information is calibrated to obtain the target intrinsic parameter information of the image acquisition device.

[0067] In the embodiments of this specification, a reprojection process can be performed on a three-dimensional point space P in the target point cloud data to a second image H, thereby obtaining a second pixel point in the second image from which the three-dimensional point P is reprojected; and a first pixel point in the second image H that matches the three-dimensional point P can be determined. Thus, the distance between the second pixel point and the first pixel point is obtained. The above reprojection process can be performed on any three-dimensional point P in the target point cloud data to obtain multiple distances. The second image H can be any one of multiple second images. Based on this, the sum of multiple distances can be determined, thereby adjusting the initial intrinsic parameter information to reduce the sum of distances. When the sum of distances is less than a distance threshold, the target intrinsic parameter information is obtained. This disclosure does not limit the specific adjustment method of the initial intrinsic parameter information.

[0068] It should be noted that, in the case of multiple acquisition trajectories, the intrinsic parameter information of each acquisition trajectory in the initial intrinsic parameter information can be optimized to obtain multiple optimized intrinsic parameter information. The target intrinsic parameter information can then be determined based on these optimized intrinsic parameter information; for example, the average of the multiple optimized intrinsic parameter information can be used as the target intrinsic parameter information. This allows the target intrinsic parameter information of the corresponding acquisition trajectory to be used during image distortion correction, ranging, and 3D reconstruction.

[0069] Multiple first images and their respective positioning information are acquired by an image acquisition device. From these first images, multiple second images with trajectory angle changes greater than an angle threshold are selected. Target point cloud data, initial pose information, and initial intrinsic parameter information are then obtained based on these second images. Furthermore, the initial intrinsic parameter information is calibrated based on the target point cloud data and initial pose information. Since the trajectory angle changes of the multiple second images are greater than the angle threshold, the intrinsic parameters can be effectively constrained, thereby improving the accuracy of intrinsic parameter calibration. In other words, the purpose of automatic and accurate intrinsic parameter calibration can be achieved.

[0070] Figure 7 This diagram illustrates a method for selecting multiple second images from multiple first images based on the positioning information of each of the multiple first images, according to an embodiment of this application. Figure 7 As shown, in one possible implementation, step S203 may include:

[0071] S701 uses the Douglas-Puk algorithm to filter the positioning information and obtain the target positioning information.

[0072] S703, select the first image corresponding to the target positioning information from multiple first images, and use it as multiple second images.

[0073] In practical applications, crowdsourced image data (multiple initial images) contains GPS information. However, the location points corresponding to GPS information are a series of discrete points, making it impossible to directly determine changes in the direction of the collected trajectory (e.g., the direction of vehicle travel). The Douglas-Pock algorithm, on the other hand, simplifies a curve into a series of points, reducing the number of points. This algorithm can filter out points with slight changes in the direction of the collected trajectory, thus identifying true turning points. Figure 5b The 502 in the image. Based on this, the positioning information corresponding to 502 can be obtained as the target positioning information. Therefore, the first image corresponding to the target positioning information can be selected from multiple first images and used as multiple second images. For example, the starting positioning point (the point corresponding to the positioning information at the starting position) in a collection trajectory can be connected to the ending positioning point (the point corresponding to the positioning information at the ending position) to obtain a straight line (e.g., ...). Figure 5bThe simplified path (dashed line) in the image allows us to determine the distance from each location point to this straight line. Location points with distances less than a preset distance can be filtered out, leaving the remaining locations. Further, the remaining locations can be iteratively processed using the line and distance parameters to filter them until no more locations are available for filtering, at which point the iteration terminates. The remaining locations at the end of the iteration can be used as target locations, and their corresponding location information can be used as target location information. Using the Douglas-Puk algorithm for turning point mining improves the accuracy of turning points, thereby enhancing the precision of multiple second images and ensuring the accuracy of subsequent intrinsic parameter calibration.

[0074] Figure 8 This diagram illustrates a method for calibrating initial intrinsic parameter information of an image acquisition device based on target point cloud data and initial pose information, according to an embodiment of this application. Figure 8 As shown, in one possible implementation, step S207 may include:

[0075] S801, determine the first pixel in a plurality of second images that matches a spatial point in the point cloud data;

[0076] S803, based on the initial pose information and initial intrinsic parameter information, reprojects the spatial points in the point cloud data to multiple second images to obtain multiple second pixel points;

[0077] S805 obtains multiple pixel deviation information based on multiple first pixel points and multiple second pixel points.

[0078] like Figure 9 As shown, taking the reprojection of a 3D spatial point P onto a second image as an example, by performing feature matching on pixels in different second images, it can be determined that p1 and p2 are projections of the same 3D spatial point P. Based on this, the 3D spatial point P can be reprojected onto the second image on the right, obtaining its position in the second image as follows: like Figure 9 As shown, p2 and There is a certain deviation distance e between them, which can be used as pixel deviation information. Here, p1 and p2 can be the first pixel points matched by the three-dimensional spatial point P in the two second images; The second pixel can be obtained by reprojecting a 3D spatial point P onto a second image to the right. It should be noted that the 3D spatial point P can be any 3D spatial point in the point cloud data, thus allowing us to obtain pixel deviation information corresponding to each spatial point, i.e., obtaining multiple pixel deviation information.

[0079] S807 calibrates the initial intrinsic parameter information based on the deviation information of multiple pixels to obtain the target intrinsic parameter information.

[0080] In one possible implementation, it's considered that the distance deviation of each 3D point is unlikely to be precisely zero or precise to a distance threshold. Therefore, the sum of the distance deviations is used to determine convergence. Thus, the cumulative deviation information and the cumulative deviation threshold of multiple pixels can be obtained; and the initial intrinsic parameter information can be updated based on the cumulative deviation information and the cumulative deviation threshold until the cumulative deviation information meets the cumulative deviation threshold. The initial intrinsic parameter information that meets the cumulative deviation threshold is then used as the target intrinsic parameter information. Here, "the cumulative deviation information meets the cumulative deviation threshold" can mean that the cumulative deviation information is less than the cumulative deviation threshold.

[0081] As an example, there can be n points in three-dimensional space, and the world coordinates of the i-th point in three-dimensional space can be represented by P. i Indicates; P i The second pixel coordinates of the pixel corresponding to the reprojection of the second image can be used Indicates; P i The first pixel point matched in the second image can be used with u i Based on the principle of projection, it can be represented by the following formula (1).

[0082]

[0083] Among them, S i ξ is the scale transformation parameter, which can be the ratio of the distance from the i-th 3D point to the focal point to the focal length; ξ^ is the Lie algebra corresponding to the pose of the image acquisition device; K is the intrinsic parameter information of the image acquisition device, which can be a 3*3 matrix, for example... f_x is the focal length in the horizontal direction, f_y is the focal length in the vertical direction, c_x is the position of the principal point of the second image in the horizontal direction, and c_y is the position of the principal point of the second image in the vertical direction.

[0084] Therefore, based on the following formula (2), the sum of the deviation information of multiple pixels can be determined as the cumulative deviation information ξ. * :

[0085]

[0086] The intrinsic parameter calibration process can be viewed as a solution process that makes the above formula (2) satisfy the cumulative deviation threshold. As an example, the above formula (2) can be solved using the bundle adjustment method. For example, gradient descent and other methods can be used for iterative solution to achieve iterative update of the initial intrinsic parameter information until the cumulative deviation information satisfies the cumulative deviation threshold. Thus, the initial intrinsic parameter information that satisfies the cumulative deviation threshold can be used as the target intrinsic parameter information.

[0087] The above formula (2) can be solved using the extended Kalman filter method, or nonlinear optimization can be performed using the gradient descent method until the cumulative deviation information meets the cumulative deviation threshold. Thus, the initial intrinsic parameter information that meets the cumulative deviation threshold can be used as the target intrinsic parameter information.

[0088] It should be noted that in the case of multiple acquisition trajectories, such as two acquisition trajectories, after 3D reconstruction, we can obtain point cloud data 1, initial pose 1, and initial intrinsic parameters 1, as well as point cloud data 2, initial pose 2, and initial intrinsic parameters 2. In this case, we can perform calibration processing on the initial intrinsic parameters 1 and 2 separately, i.e., perform nonlinear optimization, to obtain two optimized intrinsic parameter information. Then, we can select one of the two optimized intrinsic parameter information as the target intrinsic parameter information through median filtering or mean filtering. For example, we can choose the optimized intrinsic parameter information with the smallest cumulative deviation information as the target intrinsic parameter information.

[0089] Figure 10 This diagram illustrates a block diagram of an information processing apparatus according to an embodiment of this application. Figure 10 As shown, the device may include:

[0090] The acquisition module 1001 is used to acquire the acquisition trajectory information of the image acquisition device, which includes multiple first images and the positioning information of each of the multiple first images.

[0091] The filtering module 1003 is used to filter multiple second images from multiple first images based on the positioning information of each of the multiple first images, wherein the trajectory angle change information corresponding to the positioning information of the multiple second images is greater than the angle threshold.

[0092] The three-dimensional recovery module 1005 is used to acquire target point cloud data, initial pose information and initial intrinsic parameter information corresponding to the acquisition trajectory information based on the plurality of second images;

[0093] The target intrinsic parameter information acquisition module 1007 is used to calibrate the initial intrinsic parameter information based on the target point cloud data and the initial pose information to obtain the target intrinsic parameter information of the image acquisition device.

[0094] In one possible implementation, the filtering module 1003 described above may include:

[0095] The target location information acquisition module is used to filter the location information based on the Douglas-Puk algorithm to obtain the target location information;

[0096] A filtering unit is used to filter out the first image corresponding to the target positioning information from the plurality of first images, and use it as the plurality of second images.

[0097] In one possible implementation, the aforementioned three-dimensional restoration module 1005 may include:

[0098] The first feature extraction unit is used to extract first corner point feature information from the plurality of second images;

[0099] The first reconstruction unit is used to perform trajectory three-dimensional reconstruction processing based on the first corner feature information to obtain the target point cloud data, the initial pose information and the initial intrinsic parameter information.

[0100] In one possible implementation, the acquisition trajectory information comprises information on multiple acquisition trajectories; the aforementioned three-dimensional reconstruction module may include:

[0101] A trajectory segmentation unit is used to determine the second image corresponding to each acquisition trajectory from the plurality of second images;

[0102] The second feature extraction unit is used to perform corner feature extraction processing on the second image corresponding to each acquisition trajectory to obtain the second corner feature information corresponding to each acquisition trajectory.

[0103] The second reconstruction unit is used to perform three-dimensional reconstruction of the trajectory based on the feature information of the second corner point of each acquisition trajectory, so as to obtain multiple point cloud data, multiple pose information and multiple intrinsic parameter information corresponding to the multiple acquisition trajectories.

[0104] The information acquisition unit is used to obtain the target point cloud data, the initial pose information, and the initial intrinsic parameter information based on the multiple point cloud data, multiple pose information, and multiple intrinsic parameter information.

[0105] In one possible implementation, the target intrinsic parameter information acquisition module mentioned above may include:

[0106] The first pixel point determination unit is used to determine the first pixel point in the plurality of second images that matches the spatial point in the point cloud data;

[0107] The second pixel point determination unit is used to reproject the spatial points in the point cloud data to the multiple second images based on the initial pose information and the initial intrinsic parameter information to obtain multiple second pixel points.

[0108] The deviation information acquisition unit is used to obtain deviation information of multiple pixels based on the multiple first pixels and the multiple second pixels;

[0109] The calibration unit is used to calibrate the initial intrinsic parameter information based on the deviation information of the multiple pixels to obtain the target intrinsic parameter information.

[0110] In one possible implementation, the calibration unit may include:

[0111] The cumulative deviation acquisition subunit is used to acquire the cumulative deviation information and the cumulative deviation threshold of the multiple pixel deviation information;

[0112] The calibration subunit is used to update the initial intrinsic parameter information based on the cumulative deviation information and the cumulative deviation threshold until the cumulative deviation information meets the cumulative deviation threshold, and then uses the initial intrinsic parameter information that meets the cumulative deviation threshold as the target intrinsic parameter information.

[0113] In one possible implementation, the calibration subunit can also be used to update the initial intrinsic parameter information based on the cumulative deviation information and the cumulative deviation threshold using a bundle adjustment method or an extended Kalman filter.

[0114] Regarding the apparatus in the above embodiments, the specific manner in which each module and unit performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0115] On the other hand, this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the data recommendation method provided in the various optional implementations described above.

[0116] Figure 11 This diagram illustrates a block diagram of an electronic device for information processing according to an embodiment of this application. The electronic device may be a server, and its internal structure diagram may be as follows: Figure 11 As shown, this electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an information processing method.

[0117] Those skilled in the art will understand that Figure 11The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0118] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the information processing method as described in the embodiments of this application.

[0119] In an exemplary embodiment, a storage medium is also provided, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the information processing method of the embodiments of this application.

[0120] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the information processing method described in the embodiments of this application.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0122] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0123] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An information processing method, characterized in that, The method includes: Acquire acquisition trajectory information of an image acquisition device, wherein the acquisition trajectory information includes multiple first images and the positioning information of each of the multiple first images; Based on the positioning information of each of the multiple first images, multiple second images are selected from the multiple first images, and the trajectory angle change information corresponding to the positioning information of the multiple second images is greater than the angle threshold. Based on the multiple second images, acquire the target point cloud data, initial pose information, and initial intrinsic parameter information corresponding to the acquired trajectory information; Based on the target point cloud data and the initial pose information, the initial intrinsic parameter information is calibrated to obtain the target intrinsic parameter information of the image acquisition device.

2. The method according to claim 1, characterized in that, The step of filtering multiple second images from the multiple first images based on the positioning information of each of the multiple first images includes: The location information is filtered based on the Douglas-Puk algorithm to obtain the target location information; The first image corresponding to the target location information is selected from the plurality of first images and used as the plurality of second images.

3. The method according to claim 1, characterized in that, The step of acquiring target point cloud data, initial pose information, and initial intrinsic parameter information corresponding to the acquired trajectory information based on the plurality of second images includes: Extract first corner feature information from the plurality of second images; Based on the first corner feature information, a three-dimensional trajectory reconstruction process is performed to obtain the target point cloud data, the initial pose information, and the initial intrinsic parameter information.

4. The method according to claim 1, characterized in that, The acquired trajectory information comprises information from multiple acquired trajectories; the step of acquiring target point cloud data, initial pose information, and initial intrinsic parameter information corresponding to the acquired trajectory information based on the multiple second images includes: From the plurality of second images, determine the second image corresponding to each acquisition trajectory; Corner feature extraction is performed on the second image corresponding to each acquisition trajectory to obtain the second corner feature information corresponding to each acquisition trajectory. Based on the feature information of the second corner point of each acquisition trajectory, a three-dimensional reconstruction of the trajectory is performed to obtain multiple point cloud data, multiple pose information and multiple intrinsic parameter information corresponding to the multiple acquisition trajectories; Based on the multiple point cloud data, multiple pose information and multiple intrinsic parameter information, the target point cloud data, the initial pose information and the initial intrinsic parameter information are obtained.

5. The method according to any one of claims 1-4, characterized in that, Based on the target point cloud data and the initial pose information, the initial intrinsic parameter information is calibrated to obtain the target intrinsic parameter information of the image acquisition device, including: Determine the first pixel in the plurality of second images that matches the spatial point in the point cloud data; Based on the initial pose information and the initial intrinsic parameter information, the spatial points in the point cloud data are reprojected onto the multiple second images to obtain multiple second pixel points; Based on the first pixel and the plurality of second pixels, multiple pixel deviation information is obtained; Based on the deviation information of the multiple pixels, the initial intrinsic parameter information is calibrated to obtain the target intrinsic parameter information.

6. The method according to claim 5, characterized in that, The step of calibrating the initial intrinsic parameter information based on the multiple pixel deviation information to obtain the target intrinsic parameter information includes: Obtain the cumulative deviation information and the cumulative deviation threshold of the multiple pixel deviation information; Based on the cumulative deviation information and the cumulative deviation threshold, the initial intrinsic parameter information is updated until the cumulative deviation information meets the cumulative deviation threshold. The initial intrinsic parameter information that meets the cumulative deviation threshold is then used as the target intrinsic parameter information.

7. The method according to claim 6, characterized in that, The step of updating the initial intrinsic parameter information based on the cumulative deviation information and the cumulative deviation threshold includes: Based on the cumulative deviation information and the cumulative deviation threshold, the initial intrinsic parameter information is updated using the bundle adjustment method or the extended Kalman filter.

8. An information processing device, characterized in that, include: The acquisition module is used to acquire acquisition trajectory information of the image acquisition device, the acquisition trajectory information including multiple first images and the positioning information of each of the multiple first images; A filtering module is used to filter multiple second images from the multiple first images based on the positioning information of each of the multiple first images, wherein the trajectory angle change information corresponding to the positioning information of the multiple second images is greater than an angle threshold. The three-dimensional recovery module is used to acquire target point cloud data, initial pose information and initial intrinsic parameter information corresponding to the acquisition trajectory information based on the multiple second images; The target intrinsic parameter information acquisition module is used to calibrate the initial intrinsic parameter information based on the target point cloud data and the initial pose information to obtain the target intrinsic parameter information of the image acquisition device.

9. The apparatus according to claim 8, characterized in that, The filtering module includes: The target positioning information acquisition unit is used to filter the positioning information based on the Douglas-Puk algorithm to obtain the target positioning information; A filtering unit is used to filter out the first image corresponding to the target positioning information from the plurality of first images, and use it as the plurality of second images.

10. The apparatus according to claim 8, characterized in that, The three-dimensional recovery module includes: The first feature extraction unit is used to extract first corner point feature information from the plurality of second images; The first reconstruction unit is used to perform trajectory three-dimensional reconstruction processing based on the first corner feature information to obtain the target point cloud data, the initial pose information and the initial intrinsic parameter information.

11. The apparatus according to claim 8, characterized in that, The collected trajectory information includes information from multiple collected trajectories; The three-dimensional recovery module includes: A trajectory segmentation unit is used to determine the second image corresponding to each acquisition trajectory from the plurality of second images; The second feature extraction unit is used to perform corner feature extraction processing on the second image corresponding to each acquisition trajectory to obtain the second corner feature information corresponding to each acquisition trajectory. The second reconstruction unit is used to perform three-dimensional reconstruction of the trajectory based on the feature information of the second corner point of each acquisition trajectory, so as to obtain multiple point cloud data, multiple pose information and multiple intrinsic parameter information corresponding to the multiple acquisition trajectories. The information acquisition unit is used to obtain the target point cloud data, the initial pose information, and the initial intrinsic parameter information based on the multiple point cloud data, multiple pose information, and multiple intrinsic parameter information.

12. The apparatus according to any one of claims 8-11, characterized in that, The target intrinsic parameter information acquisition module includes: The first pixel point determination unit is used to determine the first pixel point in the plurality of second images that matches the spatial point in the point cloud data; The second pixel point determination unit is used to reproject the spatial points in the point cloud data to the multiple second images based on the initial pose information and the initial intrinsic parameter information to obtain multiple second pixel points. The deviation information acquisition unit is used to obtain deviation information of multiple pixels based on the first pixel and the multiple second pixels; The calibration unit is used to calibrate the initial intrinsic parameter information based on the deviation information of the multiple pixels to obtain the target intrinsic parameter information.

13. The apparatus according to claim 12, characterized in that, The calibration unit includes: The cumulative deviation acquisition subunit is used to acquire the cumulative deviation information and the cumulative deviation threshold of the multiple pixel deviation information; The calibration subunit is used to update the initial intrinsic parameter information based on the cumulative deviation information and the cumulative deviation threshold until the cumulative deviation information meets the cumulative deviation threshold, and then uses the initial intrinsic parameter information that meets the cumulative deviation threshold as the target intrinsic parameter information.

14. The apparatus according to claim 13, characterized in that, The calibration subunit is also used to update the initial intrinsic parameter information based on the cumulative deviation information and the cumulative deviation threshold, using the bundle adjustment method or the extended Kalman filter.

15. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions to implement the method according to any one of claims 1 to 7.

16. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

17. A computer program product, characterized in that, Includes computer instructions, which, when executed by a processor, cause the computer to perform the method as described in any one of claims 1 to 7.

Images