Camera parameter calibration method, electronic device and storage medium

Abstract

Embodiments of the present disclosure disclose a method for calibrating extrinsic parameters of a camera device, an electronic device and a storage medium, wherein the method comprises: performing semantic segmentation processing on a current frame image to obtain a segmented image comprising a plurality of first segmented regions; wherein the current frame image is obtained by image acquisition of a preset space at a current time by the camera device; determining whether the extrinsic parameter information of the camera device changes based on the plurality of first segmented regions and a plurality of second segmented regions corresponding to a pre-stored reference frame image; wherein the reference frame image is obtained by image acquisition of the preset space by the camera device; in response to the extrinsic parameter information of the camera device changing, obtaining a building informatization model corresponding to the preset space to determine a structured image; and determining the extrinsic parameter information of the camera device at the current time based on the segmented image and the structured image matching the position of the segmented image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to computer vision technology, and in particular to a method for calibrating extrinsic parameters of a camera device, an electronic device and a storage medium. BACKGROUND

[0002] With the rapid development of computer vision technology, it has been not satisfied with using a camera for monitoring, snapshotting and other relatively simple functions. More users prefer its application in non-contact three-dimensional size measurement. The so-called three-dimensional measurement is a broad sense of three-dimensional measurement, which not only includes the reconstruction and measurement of three-dimensional objects, but also includes the identification of the size and position of any two-dimensional plane in three-dimensional space. Therefore, the intrinsic and extrinsic parameters of the camera need to be calibrated, and camera calibration has been gradually applied to building, decoration, medical treatment, food, abrasive tool production, semiconductor production and many other detection systems. The intrinsic parameters of the camera can be obtained based on the data provided by the manufacturer, and the extrinsic parameter calibration in the prior art is semi-automatic calibration, which needs manual participation. SUMMARY

[0003] In order to solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide a method for calibrating extrinsic parameters of a camera device, an electronic device and a storage medium.

[0004] According to an aspect of an embodiment of the present disclosure, a method for calibrating extrinsic parameters of a camera device is provided, comprising:

[0005] performing semantic segmentation processing on a current frame image to obtain a segmented image comprising a plurality of first segmented regions; wherein the current frame image is obtained by image acquisition of a preset space at a current time by a camera device;

[0006] determining whether the extrinsic parameter information of the camera device changes based on the plurality of first segmented regions and a plurality of second segmented regions corresponding to a pre-stored reference frame image; wherein the reference frame image is obtained by image acquisition of the preset space by the camera device;

[0007] in response to the extrinsic parameter information of the camera device changing, obtaining a building informatization model corresponding to the preset space to determine a structured image;

[0008] determining the extrinsic parameter information of the camera device at the current time based on the segmented image and the structured image matching the position of the segmented image.

[0009] Optionally, the obtaining the building informatization model corresponding to the preset space to determine the structured image comprises:

[0010] obtaining a building informatization model corresponding to the preset space;

[0011] Based on the assumed position of the camera device in the preset space, a projection plane is determined, and the building information model is projected onto the projection plane to obtain the structured image.

[0012] Optionally, obtaining the structured image from the building information model corresponding to the preset space further includes:

[0013] The building information model is simplified to obtain a simplified building information model;

[0014] The step of determining a projection plane based on the assumed position of the camera device in the preset space, and projecting the building information model onto the projection plane to obtain the structured image includes:

[0015] Based on the assumed position of the camera device in the preset space, a projection plane is determined, and the simplified building information model is projected onto the projection plane to obtain the structured image.

[0016] Optionally, after performing semantic segmentation on the current frame image to obtain a segmented image including multiple first segmentation regions, the method further includes:

[0017] Feature points are extracted from the segmented image to determine at least three first feature points in the segmented image; wherein each first feature point corresponds to a feature point category;

[0018] The step of determining the extrinsic information of the camera device at the current moment based on the segmented image and the structured image matching the position of the segmented image includes:

[0019] Based on the segmented image and the structured image that matches the position of the segmented image, determine the three-dimensional information corresponding to at least three first feature points in the segmented image;

[0020] Based on the three-dimensional information corresponding to the at least three first feature points and the coordinate information of the at least three first feature points in the segmented image, the extrinsic parameter information of the camera device at the current moment is determined.

[0021] Optionally, before obtaining the building information model corresponding to the preset space to determine the structured image, the method further includes:

[0022] Based on the at least three first feature points, at least three second feature points are determined in the building information model; wherein each second feature point corresponds to a feature point category;

[0023] The step of determining the three-dimensional information corresponding to at least three first feature points in the segmented image based on the segmented image and the structured image matching the position of the segmented image includes:

[0024] determining a feature point distance between the at least three second feature points of the same feature point category as each of the at least three first feature points;

[0025] determining at least three second feature points corresponding to the at least three first feature points based on the feature point distance; wherein each of the first feature points corresponds to one of the second feature points;

[0026] determining three-dimensional information corresponding to the at least three first feature points based on three-dimensional information corresponding to the second feature points corresponding to the first feature points.

[0027] Optionally, before determining the extrinsic information of the camera device at the current time based on the segmented image and the structured image matching the position of the segmented image, the method further comprises:

[0028] performing angle adjustment on the structured image based on the plurality of first segmented regions in the segmented image and a plurality of third segmented regions in the structured image;

[0029] determining the structured image matching the position of the segmented image based on the structured image after the angle adjustment.

[0030] Optionally, the performing angle adjustment on the structured image based on the plurality of first segmented regions in the segmented image and the plurality of third segmented regions in the structured image comprises:

[0031] determining a center position of at least one of the plurality of first segmented regions to obtain a rotation center;

[0032] performing at least one rotation on the structured image based on the rotation center to obtain at least one rotated structured image;

[0033] determining an intersection-over-union between the at least one third segmented region and the at least one first segmented region in each of the rotated structured images to obtain at least one intersection-over-union;

[0034] determining one of the rotated structured images as the structured image after the angle adjustment based on the intersection-over-union.

[0035] Optionally, the method further comprises:

[0036] in response to no change in the extrinsic information of the camera device, determining the extrinsic information of the camera device at the current time based on the extrinsic information corresponding to the reference frame image.

[0037] Optionally, the determining whether the extrinsic parameter information of the camera device changes based on the plurality of first segmentation regions and a plurality of second segmentation regions corresponding to a pre-stored reference frame image comprises:

[0038] determining a plurality of region pairs based on the correspondence between each of the first segmentation regions and the second segmentation regions; wherein the plurality of first segmentation regions and the plurality of second segmentation regions correspond one-to-one; and each of the region pairs comprises the first segmentation region and the second segmentation region that have a correspondence;

[0039] determining an intersection-over-union between the first segmentation region and the second segmentation region in each of the plurality of region pairs, to obtain a plurality of intersection-overs-union;

[0040] determining whether the extrinsic parameter information of the camera device changes based on the plurality of intersection-overs-union.

[0041] Optionally, before the determining whether the extrinsic parameter information of the camera device changes based on the plurality of first segmentation regions and a plurality of second segmentation regions corresponding to a pre-stored reference frame image, the method further comprises:

[0042] acquiring the reference frame image by performing image acquisition on the preset space by the camera device at a previous time point before the current time point, and storing the reference frame image; or

[0043] when the extrinsic parameter information of the camera device is updated before the current time point, acquiring the reference frame image by performing image acquisition on the preset space by the camera device, and storing the reference frame image.

[0044] According to another aspect of the embodiments of the present disclosure, a device for calibrating extrinsic parameters of a camera device is provided, comprising:

[0045] an image segmentation module configured to perform semantic segmentation processing on a current frame image to obtain a segmentation image comprising a plurality of first segmentation regions; wherein the current frame image is an image acquired by performing image acquisition on a preset space by a camera device at a current time point;

[0046] an extrinsic parameter change determination module configured to determine whether the extrinsic parameter information of the camera device changes based on the plurality of first segmentation regions and a plurality of second segmentation regions corresponding to a pre-stored reference frame image; wherein the reference frame image is an image acquired by performing image acquisition on the preset space by the camera device;

[0047] an information acquisition module configured to acquire a building informatization model corresponding to the preset space to determine a structured image in response to the extrinsic parameter information of the camera device changing;

[0048] An external parameter determination module is configured to determine external parameter information of the camera device at the current time based on the segmented image and the structured image matched with the position of the segmented image.

[0049] Optionally, the information acquisition module is specifically configured to acquire a building information model corresponding to the preset space; determine a projection plane based on the assumed position of the camera device in the preset space, project the building information model into the projection plane to obtain the structured image.

[0050] Optionally, the information acquisition module is further configured to acquire a building information model corresponding to the preset space; perform simplification processing on the building information model to obtain a simplified building information model; determine a projection plane based on the assumed position of the camera device in the preset space, project the simplified building information model into the projection plane to obtain the structured image.

[0051] Optionally, the apparatus further comprises:

[0052] A feature point extraction module is configured to perform feature point extraction on the segmented image to determine at least three first feature points in the segmented image; wherein each first feature point corresponds to a feature point category;

[0053] The external parameter determination module comprises:

[0054] An image three-dimensional information determination unit is configured to determine three-dimensional information corresponding to the at least three first feature points in the segmented image based on the segmented image and the structured image matched with the position of the segmented image;

[0055] A device external parameter unit is configured to determine external parameter information of the camera device at the current time based on the three-dimensional information corresponding to the at least three first feature points and coordinate information of the at least three first feature points in the segmented image.

[0056] Optionally, the apparatus further comprises:

[0057] A model feature point module is configured to determine at least three second feature points in the building information model based on the at least three first feature points; wherein each second feature point corresponds to a feature point category;

[0058] The image three-dimensional information determination unit is specifically configured to determine a feature point distance between each first feature point and the at least three second feature points of the same feature point category in the at least three first feature points; determine at least three second feature points corresponding to the at least three first feature points based on the feature point distance; each first feature point corresponds to a second feature point; and determine three-dimensional information corresponding to the at least three first feature points based on three-dimensional information corresponding to the second feature points corresponding to the first feature points.

[0059] Optionally, the device further comprises:

[0060] The angle adjustment module is configured to perform angle adjustment on the structured image based on the plurality of first segmentation regions in the segmentation image and a plurality of third segmentation regions in the structured image.

[0061] The position matching module is configured to perform position matching between the structured image and the segmentation image based on the angle-adjusted structured image.

[0062] Optionally, the angle adjustment module is specifically configured to determine a center position of at least one first segmentation region in the plurality of first segmentation regions to obtain a rotation center; perform at least one rotation on the structured image based on the rotation center to obtain at least one rotated structured image; determine an intersection-over-union between the at least one third segmentation region and the at least one first segmentation region in each rotated structured image to obtain at least one intersection-over-union; and determine one rotated structured image as the angle-adjusted structured image based on the intersection-over-union.

[0063] Optionally, the device further comprises:

[0064] The extrinsic parameter multiplexing module is configured to, in response to the extrinsic parameter information of the camera device not changing, use the extrinsic parameter information corresponding to the reference frame image as the extrinsic parameter information corresponding to the camera device at the current time.

[0065] Optionally, the extrinsic parameter change determination module is specifically configured to determine a plurality of region pairs based on the correspondence between each first segmentation region and the second segmentation region; the plurality of first segmentation regions and the plurality of second segmentation regions correspond one-to-one; each region pair includes the first segmentation region and the second segmentation region that have a correspondence; determine an intersection-over-union between the first segmentation region and the second segmentation region in each region pair in the plurality of region pairs to obtain a plurality of intersection-over-unions; and determine whether the extrinsic parameter information of the camera device changes based on the plurality of intersection-over-unions.

[0066] Optionally, the device further comprises:

[0067] The reference frame determination module is configured to: at a last time point before the current time point, perform image acquisition on the preset space by the camera device to obtain the reference frame image and store the reference frame image; or, when the extrinsic parameter information of the camera device is updated before the current time point, perform image acquisition on the preset space by the camera device to obtain the reference frame image and store the reference frame image.

[0068] According to still another aspect of the embodiments of the present disclosure, an electronic device is provided, comprising:

[0069] The memory is configured to store a computer program product.

[0070] The processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, the extrinsic parameter calibration method of the camera device in any of the above embodiments is implemented.

[0071] According to still another aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores computer program instructions, and when the computer program instructions are executed by a processor, the extrinsic parameter calibration method of the camera device in any of the above embodiments is implemented.

[0072] According to still another aspect of the embodiments of the present disclosure, a computer program product is provided, comprising computer program instructions, and when the computer program instructions are executed by a processor, the extrinsic parameter calibration method of the camera device in any of the above embodiments is implemented.

[0073] The extrinsic parameter calibration method of the camera device, the electronic device and the storage medium provided by the above embodiments of the present disclosure comprise: performing semantic segmentation processing on a current frame image to obtain a segmented image comprising a plurality of first segmented regions; wherein the current frame image is an image obtained by performing image acquisition on a preset space by a camera device at a current time point; determining whether the extrinsic parameter information of the camera device changes based on the plurality of first segmented regions and a plurality of second segmented regions corresponding to a pre-stored reference frame image; wherein the reference frame image is an image obtained by performing image acquisition on the preset space by the camera device; in response to the extrinsic parameter information of the camera device changing, obtaining a building informationization model corresponding to the preset space to determine a structured image; and determining the extrinsic parameter information of the camera device at the current time point based on the segmented image and the structured image matching the position of the segmented image. In the embodiments of the present disclosure, the second segmented regions in the reference frame image are first collected to determine whether the camera device corresponding to the current frame image changes the extrinsic parameter, and when it is determined that the extrinsic parameter information of the camera device changes, the extrinsic parameter of the camera device corresponding to the current frame image is calibrated in combination with the building informationization model, thereby realizing real-time extrinsic parameter calibration of the camera device.

[0074] The technical solutions of the present disclosure are described in further detail below with reference to the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0075] The accompanying drawings, which form a part of the specification, illustrate the embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0076] The present disclosure can be understood more clearly with reference to the following detailed description when read in conjunction with the accompanying drawings, in which:

[0077] Figure 1 is a flowchart of a method for calibrating extrinsic parameters of a camera device provided by an exemplary embodiment of the present disclosure;

[0078] Figure 2-1 is a flowchart of step 106 in the embodiment shown in Figure 1

[0079] Figure 2-2 is a structured image diagram obtained based on a building information model in an optional example of the present disclosure;

[0080] Figure 2-3 is a structured image diagram obtained by simplifying the 2D image shown in Figure 2-2

[0081] Figure 3-1 is a current frame image diagram provided by an exemplary embodiment of the present disclosure;

[0082] Figure 3-2 is a segmented image diagram obtained by segmenting the current frame image shown in Figure 3-1

[0083] Figure 3-3 is a position overlap diagram of the structured image shown in Figure 2-3 Figure 3-2

[0084] Figure 4 is a structural diagram of a device for calibrating extrinsic parameters of a camera device provided by an exemplary embodiment of the present disclosure;

[0085] Figure 5 Fig. 1 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0086] The example embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. It is obvious that the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited by the example embodiments described herein.

[0087] ​​​​​It should be noted that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0088] Those skilled in the art can understand that the terms "first", "second", and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they represent a logical sequence between them.

[0089] It should also be understood that in the embodiments of the present disclosure, "multiple" can mean two or more, and "at least one" can mean one, two, or more.

[0090] It should also be understood that for any component, data, or structure mentioned in the embodiments of the present disclosure, unless specifically limited or the context gives the opposite indication, it can be understood as one or more in general.

[0091] In addition, the term "and / or" in the present disclosure is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects. The data referred to in the present disclosure can include unstructured data such as text, images, and videos, and can also be structured data.

[0092] It should also be understood that the description of the embodiments of the present disclosure focuses on the differences between the embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0093] At the same time, it should be understood that in order to facilitate the description, the size of each part shown in the drawings is not drawn according to the actual proportional relationship.

[0094] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way limiting on the disclosure or its application or uses.

[0095] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0096] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0097] The disclosed embodiments can be applied to terminal devices, computer systems, servers, and other electronic devices, which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above systems, and the like.

[0098] Terminal devices, computer systems, servers, and other electronic devices can be described in the general context of computer system-executable instructions, such as program modules, being executed by the computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like, which perform particular tasks or implement particular abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, in which tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in local or remote computer system storage media including storage devices.

[0099] The inventors found that the existing camera extrinsic calibration is semi-automatic and requires human intervention. However, with the rapid development of camera application business, it is necessary to speed up the business response, free the manpower, and save the labor cost.

[0100] The semi-automatic calibration of the camera extrinsic parameter is solved by manually selecting some point pairs in the 2D image and the 3D world online. For example, in the decoration field, the corner points of the house in the 2D image and the corresponding points in the 3D world can be manually selected to form at least four groups of point pairs for camera extrinsic calibration. However, the camera is not always fixed during construction. Sometimes, the workers will take down the camera for top construction, and then install it again after the construction is completed. The angle of the camera after reinstallation will be different from before, and the original extrinsic parameter will no longer be applicable. If the original extrinsic parameter is still used, it will cause the 3D space generated from the 2D image captured by the camera to be disordered, which greatly affects the visual effect. The camera position needs to be responded in a timely manner, and the manual response speed is not timely. If the camera extrinsic parameter change is not handled in a timely manner, it will seriously affect the user experience.

[0101] Exemplary method

[0102] Figure 1 FIG. 1 is a flowchart of a camera extrinsic calibration method according to an example embodiment of the present disclosure. The method can be applied to an electronic device, such as a terminal device, a computer system, a server, or the like.Figure 1 As shown, the method comprises the following steps:

[0103] In step 102, a semantic segmentation process is performed on the current frame image to obtain a segmented image comprising a plurality of first segmented regions.

[0104] The current frame image is an image obtained by image acquisition of the preset space at the current time by the camera device.

[0105] Optionally, the preset space can be a space scene such as a building or a vehicle. The semantic segmentation can be implemented based on a deep learning model. For example, the current frame image is segmented by a segmentation model. In an optional example, the preset space is a building, and the segmentation types can be divided into three categories: surface, line, and point. The surface includes wall surface, floor, door, window, door and window, etc. The line includes wall-wall line, wall-floor line, etc. The point includes house corner point, door corner point, window corner point, door and window corner point, etc. For example, the segmentation types include 11 subcategories. In order to achieve better segmentation effect, the segmentation model needs to be trained. The actual data at each house type and each construction stage is collected as training data, and data labeling of 11 categories is performed. After the training data is prepared, the segmentation model is trained. Optionally, the deep learning model can use HRnet-32-v2 followed by softmax for classification, using LovaszSoftmax Loss and focal cross entropy loss. The embodiments of the present application do not limit the deep learning model, and those skilled in the art can understand that other models suitable for image semantic segmentation can also be used. When predicting based on the trained segmentation model, for point extraction, the highest confidence point in the 5*5 pixel neighborhood can be taken as the segmentation result of the point.

[0106] In step 104, based on the plurality of first segmented regions and a plurality of second segmented regions corresponding to the pre-stored reference frame image, it is determined whether the extrinsic information of the camera device has changed.

[0107] The reference frame image is an image obtained by image acquisition of the preset space by the camera device and stored.

[0108] Optionally, the reference frame image can be segmented based on the same processing method as the current frame to obtain a plurality of second segmented regions.

[0109] In step 106, in response to the change of the extrinsic information of the camera device, an architectural informationization model corresponding to the preset space is obtained to determine the structured image.

[0110] Building Information Modeling (BIM) integrates all relevant information of a building project through three-dimensional digital technology. It provides a detailed representation of the entire project lifecycle and is a direct application of digital visualization technology in building engineering. It also provides early warnings and analysis of various issues before the project, enabling all stakeholders to understand and respond to them, while providing a solid foundation for collaborative work. The presentation format can be CAD floor plans, with spatial information obtained by professionals. This spatial information may include, but is not limited to, wall information, floor information, corner information, door and window information, etc., and each piece of information includes 3D coordinates.

[0111] Optionally, the structured image can be a structured 2D image (structure_image) obtained by projecting 3D information from the building information model onto a 2D plane.

[0112] Step 108: Based on the segmented image and the structured image that matches the position of the segmented image, determine the extrinsic information of the camera device at the current moment.

[0113] Optionally, the segmented image contains 2D information, while the structured image, due to its corresponding building information model, contains corresponding 3D information. Based on the positional matching relationship between the two, multiple sets of corresponding 2D and 3D feature points can be determined. Based on the corresponding 2D and 3D feature points, the extrinsic information of the camera device can be determined using existing extrinsic inference methods.

[0114] Based on the above embodiments of this disclosure, a method for extrinsic parameter calibration of a camera device is provided. Semantic segmentation processing is performed on the current frame image to obtain a segmented image including multiple first segmentation regions. The current frame image is obtained by the camera device capturing an image of a preset space at the current time. Based on the multiple first segmentation regions and multiple second segmentation regions corresponding to a pre-stored reference frame image, it is determined whether the extrinsic parameter information of the camera device has changed. The reference frame image is obtained by the camera device capturing an image of the preset space at a previous time. In response to a change in the extrinsic parameter information of the camera device, a building information model (BIM) corresponding to the preset space is obtained to determine a structured image. Based on the segmented image and the structured image matching the position of the segmented image, the extrinsic parameter information of the camera device at the current time is determined. In this embodiment, the second segmentation regions in the reference frame image are first used to determine whether the extrinsic parameters of the camera device corresponding to the current frame image have changed. When it is determined that the extrinsic parameter information of the camera device has changed, the extrinsic parameters of the camera device corresponding to the current frame image are calibrated in conjunction with the BIM model, thus achieving real-time extrinsic parameter calibration of the camera device.

[0115] like Figure 2-1 As shown above, in the above Figure 1Based on the embodiment shown, step 106 can include the following steps:

[0116] Step 1061, obtain the building informationization model corresponding to the preset space.

[0117] Step 1062, determine the projection plane based on the assumed position of the camera equipment in the preset space, project the building informationization model into the projection plane, and obtain a structured image.

[0118] In this embodiment, the position and orientation of the camera equipment can be assumed, the imaging of the 3D room in 2D (3D projection 2D plane) can be simulated, and a structured 2D image can be formed according to the information of the walls, floors, doors, windows, etc. For example, as shown in Figure 2-2 In an example, a structured image is obtained based on the building informationization model, and optionally, the room center point information is obtained according to the BIM information, and the extrinsic parameter data is assumed (for example, the extrinsic parameter position can be assumed to be the geometric center of the ceiling, the height and geometric center of the ceiling can be determined based on the BIM information, and the lens is vertically downward). According to the 3D information of the corner points of the walls, floors, doors, windows, etc. in the BIM, the camera intrinsic parameter (known) and the assumed camera extrinsic parameter are used, the 3D world coordinates are one-to-one corresponding to the 2D image coordinates, the position of each 3D coordinate in the 2D image is obtained, that is, the conversion of 3d_bim_points to 2d_bim_points. According to the connection in the BIM, these points are connected to form a 2D structured image.

[0119] Optionally, between step 1061 and step 1062, the following steps can also be included:

[0120] simplifying the building informationization model to obtain a simplified building informationization model;

[0121] Correspondingly, step 1062 includes: determining the projection plane based on the assumed position of the camera equipment in the preset space, projecting the simplified building informationization model into the projection plane, and obtaining a structured image.

[0122] In some optional examples, since the camera cannot see the blocked area, the building informationization model is simplified before obtaining the structured image. Any one or more of the combination of disassembly, deletion, supplement, and combination can be used to realize the simplification, for example, in an optional example, the 2D image shown in Figure 2-2 is simplified to obtain the structured image as shown in Figure 2-3 The purpose of simplification is to regularize the shape, and the regularized structured image can be matched with the segmented image more quickly.

[0123] Optionally, in the above Figure 1Based on the embodiment shown, after step 102, it can also include:

[0124] Feature point extraction is performed on the segmented image to determine at least three first feature points in the segmented image.

[0125] Each first feature point corresponds to a feature point category.

[0126] Optionally, in the embodiment, when the current frame image is segmented, multiple categories of feature points can be segmented, such as house corner points, door corner points, window corner points, etc.

[0127] In this embodiment, step 108 can include:

[0128] Based on the segmented image and the structured image matching the position of the segmented image, three-dimensional information corresponding to the at least three first feature points in the segmented image is determined.

[0129] Based on the three-dimensional information corresponding to the at least three first feature points and the coordinate information of the at least three first feature points in the segmented image, the extrinsic information of the camera device at the current time is determined.

[0130] In this embodiment, the three-dimensional information corresponding to the at least three first feature points can be determined through the structured image matching the position of the segmented image, and the extrinsic information of the camera device at the current time can be determined through the two-dimensional information and the three-dimensional information corresponding to the at least three first feature points.

[0131] Optionally, before step 106, it can also include:

[0132] Based on the at least three feature point categories, at least three second feature points in the building information model are determined.

[0133] In this embodiment, each first feature point corresponds to a feature point category. Optionally, the feature point category can include a house corner point, a door corner point, a window corner point, etc. The building information model can include the category of each feature point, and based on these categories, at least three second feature points can be determined.

[0134] Based on the segmented image and the structured image matching the position of the segmented image, the three-dimensional information corresponding to the at least three first feature points in the segmented image can include:

[0135] Determine the feature point distance between the at least three second feature points whose feature point category is the same as each first feature point in the at least three first feature points.

[0136] Based on the feature point distance, at least three second feature points corresponding to the at least three first feature points are determined.

[0137] Each first feature point corresponds to a second feature point.

[0138] determine the three-dimensional information corresponding to the at least three first feature points based on the three-dimensional information corresponding to the second feature points corresponding to the first feature points.

[0139] In this embodiment, the second feature point corresponding to the minimum distance can be determined as the second feature point corresponding to the first feature point by calculating the distance (for example, the Euclidean distance, the cosine distance, etc.) between each first feature point and all second feature points of the same feature point category as the first feature point. Alternatively, the second feature point corresponding to the minimum distance can be determined as the second feature point corresponding to the first feature point by calculating the distance between the first feature point and at least one second feature point (of the same feature point category as the first feature point) close to the first feature point in the position-matched structured image, so as to achieve the best distance matching under the same category. In addition, if the minimum distance is greater than a preset threshold, the first feature point is deleted, and the feature point is not used as a feature point for determining the camera external parameter, so as to remove the feature points that do not meet the requirements by screening, and improve the accuracy of the external parameter information determined based on the feature points.

[0140] In some optional embodiments, before step 108, the method further includes:

[0141] adjusting the angle of the structured image based on the plurality of first segmentation regions in the segmented image and the plurality of third segmentation regions in the structured image;

[0142] determining the structured image that is position-matched with the segmented image based on the structured image after the angle adjustment.

[0143] In this embodiment, since the structured image is determined based on the building information model, the structured image has segmentation results corresponding to different points, surfaces and lines, that is, the plurality of third segmentation regions in the structured image can be directly obtained based on the building information model. In order to make the segmented image and the structured image coincide as much as possible and match the feature points, the structured image needs to be rotated, for example, in an optional example, the current frame image as shown in FIG. 11 is segmented to obtain the segmented image as shown in FIG. 12, and the segmented image is position-matched with the structured image as shown in FIG. 13 to obtain the position-matched structured image as shown in FIG. 14. Figure 3-1 Figure 3-2 Figure 2-3 Figure 3-3 Figure 2-3 Figure 3-2 Optionally, the angle adjustment of the structured image based on the plurality of first segmentation regions in the segmented image and the plurality of third segmentation regions in the structured image can include:

[0144] ​​​​​determine a center position of at least one first segmentation region in the plurality of first segmentation regions, to obtain a rotation center;

[0145] perform at least one rotation on the structured image based on the rotation center, to obtain at least one rotated structured image;

[0146] determine an intersection-over-union between at least one third segmentation region in each rotated structured image and the at least one first segmentation region, to obtain at least one intersection-over-union;

[0147] determine one rotated structured image as an angle-adjusted structured image based on the intersection-over-union.

[0148] In this embodiment, the center position of the segmentation image can be determined based on the at least one segmentation region, and the structured image is rotated with the center position as the rotation center. The structured image can be rotated multiple times, each time by a preset angle, and the intersection-over-union (IOU) between each third segmentation region in the rotated structured image and the corresponding first segmentation region is calculated each time. Alternatively, the intersection-over-union (IOU) between each third segmentation region in the multiple rotated structured images and the corresponding first segmentation region is calculated after multiple rotations, the angle of rotation is determined based on the maximum value of the intersection-over-union, and an angle-adjusted structured image is obtained. The structured image is in the best position matching with the segmentation image, and on this basis, the feature point pair can be determined more quickly, and the efficiency of the external parameter calibration is improved.

[0149] In some optional examples, step 104 can further include:

[0150] determine a plurality of region pairs based on the correspondence between each first segmentation region and the second segmentation region; wherein the plurality of first segmentation regions and the plurality of second segmentation regions are in one-to-one correspondence; each region pair includes a first segmentation region and a second segmentation region that have a correspondence relationship;

[0151] determine an intersection-over-union between the first segmentation region and the second segmentation region in each region pair in the plurality of region pairs, to obtain a plurality of intersection-over-union;

[0152] determine whether the external parameter information of the camera device has changed based on the plurality of intersection-over-union.

[0153] Optionally, it is determined whether the extrinsic parameter information of the camera device changes based on an intersection-over-union between each first segmentation region and its corresponding second segmentation region; for example, when the intersection-over-union is greater than or equal to a preset intersection-over-union (the specific value can be set according to the actual application scenario), it indicates that the camera device does not change in pose, and it is not necessary to recalculate the extrinsic parameter information of the camera device, and the extrinsic parameter information corresponding to the reference frame image can be directly reused. Optionally, after step 104, the method can further include: in response to the extrinsic parameter information of the camera device not changing, taking the extrinsic parameter information corresponding to the reference frame image as the extrinsic parameter information corresponding to the camera device at the current time. When the intersection-over-union is less than the preset intersection-over-union, it indicates that the pose of the camera device has changed, and it is necessary to recalculate the extrinsic parameter information of the current frame. In this embodiment, by judging and reusing the extrinsic parameter information of the reference frame image, it is realized that the calculation is not necessary when the pose of the camera device does not change, a large amount of calculation resources is saved, and the efficiency of extrinsic parameter calibration is improved.

[0154] In some optional embodiments, before step 104, the method can further include:

[0155] At a last time point before the current time point, the camera device is used to collect images of the preset space to obtain a reference frame image and store the reference frame image; or, when the extrinsic parameter information of the camera device is updated before the current time point, the camera device is used to collect images of the preset space to obtain a reference frame image and store the reference frame image.

[0156] In this embodiment, the reference frame image and the current frame image can be time-continuous or time-discontinuous. Time-continuous means that when multiple frames of images are collected at consecutive time points, two adjacent image frames are the reference frame image and the current frame image, and the reference frame image is collected at a time point one moment before the current frame image. Time-discontinuous means that the reference frame image can be collected at any time point before the current frame image, for example, several minutes or several hours or even several days ago. In addition, in this embodiment, the reference frame can also be obtained by collecting images of the preset space based on the camera device after updating the extrinsic parameter information when the extrinsic parameter information of the camera device is updated last time. In this embodiment, no matter at which time point the reference frame image is obtained, the reference frame image is stored to serve as a reference benchmark when it is determined whether the extrinsic parameter information changes.

[0157] Any of the extrinsic parameter calibration methods of the camera device provided in the embodiments of the present disclosure can be executed by any appropriate device with data processing capability, including but not limited to: terminal devices and servers, etc. Alternatively, any of the extrinsic parameter calibration methods of the camera device provided in the embodiments of the present disclosure can be executed by a processor, such as a processor executing any of the extrinsic parameter calibration methods of the camera device mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. Details are not described herein.

[0158] Exemplary apparatus

[0159] Figure 4 is a structural schematic diagram of an external parameter calibration device of a camera device provided by an example embodiment of the present disclosure. As shown in the figure, the device provided by the embodiment includes: Figure 4

[0160] an image segmentation module 41, configured to perform semantic segmentation processing on a current frame image to obtain a segmented image including a plurality of first segmentation regions.

[0161] The current frame image is an image obtained by image acquisition of a preset space by the camera device at a current time.

[0162] an external parameter change determination module 42, configured to determine whether the external parameter information of the camera device changes based on the plurality of first segmentation regions and a plurality of second segmentation regions corresponding to a reference frame image pre-stored.

[0163] The reference frame image is an image obtained by image acquisition of the preset space by the camera device.

[0164] an information acquisition module 43, configured to acquire a building informationization model corresponding to the preset space to determine a structured image in response to the external parameter information of the camera device changing.

[0165] an external parameter determination module 44, configured to determine the external parameter information of the camera device at the current time based on the segmented image and a structured image matching the position of the segmented image.

[0166] Based on the external parameter calibration device of the camera device provided by the above-mentioned embodiment of the present disclosure, in the embodiment of the present disclosure, the second segmentation regions in the reference frame image are first collected to determine whether the camera device corresponding to the current frame image changes in external parameter, and when it is determined that the external parameter information of the camera device changes, the external parameter of the camera device corresponding to the current frame image is calibrated in combination with the building informationization model, thereby realizing real-time external parameter calibration of the camera device.

[0167] Optionally, the information acquisition module 43 is specifically configured to acquire the building informationization model corresponding to the preset space; determine a projection plane based on the assumed position of the camera device in the preset space, project the building informationization model into the projection plane to obtain the structured image.

[0168] Optionally, the information acquisition module 43 is further configured to acquire the building informationization model corresponding to the preset space; perform simplification processing on the building informationization model to obtain a simplified building informationization model; determine a projection plane based on the assumed position of the camera device in the preset space, project the simplified building informationization model into the projection plane to obtain the structured image.

[0169] ​Optionally, the apparatus provided in the embodiment can further include:

[0170] The feature point extraction module is configured to perform feature point extraction on the segmented image to determine at least three first feature points in the segmented image, wherein each first feature point corresponds to a feature point category.

[0171] The extrinsic parameter determination module 44 includes:

[0172] The image three-dimensional information determination unit is configured to determine three-dimensional information corresponding to the at least three first feature points in the segmented image based on the segmented image and the structured image matched in position with the segmented image.

[0173] The device extrinsic parameter unit is configured to determine extrinsic parameter information of the camera device at the current time based on the three-dimensional information corresponding to the at least three first feature points and coordinate information of the at least three first feature points in the segmented image.

[0174] Optionally, the apparatus provided in the embodiment can further include:

[0175] The model feature point module is configured to determine at least three second feature points in the building information model based on the at least three feature point categories, wherein each second feature point corresponds to a feature point category.

[0176] The image three-dimensional information determination unit is specifically configured to determine a feature point distance between at least three second feature points corresponding to each first feature point in the at least three first feature points and the feature point category, and determine at least three second feature points corresponding to the at least three first feature points based on the feature point distance.

[0177] Each first feature point corresponds to a second feature point, and the three-dimensional information corresponding to the at least three first feature points is determined based on three-dimensional information corresponding to the second feature point corresponding to the first feature point.

[0178] Optionally, the apparatus provided in the embodiment can further include:

[0179] The angle adjustment module is configured to perform angle adjustment on the structured image based on the plurality of first segmentation regions in the segmented image and the plurality of third segmentation regions in the structured image.

[0180] The position matching module is configured to determine the structured image matched in position with the segmented image based on the structured image after the angle adjustment.

[0181] Optionally, the angle adjustment module is specifically used to determine the center position of at least one of the multiple first segmentation regions to obtain a rotation center; to rotate the structured image at least once based on the rotation center to obtain at least one rotated structured image; to determine the intersection-union ratio (IUR) between at least one third segmentation region and at least one first segmentation region in each rotated structured image to obtain at least one IUR; and to determine a rotated structured image as the angle-adjusted structured image based on the IUR.

[0182] Optionally, the apparatus provided in this embodiment may further include:

[0183] The extrinsic parameter multiplexing module is used to respond to situations where the extrinsic parameter information of the camera device has not changed, and to use the extrinsic parameter information corresponding to the reference frame image as the extrinsic parameter information of the camera device at the current moment.

[0184] Optionally, the extrinsic parameter change determination module 42 is specifically used to determine multiple pairs of regions based on the correspondence between each first segmentation region and the second segmentation region; wherein, multiple first segmentation regions correspond one-to-one with multiple second segmentation regions; each pair of regions includes first segmentation regions and second segmentation regions that have a corresponding relationship; determine the intersection-union ratio (IUGR) between the first segmentation region and the second segmentation region in each pair of regions to obtain multiple IUGRs; and determine whether the extrinsic parameter information of the camera device has changed based on the multiple IUGRs.

[0185] Optionally, the apparatus provided in this embodiment may further include:

[0186] The reference frame determination module is used to acquire and store a reference frame image by using a camera device to capture images of a preset space at the previous time before the current time; or, when updating the external parameter information of the camera device before the current time, to acquire and store a reference frame image by using a camera device to capture images of a preset space.

[0187] Exemplary electronic device

[0188] Below, for reference Figure 5 This describes an electronic device according to embodiments of the present disclosure. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0189] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0190] like Figure 5 As shown, the electronic device includes one or more processors and memory.

[0191] The processor can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction executing capabilities, and can control other components in the electronic device to perform desired functions.

[0192] The memory can store one or more computer program products, which can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program products can be stored on the computer-readable storage media, and the processor can execute the computer program products to implement the extrinsic parameter calibration method of the camera device according to various embodiments of the present disclosure described above and / or other desired functions.

[0193] In one example, the electronic device can further include an input device and an output device, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0194] In addition, the input device can further include, for example, a keyboard, a mouse, and / or the like.

[0195] The output device can output various information, including the determined distance information, direction information, and / or the like, to the outside. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0196] Of course, in order to simplify, Figure 5 Only some of the components in the electronic device related to the present disclosure are shown in FIG. 1, and components such as buses, input / output interfaces, and / or the like are omitted. In addition, the electronic device can further include any other appropriate components according to specific application cases.

[0197] In addition to the above-described method and device, embodiments of the present disclosure can also be a computer program product including computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the extrinsic parameter calibration method of the camera device according to various embodiments of the present disclosure described in the above parts of the specification.

[0198] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0199] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the external parameter calibration method for a camera device according to various embodiments of this disclosure as described in the foregoing portion of this specification.

[0200] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0201] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0202] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0203] The block diagrams of devices, apparatuses, equipment, systems referred to in this disclosure are merely illustrative examples and are not intended to require or imply that the connection, arrangement, configuration must be as shown in the block diagrams. These devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner as will be appreciated by those skilled in the art. Words such as "include," "contain," "have," and the like are open-ended words that are to be interpreted to mean "including but not limited to," and are not to be interpreted as limiting the described embodiment to features, elements, and / or steps disclosed herein. The words "or" and "and" as used herein are to be interpreted as the word "and / or," and are not to be interpreted as requiring both features, elements, and / or steps disclosed herein. The word "such as" as used herein is to be interpreted as the phrase "such as but not limited to," and is not to be interpreted as limiting the described embodiment to features, elements, and / or steps disclosed herein.

[0204] The methods and apparatuses of this disclosure can be implemented in a number of ways. For example, the methods and apparatuses of this disclosure can be implemented using software, hardware, firmware, or any combination of these. The above described order of steps for the methods is merely illustrative, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the disclosure can also be implemented as a program recorded on a recording medium, which includes machine readable instructions for implementing the methods according to the disclosure. Thus, the disclosure also covers a recording medium storing a program for executing the methods according to the disclosure.

[0205] It is also important to note that the devices, equipment, and methods of this disclosure can be embodied in a variety of ways. These variations are contemplated as being equivalent to what is described but not necessarily equivalent to each other.

[0206] The above description of the disclosed aspects is given for illustrative purposes and not intended to limit the scope or applicability of the aspects. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Therefore, the disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0207] The above description has been given for illustrative and descriptive purposes. In addition, this description is not intended to limit embodiments of the disclosure to the forms disclosed herein. Although several example aspects and embodiments have been discussed above, those of ordinary skill in the art will recognize certain variations, modifications, changes, additions, and sub-combinations thereof.

Claims

1. A method for calibrating the external parameters of a camera device, characterized in that, include: Semantic segmentation processing is performed on the current frame image to obtain a segmented image including multiple first segmentation regions; wherein, the current frame image is an image obtained by a camera device capturing images of a preset space at the current time; Based on the plurality of first segmented regions and the plurality of second segmented regions corresponding to the pre-stored reference frame image, it is determined whether the external parameter information of the camera device has changed; wherein, the reference frame image is an image obtained by the camera device through image acquisition of the preset space; In response to changes in the external parameters of the camera device, the building information model corresponding to the preset space is acquired to determine the structured image; Based on the segmented image and the structured image that matches the position of the segmented image, the extrinsic parameter information of the camera device at the current moment is determined.

2. The method according to claim 1, characterized in that, The step of obtaining the structured image from the building information model corresponding to the preset space includes: Obtain the building information model corresponding to the preset space; Based on the assumed position of the camera device in the preset space, a projection plane is determined, and the building information model is projected onto the projection plane to obtain the structured image.

3. The method according to claim 2, characterized in that, The step of obtaining the building information model corresponding to the preset space to determine the structured image further includes: The building information model is simplified to obtain a simplified building information model; The step of determining a projection plane based on the assumed position of the camera device in the preset space, and projecting the building information model onto the projection plane to obtain the structured image includes: Based on the assumed position of the camera device in the preset space, a projection plane is determined, and the simplified building information model is projected onto the projection plane to obtain the structured image.

4. The method according to claim 1, characterized in that, After performing semantic segmentation on the current frame image to obtain a segmented image including multiple first segmentation regions, the process also includes: Feature points are extracted from the segmented image to determine at least three first feature points in the segmented image; wherein each first feature point corresponds to a feature point category; The step of determining the extrinsic information of the camera device at the current moment based on the segmented image and the structured image matching the position of the segmented image includes: Based on the segmented image and the structured image that matches the position of the segmented image, determine the three-dimensional information corresponding to at least three first feature points in the segmented image; Based on the three-dimensional information corresponding to the at least three first feature points and the coordinate information of the at least three first feature points in the segmented image, the extrinsic parameter information of the camera device at the current moment is determined.

5. The method according to claim 4, characterized in that, Before obtaining the building information model corresponding to the preset space to determine the structured image, the process also includes: Based on the at least three first feature points, at least three second feature points are determined in the building information model; wherein each second feature point corresponds to a feature point category; The step of determining the three-dimensional information corresponding to at least three first feature points in the segmented image based on the segmented image and the structured image matching the position of the segmented image includes: Determine the feature point distance between each of the at least three first feature points and the at least three second feature points of the same feature point category; Based on the feature point distance, at least three second feature points corresponding to at least three first feature points are determined; wherein, each first feature point corresponds to one second feature point; Based on the three-dimensional information corresponding to the second feature point corresponding to the first feature point, the three-dimensional information corresponding to the at least three first feature points is determined.

6. The method according to claim 1, characterized in that, Before determining the extrinsic parameter information of the camera device at the current moment based on the segmented image and the structured image matching the position of the segmented image, the method further includes: Based on the plurality of first segmentation regions in the segmented image and the plurality of third segmentation regions in the structured image, the angle of the structured image is adjusted; Based on the structured image after angle adjustment, determine the structured image that matches the position of the segmented image.

7. The method according to claim 6, characterized in that, The step of adjusting the angle of the structured image based on the plurality of first segmentation regions in the segmented image and the plurality of third segmentation regions in the structured image includes: Determine the center position of at least one of the plurality of first segmented regions to obtain the rotation center; The structured image is rotated at least once based on the rotation center to obtain at least one rotated structured image; Determine the intersection-union ratio (CUI) between the at least one third segmentation region and the at least one first segmentation region in each rotated structured image to obtain at least one CUI; Based on the intersection-union ratio, a rotated structured image is determined as the angle-adjusted structured image.

8. The method according to any one of claims 1-7, characterized in that, Also includes: In response to the fact that the extrinsic information of the camera device has not changed, the extrinsic information corresponding to the reference frame image is used as the extrinsic information of the camera device at the current moment.

9. The method according to any one of claims 1-7, characterized in that, The step of determining whether the extrinsic parameters of the camera device have changed based on the plurality of first segmented regions and the plurality of second segmented regions corresponding to the pre-stored reference frame image includes: Based on the correspondence between each first segmentation region and the second segmentation region, multiple pairs of regions are determined; wherein, the multiple first segmentation regions correspond one-to-one with the multiple second segmentation regions; each pair of regions includes the first segmentation region and the second segmentation region that have a corresponding relationship. Determine the intersection-union ratio (IUGR) between the first segmented region and the second segmented region in each of the multiple region pairs to obtain multiple IUGRs. Based on the multiple cross-union ratios, it is determined whether the external parameter information of the camera device has changed.

10. The method according to any one of claims 1-7, characterized in that, Before determining whether the extrinsic parameters of the camera device have changed based on the plurality of first segmented regions and the plurality of second segmented regions corresponding to the pre-stored reference frame image, the method further includes: At the previous moment before the current moment, the camera device captures an image of the preset space to obtain and store the reference frame image; or... Before the current moment, when updating the external parameter information of the camera device, the camera device acquires images of the preset space to obtain and store the reference frame image.

11. An electronic device, characterized in that, include: Memory, used to store computer program products; A processor is configured to execute a computer program product stored in the memory, wherein when the computer program product is executed, it implements the external parameter calibration method of the camera device according to any one of claims 1-8.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the external parameter calibration method of the camera device as described in any one of claims 1-8.

Images