External parameter calibration method, device and equipment and readable storage medium

Abstract

The invention discloses an external parameter calibration method, apparatus and device, and a readable storage medium, after an electronic device receives a calibration instruction for calibrating a camera, a shot image obtained by shooting a calibration board by the camera is obtained, and a plurality of effective areas including target patterns are determined from the shot image; and further determining the three-dimensional coordinates of the central point of each effective area, and performing external parameter calibration on the camera according to the three-dimensional coordinates of the central point of each effective area. By adopting the scheme, the electronic equipment carries out feature region extraction, classification and the like through deep learning, the robustness is high, the method is suitable for various calibration plate and light source conditions, and the purpose of improving the calibration quality is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of camera technology, and in particular to an external parameter calibration method, device, equipment and readable storage medium. Background Art

[0002] Camera calibration is the process of obtaining camera parameters. Camera parameters include internal parameters and external parameters. Internal parameters are the parameters of the camera itself, and external parameters are parameters related to the camera's installation position, such as pitch angle, rotation angle, yaw angle, etc.

[0003] Before the camera leaves the factory, it is calibrated on the production line to obtain the camera's calibration information, so that the camera can process the image according to the calibration information, so that the processed image can restore the object in the three-dimensional space. In the traditional external parameter calibration method, a calibration plate is set on the calibration tooling of the production line, and the camera shoots the calibration plate to obtain the captured image, and the captured image is processed by the image processing algorithm to perform external parameter calibration on the camera. In order to improve the calibration quality, it is necessary to adjust the calibration plate material, light source angle, light source brightness and color, etc.

[0004] In the above-mentioned external parameter calibration methods, the light source, calibration plate, etc. often fail to meet the ideal requirements of the image processing algorithm, resulting in poor calibration quality. Summary of the invention

[0005] The embodiments of the present application provide an external parameter calibration method, device, equipment and readable storage medium, which perform feature area extraction and classification through deep learning, are applicable to various calibration plates and light source conditions, and achieve the purpose of improving the calibration quality.

[0006] In a first aspect, an embodiment of the present application provides an external parameter calibration method, comprising:

[0007] Receive calibration instructions for calibrating camera external parameters;

[0008] In response to the calibration instruction, obtaining a captured image of the calibration plate by a camera;

[0009] Determine a plurality of valid areas from the captured image, wherein the valid areas are areas in the captured image that contain target patterns;

[0010] Determine the three-dimensional coordinates of the center point of each of the effective areas;

[0011] The external parameters of the camera are calibrated according to the three-dimensional coordinates of the center points of each of the effective areas.

[0012] In a second aspect, an embodiment of the present application provides an external parameter calibration device, including:

[0013] A receiving module, used for receiving a calibration instruction for calibrating the external parameters of the camera;

[0014] A response module, used for obtaining a captured image of the calibration plate captured by a camera in response to the calibration instruction;

[0015] A processing module, used to determine a plurality of valid areas from the captured image, wherein the valid areas are areas in the captured image that contain target patterns;

[0016] A determination module, used to determine the three-dimensional coordinates of the center point of each of the effective areas;

[0017] The calibration module is used to calibrate the external parameters of the camera according to the three-dimensional coordinates of the center point of each effective area.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method described in the first aspect or various possible implementation methods of the first aspect.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed by a processor, they are used to implement the method described in the first aspect or various possible implementation methods of the first aspect.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product comprising a computing program, and when the computer program is executed by a processor, the method described in the first aspect or various possible implementation methods of the first aspect is implemented.

[0021] The external parameter calibration method, device, equipment and readable storage medium provided in the embodiments of the present application, after the electronic device receives the calibration instruction of the calibration camera, obtains the captured image obtained by the camera shooting the calibration plate, determines multiple valid areas containing the target pattern from the captured image, and then determines the three-dimensional coordinates of the center point of each valid area, and performs external parameter calibration on the camera according to the three-dimensional coordinates of the center point of each valid area. With this solution, the electronic device extracts and classifies feature areas through deep learning, has high robustness, is applicable to various calibration plates and light source conditions, and achieves the purpose of improving the calibration quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of the external parameter calibration method provided in an embodiment of the present application;

[0024] Figure 2 It is a flow chart of determining the three-dimensional coordinates of the center point of the effective area in the external parameter calibration method provided in the embodiment of the present application;

[0025] Figure 3A is a schematic diagram of capturing an image in the extrinsic parameter calibration method provided in an embodiment of the present application;

[0026] Figure 3B is a schematic diagram of a positioning image in the extrinsic parameter calibration method provided in an embodiment of the present application;

[0027] Figure 3C is a schematic diagram of a characteristic region in an external parameter calibration method provided in an embodiment of the present application;

[0028] Figure 3D is a schematic diagram of a rectangular area in the extrinsic parameter calibration method provided in an embodiment of the present application;

[0029] Figure 4 is a flow chart for determining the effective area in the external parameter calibration method provided in an embodiment of the present application;

[0030] Figure 5 It is a flow chart of performing external parameter calibration based on a valid area in the external parameter calibration method provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of an external parameter calibration device provided in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In recent years, there are more and more applications that use cameras to acquire images and further analyze and process them for target positioning, detection or tracking. In order to accurately execute the above applications, it is usually necessary to calibrate the camera in advance to obtain accurate camera parameters. The parameters of the camera include intrinsic parameters and extrinsic parameters. Intrinsic parameters are parameters inherent to the camera itself, such as focal length, optical center position, etc.; extrinsic parameters include rotation vectors, translation vectors, pitch angles, rotation angles, yaw angles, etc., which represent the position and direction of the camera in the world coordinate system, and characterize the coordinate transformation relationship between the image pixel coordinate system, the camera coordinate system and the world coordinate system.

[0034] In the field of machine vision, camera calibration is a key link that determines whether the machine vision system can effectively locate and calculate the position of the target object. If the wrong camera external parameters are used, other applications involving the camera will be greatly affected. Therefore, before the camera leaves the factory, a calibration tool is set up on the production line to calibrate the camera. The calibration tool used for external parameter calibration includes a calibration table, a calibration plate, etc.

[0035] The traditional external parameter calibration method mainly uses image processing algorithms, combined with adjustments to the calibration plate material, light source angle, brightness and color, etc., to improve the calibration quality.

[0036] However, the robustness of traditional image processing algorithms is generally average. Different cameras and calibration tools require calibration engineers to adjust parameters, which takes a long time. Moreover, the adjustment of light sources and calibration boards often fails to meet the ideal requirements of image processing algorithms, resulting in poor calibration quality.

[0037] Based on this, the embodiments of the present application provide an external parameter calibration method, device, equipment and readable storage medium, which perform feature area extraction and classification through deep learning, have high robustness, are applicable to various calibration plates and light source conditions, and achieve the purpose of improving the calibration quality.

[0038] In an embodiment of the present application, the camera can be calibrated with external parameters and then assembled on self-moving devices such as sweeping robots, lawn mowing robots, and air purification robots; or the camera to be calibrated can be assembled on a self-moving device and then the camera can be calibrated with external parameters.

[0039] When calibrating external parameters for cameras that are not assembled on self-mobile devices, terminal devices such as desktops and laptops are set up at the calibration tooling for external parameter calibration on the production line. After the camera to be calibrated flows from the production line to the calibration tooling, a connection is established between the terminal device and the camera to be calibrated, such as a wireless connection or a wired connection, and the camera obtains the image obtained by shooting the calibration board through the connection. Afterwards, the terminal device uses a deep learning algorithm to process the captured image to calibrate the external parameters of the camera.

[0040] When the external parameters of the camera assembled on the self-moving device are calibrated, after the self-moving device with the camera to be calibrated arrives at the calibration fixture, the camera shoots the calibration board to obtain a captured image. After that, the self-moving device processes the captured image using a deep learning algorithm to calibrate the external parameters of the camera. Alternatively, a terminal device such as a desktop computer, a laptop computer, etc. is set at the calibration fixture used for external parameter calibration on the production line. After the self-moving device arrives at the calibration fixture, the terminal device establishes a connection with the self-moving device. After the camera shoots the calibration board to obtain a captured image, the self-moving device sends the captured image to the terminal device. After that, the terminal device processes the captured image using a deep learning algorithm to calibrate the external parameters of the camera.

[0041] Figure 1 : is a flow chart of the external parameter calibration method provided in an embodiment of the present application. The execution subject of this embodiment is an electronic device, and the electronic device is the above-mentioned self-equipment device or terminal device. This embodiment includes:

[0042] 101. Receive a calibration instruction for calibrating camera external parameters.

[0043] In the embodiment of the present application, the production line is provided with a calibration tool for external parameter calibration. When the camera to be calibrated or the mobile device assembled with the camera to be calibrated arrives at the calibration tool, a calibration instruction is sent to the electronic device to trigger the external parameter calibration.

[0044] In the embodiment of the present application, a calibration instruction can be flexibly issued to the electronic device. In one embodiment, when the electronic device detects that the camera to be calibrated or the self-mobile device assembled with the camera to be calibrated arrives at the calibration tooling, it is equivalent to receiving the calibration instruction, thereby automatically triggering the external parameter calibration.

[0045] In another method, a calibration control is displayed on a display screen of the electronic device, and the user clicks the calibration control by touching or clicking the mouse, thereby sending a calibration instruction to the electronic device.

[0046] In the embodiment of the present application, the camera is, for example, a camera installed on a mobile device, such as a depth camera, a red green blue (RGB) camera, a fisheye camera, a binocular camera, etc.

[0047] 102. In response to the calibration instruction, obtain an image obtained by photographing the calibration plate with a camera.

[0048] After receiving the calibration instruction, the electronic device triggers the camera to shoot the calibration plate to obtain a captured image, and the electronic device acquires the captured image.

[0049] In the embodiment of the present application, the calibration plate can be made of any material, such as acrylic, glass, carbon fiber, etc. The pattern of the calibration plate can be any pattern, such as a checkerboard, a circular grid, etc.

[0050] 103. Determine a plurality of valid areas from the captured image, where the valid areas are areas in the captured image that contain target patterns.

[0051] In the embodiment of the present application, the target pattern refers to the pattern on the calibration board. For example, if the calibration board is a checkerboard, the target pattern is a black square, and the effective area is the area containing the black squares. For another example, if the calibration board is a circular grid, the target pattern is a black circle, and the effective area is the area containing the black circles. The area of ​​the target pattern occupies most of the effective area.

[0052] The calibration plate contains multiple rows and columns of target patterns, each of which has an index number that indicates the position of the target pattern. The electronic device pre-acquires the three-dimensional coordinates of the center point of each target pattern and stores the index number and the corresponding three-dimensional coordinates in a mapping table.

[0053] After the electronic device obtains the captured image, it segments the captured image based on a deep learning algorithm, etc., and divides the captured image into multiple feature areas. For each feature area, it is determined whether the feature area is valid, thereby obtaining multiple valid areas. The valid area refers to the area containing the target pattern.

[0054] 104. Determine the three-dimensional coordinates of the center point of each effective area.

[0055] External parameter calibration is the process of converting two-dimensional coordinates to three-dimensional coordinates. For each valid area, the electronic device can determine the two-dimensional coordinates of the center point of the valid area, and determine which row and column of the calibration plate the valid area is based on the two-dimensional coordinates, that is, determine the index number of the valid area. After that, the three-dimensional coordinates of the center point of the valid area are obtained by querying the index number.

[0056] The above-mentioned segmentation of feature areas from captured images and classification of effective areas from feature areas are both achieved by using deep learning algorithms.

[0057] 105. Calibrate the external parameters of the camera according to the three-dimensional coordinates of the center point of each effective area.

[0058] Usually, the number of effective areas is often less than or equal to the number of characteristic areas in the calibration plate due to the influence of the position, performance, environment, etc. of the camera to be calibrated. After the electronic device determines the three-dimensional coordinates of the center point of each effective area, it performs external parameter calibration on the camera according to the three-dimensional coordinates of the center point of each effective area.

[0059] The external parameter calibration method provided in the embodiment of the present application is that after the electronic device receives the calibration instruction of the calibration camera, it obtains the captured image obtained by the camera shooting the calibration plate, determines multiple valid areas containing the target pattern from the captured image, and then determines the three-dimensional coordinates of the center point of each valid area, and performs external parameter calibration on the camera according to the three-dimensional coordinates of the center point of each valid area. With this solution, the electronic device extracts and classifies feature areas through deep learning, has high robustness, is applicable to various calibration plates and light source conditions, and achieves the purpose of improving the calibration quality.

[0060] Figure 2 : is a flow chart of determining the three-dimensional coordinates of the center point of the effective area in the external parameter calibration method provided in an embodiment of the present application. This embodiment includes:

[0061] 201. Generate a template including a feature area according to historical data, wherein the feature area on the template corresponds one-to-one to the feature area on the calibration plate, and the historical data includes the position of the center point of the feature area obtained before this calibration.

[0062] At the calibration tooling, each time the calibration is performed, the electronic device records the two-dimensional coordinates of the center point of each effective area, thereby obtaining historical data. The two-dimensional coordinates are also called the position of the center point of the effective area.

[0063] For example, before this calibration, camera A (camera A or a self-moving device assembled with camera A), camera B, and camera C passed through the calibration tooling in turn and were calibrated once each. Then the historical data includes data from three calibrations.

[0064] For another example, the camera to be calibrated is camera D, and camera D has been calibrated twice at the calibration tooling, so the historical data includes data from the two calibrations.

[0065] Before this calibration, the electronic device generates a template based on historical calibration data. For example, for the same feature area in three calibrations, the electronic device determines the average value of the horizontal coordinate and the average value of the vertical coordinate of the center point of the feature area, and generates the feature area in the template based on the average value of the horizontal and vertical coordinates.

[0066] The number of feature regions in the template is the same as the number of feature regions in the calibration plate. For example, if there are 48 feature regions in the calibration plate, such as 48 black circles, there are also 48 feature regions in the template.

[0067] 202. According to the positions of the center points of the template and each effective area, a target set is determined from the template, and the feature areas in the target set correspond to the effective areas one by one.

[0068] After the electronic device determines the position of the center point of each valid area, it determines the feature area matching the valid area from the template according to the position, and the difference between the coordinates of the center point of the valid area and the matching feature area is less than a preset threshold. In other words, for each valid area, the electronic device can find a feature area matching it on the template, thereby obtaining a target set.

[0069] 203. Determine the three-dimensional coordinates of the center point of each of the valid areas according to the three-dimensional coordinates of the center point of each feature area in the target set.

[0070] Since it is known which row and column of the calibration plate each feature region on the template corresponds to, after the electronic device determines the target set, it can determine which row and column of the calibration plate each feature region in the target set corresponds to, and then determine the index of each feature region in the target set. The feature regions and valid regions in the target set correspond one-to-one, so the electronic device can determine the index of each valid region.

[0071] The electronic device pre-acquires and stores the index of each feature area on the calibration plate and the mapping table of the corresponding three-dimensional coordinates. Therefore, the electronic device can determine the multiple three-dimensional coordinates of the corresponding effective area by querying the mapping table according to the index of each feature area in the target set.

[0072] With this solution, the electronic device generates a template containing feature areas based on historical data, matches the feature areas corresponding to each valid area from the template to obtain a target set, and then obtains the three-dimensional coordinates of each valid area based on the index query of each feature area in the target set. This solution is fast and has high accuracy.

[0073] In an embodiment of the present application, the electronic device uses a deep learning method to locate, segment, and classify the captured image. Specifically, the electronic device obtains a large number of sample images in advance, trains a positioning model using a deep learning method, and uses the positioning model to locate the captured image during external parameter calibration. Similarly, the electronic device obtains a large number of sample images, trains a segmentation model using a deep learning method, and uses the segmentation model to segment the captured image after positioning, that is, the positioning image during external parameter calibration, so as to divide the positioning image into multiple feature areas. The electronic device also uses a deep learning method to train a classification model for classifying feature areas, so as to divide multiple feature areas into two categories, one is a valid area, and the other is an invalid area. Below, the positioning, segmentation and classification in the external parameter calibration process are described in detail.

[0074] First, the positioning process.

[0075] After the electronic device obtains the captured image obtained by the camera shooting the calibration plate, the captured image is input into a pre-trained positioning model so that the positioning model outputs a positioning image within the positioning frame.

[0076] For example, the captured image includes not only the positioning plate but also other areas around the positioning plate. Figure 3A and Figure 3B .

[0077] Figure 3A It is a schematic diagram of capturing an image in the external parameter calibration method provided in an embodiment of the present application. Figure 3B is a schematic diagram of a positioning image in the external parameter calibration method provided in the embodiment of the present application. Figure 3A The captured image is shown as the dotted rectangle in the figure. In addition to the calibration plate, it also includes the surrounding environment of the calibration plate. The calibration plate contains a 7×7 feature area.

[0078] The electronic device inputs the captured image into the positioning model, and the positioning model outputs the positioning image within the positioning frame. Figure 3A That is, the positioning model cuts out the image in the positioning frame as the positioning image and outputs it. The positioning image is shown in Figure 3B shown.

[0079] It should be noted that the positioning box is invisible during the actual external parameter positioning process. It is only shown here for ease of understanding. Figure 3A The positioning frame is displayed in the image. The positioning image can be understood as an image cut out along the positioning frame.

[0080] Second, the segmentation process.

[0081] The electronic device inputs the positioning image into a pre-trained segmentation model so that the segmentation model outputs at least one feature area. Figure 3C .

[0082] Figure 3C It is a schematic diagram of the characteristic area in the external parameter calibration method provided in the embodiment of the present application. Figure 3C In the process, the electronic device extracts each feature region in the positioning image, thereby obtaining a series of feature regions. Figure 3C As shown by the black circle in .

[0083] It should be noted that although Figure 3C The number of feature regions illustrated in FIG is the same as the number of feature regions in the calibration plate. However, in practice, due to the influence of camera performance, environment, etc., the number of feature regions output by the segmentation model is less than or equal to the number of feature regions in the calibration plate.

[0084] By adopting this solution, the electronic device uses deep learning to locate the captured image and segment the located image. It is not affected by the camera performance, light source, calibration plate material, etc. The extracted feature area has high accuracy, thereby achieving the purpose of improving the quality of external parameter calibration.

[0085] Again, classification.

[0086] After the electronic device segments the positioning image to obtain at least one characteristic region, it further classifies the characteristic region into a valid region and an invalid region. The valid region is a region containing a target pattern, and the invalid region is a region not containing a target pattern.

[0087] During the classification process, the electronic device determines the center point of each feature area. Afterwards, for each feature area, the electronic device determines a rectangular area according to the center point of the feature area to obtain multiple rectangular areas, wherein the rectangular area at least includes the corresponding feature area. For example, please refer to Figure 3D , Figure 3D It is a schematic diagram of a rectangular area in the extrinsic parameter calibration method provided in an embodiment of the present application.

[0088] Please refer to Figure 3D ,For each characteristic region, the electron makes a rectangle for the characteristic region with the center point of the characteristic region as the center, thus a number of rectangular regions are obtained.

[0089] Afterwards, the electronic device outputs each rectangular area to a pre-trained classification model so that the classification model classifies the feature area, thereby classifying the feature area in at least one feature area into a valid area and an invalid area.

[0090] With this solution, electronic devices use deep learning to classify feature areas, which is not affected by camera performance, light source, calibration plate material, etc., and has fast classification speed and high accuracy.

[0091] Optionally, in the above embodiment, after the electronic device obtains the captured image, it determines whether the calibration plate exists in the captured image. When the captured image contains a complete calibration plate, the captured image is positioned, segmented, classified, etc. to determine multiple valid areas from the captured image.

[0092] When the camera reaches the calibration fixture, it may not be able to capture an image that includes the entire calibration plate due to an inappropriate posture. For example, the camera may be tilted downward or upward, so that the camera can only capture half, three-quarters, etc. of the calibration plate.

[0093] If there is no complete calibration plate in the captured image, the jig on the calibration tooling is controlled to turn, or the position of the camera is adjusted so that the camera can capture the complete calibration plate, and then the external parameter calibration is continued. Alternatively, the electronic device outputs prompt information through animation, voice, etc., so that the staff can manually adjust the calibration tooling or camera to adjust the camera to the appropriate position.

[0094] By adopting this solution, when there is no calibration plate in the captured image, the electronic device can continue to perform external parameter calibration by adjusting the position of the camera, thereby achieving the purpose of improving production efficiency.

[0095] Figure 4 This is a flow chart for determining the effective area in the external parameter calibration method provided in the embodiment of the present application. This embodiment includes:

[0096] 401. Obtain an image captured by a camera shooting a calibration plate.

[0097] 402 , determining whether the captured image contains a calibration plate. If the captured image contains a calibration plate, executing step 403 ; if the captured image does not contain a calibration plate, executing step 407 .

[0098] Exemplarily, the electronic device determines whether there is a calibration plate in the captured image by using a positioning frame or the like.

[0099] 403. Input the captured image into the positioning model to obtain the position of the calibration plate in the captured image.

[0100] Exemplarily, the electronic device inputs the captured image into the positioning model, and obtains the position of the calibration plate in the captured image through the positioning frame.

[0101] 404. The electronic device crops the captured image to obtain a positioning image.

[0102] Exemplarily, the electronic device cuts out the area within the positioning frame in the captured image to obtain a positioning image, and the positioning image includes a calibration plate.

[0103] 405. Input the positioning image into a pre-trained segmentation model to output at least one feature region.

[0104] 406. Create a rectangular area for each feature area, and input each rectangular area into a pre-trained classification model to classify the feature area in at least one feature area into a valid area and an invalid area.

[0105] 407. Positioning failed. Adjust the camera position so that the camera can capture the calibration plate.

[0106] Figure 5This is a flow chart of performing external parameter calibration based on the effective area in the external parameter calibration method provided in the embodiment of the present application. This embodiment includes:

[0107] 501. The electronic device counts the number of valid areas.

[0108] 502. The electronic device determines whether the number of valid areas is greater than or equal to a preset number. When the number of valid areas is greater than or equal to the preset number, step 503 is executed; when the number of valid areas is less than the preset number, step 507 is executed.

[0109] In the embodiment of the present application, the preset number is set according to the number of characteristic regions in the calibration plate. For example, the preset number is half of the number of characteristic regions in the calibration plate; for another example, the preset number is 60% of the number of characteristic regions.

[0110] The number of valid areas is often less than or equal to the number of feature areas in the calibration plate. For example, if the camera is cracked, dirty, or facing at an incorrect angle, it may result in a relatively small number of valid areas. Therefore, during the external parameter calibration process, the electronic device determines whether the number of valid areas is greater than or equal to a preset number. When the number of valid areas is greater than or equal to the preset number, execute step 503; otherwise, execute step 507.

[0111] With this solution, the electronic device determines the number of valid areas, and only when the number of valid areas is greater than or equal to a preset number, the camera is calibrated with external parameters, avoiding the problem of poor calibration quality caused by too few valid areas.

[0112] 503. Match the feature area corresponding to the effective area from the template to obtain a target set.

[0113] Exemplarily, the electronic device generates a template in advance based on historical data, and the feature areas in the template correspond to the feature areas in the calibration plate one by one. The electronic device finds the corresponding feature area in the template based on the two-dimensional coordinates of the center point of the effective area, thereby obtaining the target set.

[0114] 504. Determine the index number of each feature region in the target set, thereby obtaining the index number of each valid region.

[0115] 505. Determine the three-dimensional coordinates of the center point of each valid area according to the index number of the valid area.

[0116] 506. Calibrate the external parameters of the camera according to the three-dimensional coordinates of the center point of each of the effective areas.

[0117] 507. Output prompt information, where the prompt information is used to prompt the user to adjust the position of the camera or check the camera.

[0118] When the number of effective areas is less than the preset number, the electronic device outputs prompt information through animation, light, voice, etc. to prompt the user to check whether the camera is dirty, cracked or has an incorrect posture.

[0119] With this solution, when the number of effective areas is less than a preset number, the electronic device outputs a prompt message, which facilitates the user to troubleshoot the problem in a timely manner, thereby achieving the purpose of improving production efficiency.

[0120] The following are device embodiments of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0121] Figure 6 A schematic diagram of an external parameter calibration device provided in an embodiment of the present application. The external parameter calibration device 600 comprises: a receiving module 61 , a responding module 62 , a processing module 63 , a determining module 64 and a calibration module 65 .

[0122] A receiving module 61 is used to receive a calibration instruction for calibrating camera external parameters;

[0123] A response module 62, configured to obtain an image obtained by a camera shooting the calibration plate in response to the calibration instruction;

[0124] A processing module 63, configured to determine a plurality of valid areas from the captured image, wherein the valid areas are areas in the captured image that contain target patterns;

[0125] A determination module 64, used to determine the three-dimensional coordinates of the center point of each of the effective areas;

[0126] The calibration module 65 is used to calibrate the external parameters of the camera according to the three-dimensional coordinates of the center point of each effective area.

[0127] In a feasible implementation, the determination module 64 is used to generate a template containing feature areas based on historical data, the feature areas on the template correspond one-to-one with the feature areas on the calibration plate, and the historical data includes the positions of the center points of the feature areas obtained before this calibration; based on the positions of the template and the center points of each valid area, a target set is determined from the template, and the feature areas in the target set correspond one-to-one with the valid areas; based on the three-dimensional coordinates of the center points of each feature area in the target set, the three-dimensional coordinates of the center points of each valid area are determined.

[0128] In a feasible implementation, the processing module 63 is also used to determine whether the number of valid areas is greater than or equal to a preset number before the determination module 64 determines the three-dimensional coordinates of the center point of each of the valid areas; when the number of valid areas is greater than or equal to the preset number, the three-dimensional coordinates of the center point of each of the valid areas are determined.

[0129] In a feasible implementation, the processing module 63 is further used to output a prompt message when the number of the valid areas is less than the preset number, and the prompt message is used to prompt the user to adjust the posture of the camera or check the camera.

[0130] In a feasible implementation, before the processing module 63 determines multiple valid areas from the captured image, it is also used to input the captured image into a pre-trained positioning model to output a positioning image within a positioning frame, and the calibration plate is located within the positioning frame; the positioning image is input into a pre-trained segmentation model to output at least one feature area.

[0131] In a feasible implementation, before the processing module 63 determines the three-dimensional coordinates of the center point of each of the valid areas, it is also used to determine the center point of each of the feature areas; for each feature area, a rectangular area is determined according to the center point of the feature area to obtain multiple rectangular areas, and the rectangular area at least contains the corresponding feature area; each of the rectangular areas is input into a pre-trained classification model to classify the feature areas in the at least one feature area into valid areas and invalid areas.

[0132] In a feasible implementation, before the processing module 63 determines multiple valid areas from the captured image, it is also used to determine whether the calibration plate exists in the captured image; when the calibration plate does not exist in the captured image, the orientation of the camera is adjusted so that the camera can capture the calibration plate.

[0133] The external parameter calibration device provided in the embodiment of the present application can execute the actions of the electronic device in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0134] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 includes:

[0135] Processor 71 and memory 72;

[0136] The memory 72 stores computer instructions;

[0137] The processor 71 executes the computer instructions stored in the memory 72, so that the processor 71 executes the external parameter calibration method of the electronic device as described above.

[0138] The specific implementation process of the processor 71 can be found in the above-mentioned method embodiment. Its implementation principle and technical effects are similar, and will not be repeated here in this embodiment.

[0139] Optionally, the electronic device 700 further includes a communication component 73 , wherein the processor 71 , the memory 72 and the communication component 73 may be connected via a bus 74 .

[0140] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored, and the computer instructions are executed by a processor to perform the external parameter calibration method as described above.

[0141] An embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is executed by a processor to perform the external parameter calibration method as described above.

[0142] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0143] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for calibrating an external parameter, characterized in that include: Receive calibration instructions for calibrating camera external parameters; In response to the calibration instruction, obtaining a captured image of the calibration plate by a camera; Determine a plurality of valid areas from the captured image, wherein the valid areas are areas in the captured image that contain target patterns; Determine the three-dimensional coordinates of the center point of each of the effective areas; The external parameters of the camera are calibrated according to the three-dimensional coordinates of the center points of each of the effective areas.

2. The method according to claim 1, characterized in that The determining of the three-dimensional coordinates of the center point of each effective area includes: Generate a template containing a feature area according to the historical data, wherein the feature area on the template corresponds to the feature area on the calibration plate one by one, and the historical data includes the position of the center point of the feature area obtained before the current calibration; According to the positions of the center points of the template and each effective area, a target set is determined from the template, wherein the feature areas in the target set correspond to the effective areas one by one; The three-dimensional coordinates of the center point of each of the effective areas are determined according to the three-dimensional coordinates of the center point of each of the characteristic areas in the target set.

3. The method according to claim 1, characterized in that Before determining the three-dimensional coordinates of the center points of the effective areas, the method further includes: Determining whether the number of the valid areas is greater than or equal to a preset number; When the number of the effective areas is greater than or equal to a preset number, the three-dimensional coordinates of the center point of each of the effective areas are determined.

4. The method according to claim 3, characterized in that Also includes: When the number of the effective areas is less than the preset number, a prompt message is output, where the prompt message is used to prompt the user to adjust the position of the camera or check the camera.

5. The method according to any one of claims 1 to 4, characterized in that: Before determining a plurality of valid areas from the captured image, the method further includes: Inputting the captured image into a pre-trained positioning model to output a positioning image within a positioning frame, wherein the calibration plate is located within the positioning frame; The positioning image is input into a pre-trained segmentation model to output at least one feature region.

6. The method according to claim 5, characterized in that Before determining the three-dimensional coordinates of the center points of the effective areas, the method further includes: Determine the center point of each of the characteristic regions; For each feature region, a rectangular region is determined according to the center point of the feature region to obtain a plurality of rectangular regions, wherein the rectangular region at least includes the corresponding feature region; Each of the rectangular regions is input into a pre-trained classification model to classify the feature regions in the at least one feature region into a valid region and an invalid region.

7. The method according to any one of claims 1 to 4, characterized in that: Before determining a plurality of valid areas from the captured image, the method further includes: Determining whether the calibration plate exists in the captured image; When the calibration plate does not exist in the captured image, the orientation of the camera is adjusted so that the camera can capture the calibration plate.

8. An external parameter calibration device, characterized in that: include: A receiving module, used for receiving a calibration instruction for calibrating the external parameters of the camera; A response module, used for obtaining a captured image of the calibration plate captured by a camera in response to the calibration instruction; A processing module, used to determine a plurality of valid areas from the captured image, wherein the valid areas are areas in the captured image that contain target patterns; A determination module, used to determine the three-dimensional coordinates of the center point of each of the effective areas; The calibration module is used to calibrate the external parameters of the camera according to the three-dimensional coordinates of the center point of each effective area.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the electronic device implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images