Calibration method of optical sensor and computer readable storage medium

Abstract

The application relates to a method for calibrating an optical sensor and a computer readable storage medium. The method comprises: acquiring a to-be-detected image comprising a marker, determining a target image comprising the marker in the to-be-detected image, and performing feature point extraction of the marker on the target image to obtain a pixel position of the marker in the to-be-detected image, and then determining an extrinsic parameter of the optical sensor according to the pixel position of the marker in the to-be-detected image and an intrinsic parameter of the optical sensor. The pixel position of the marker in the to-be-detected image can be accurately determined based on the feature points of the marker, and the accuracy of the extrinsic parameter of the optical sensor calibrated based on the accurate pixel position of the marker is improved, and the accuracy of subsequent image positioning is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sensor calibration, in particular to a calibration method of an optical sensor and a computer readable storage medium. BACKGROUND

[0002] In the existing medical image scanning device, an optical sensor is usually mounted to collect a use scene image, and some key information is obtained based on the use scene image, such as the accurate position of a target point (such as a human joint point) in the use scene.

[0003] In order to obtain the accurate position of the target point in the use scene, the optical sensor needs to be calibrated to obtain the extrinsic parameters of the optical sensor, and the conversion of the target point from a two-dimensional image to a three-dimensional space is realized based on the extrinsic parameters.

[0004] However, the accuracy of the calibration in the traditional method is poor, which affects the accuracy of subsequent image positioning. SUMMARY

[0005] Therefore, it is necessary to provide a calibration method of an optical sensor and a computer readable storage medium in view of the above technical problems.

[0006] In a first aspect, the present application provides a calibration method of an optical sensor, comprising:

[0007] obtaining a to-be-detected image including a marker, and determining a target image including the marker in the to-be-detected image;

[0008] performing feature point extraction of the marker on the target image to obtain a pixel position of the marker in the to-be-detected image;

[0009] determining extrinsic parameters of the optical sensor according to the pixel position of the marker in the to-be-detected image and intrinsic parameters of the optical sensor.

[0010] In one embodiment, performing feature point extraction of the marker on the target image to obtain a pixel position of the marker in the to-be-detected image comprises:

[0011] performing pixel processing of the target image with increased pixel quantity to obtain an expanded image with increased resolution;

[0012] performing feature point extraction of the marker on the expanded image to obtain a pixel position of the marker in the to-be-detected image.

[0013] In one embodiment, the marker is a symmetric marker, and performing feature point extraction of the marker on the expanded image to obtain a pixel position of the marker in the to-be-detected image comprises:

[0014] The symmetrical feature points on the symmetrical markers in the expanded image are extracted to obtain the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; where the symmetrical feature points are feature points on the symmetrical markers that have a symmetrical structure.

[0015] The pixel position of the symmetrical marker in the expanded image is determined by the pixel position of the symmetrical feature point on the symmetrical marker in the expanded image;

[0016] The pixel positions of the symmetrical markers in the augmented image are restored to determine the pixel positions of the symmetrical markers in the image to be detected; wherein, the restoration process is the inverse process of pixel processing with increased pixel quantity.

[0017] In one embodiment, the symmetry marker is a cross-shaped symmetry marker. The symmetry feature points on the symmetry marker are extracted from the augmented image to obtain the pixel positions of the symmetry feature points on the symmetry marker in the augmented image, including:

[0018] A preset convolution operator is used to extract symmetrical feature points from the augmented image, resulting in the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker.

[0019] In one embodiment, the preset convolution operator includes a first operator matrix, a second operator matrix, a third operator matrix, and a fourth operator matrix;

[0020] A preset convolution operator is used to extract symmetrical feature points from the augmented image, obtaining the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker, including:

[0021] The first operator matrix, the second operator matrix, the third operator matrix, and the fourth operator matrix are respectively convolved with the pixel value of each pixel in the augmented image to obtain the first convolution value, the second convolution value, the third convolution value, and the fourth convolution value of each pixel.

[0022] The pixel with the largest first convolution value is determined as the first corner point of the cross-shaped symmetrical marker, and the pixel position of the first corner point in the expanded image is obtained;

[0023] The pixel with the largest second convolution value is determined as the second corner point of the cross-shaped symmetrical marker, and the pixel position of the second corner point in the expanded image is obtained;

[0024] The pixel with the largest third convolution value is identified as the third corner point of the cross-shaped symmetrical marker, and the pixel position of the third corner point in the expanded image is obtained.

[0025] The pixel with the largest fourth convolution value is identified as the fourth corner point of the cross-shaped symmetrical marker, and the pixel position of the fourth corner point in the expanded image is obtained.

[0026] In one embodiment, determining the pixel position of a symmetrical marker in the augmented image based on the pixel positions of symmetrical feature points on the symmetrical marker in the augmented image includes:

[0027] Determine the quadrilateral formed by the first corner point, the second corner point, the third corner point, and the fourth corner point;

[0028] The pixel position of the geometric center of the quadrilateral is obtained based on the pixel positions of the first, second, third, and fourth corner points, and the pixel position of the geometric center of the quadrilateral is determined as the pixel position of the symmetry marker in the expanded image.

[0029] In one embodiment, the process of restoring the pixel position of the symmetry marker in the augmented image to determine the pixel position of the symmetry marker in the image to be detected includes:

[0030] The pixel positions of the symmetrical markers in the expanded image are restored by using the increased pixel processing factor of the increased pixel quantity, so as to obtain the pixel positions of the symmetrical markers in the target image;

[0031] The pixel position of the symmetrical marker in the image to be detected is determined based on the pixel position of the symmetrical marker in the target image and the relative positional relationship between the target image and the image to be detected.

[0032] In one embodiment, determining a target image including a marker in an image to be detected includes:

[0033] A marker detection model is used to detect and segment markers in the image to be detected, thereby obtaining the initial regions of the markers in the image; wherein, the marker detection model is a network model trained based on marker samples;

[0034] The target image is determined based on the initial region.

[0035] In one embodiment, determining the target image based on the initial region includes:

[0036] The size of the initial region is adjusted to form the target region with the smallest possible area, including the markers;

[0037] Extract the target region from the image to be detected to obtain the target image.

[0038] Secondly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0039] The aforementioned optical sensor calibration method and computer-readable storage medium acquire an image to be detected including markers, determine a target image including markers within the image to be detected, extract feature points of the markers from the target image to obtain the pixel position of the markers in the image to be detected, and then determine the extrinsic parameters of the optical sensor based on the pixel position of the markers in the image to be detected and the intrinsic parameters of the optical sensor. The accurate determination of the pixel position of the markers in the image to be detected based on their feature points improves the accuracy of the extrinsic parameters of the optical sensor calibrated based on these accurate pixel positions, ensuring the accuracy of subsequent image localization. Attached Figure Description

[0040] Figure 1 This is an internal structural diagram of a computer device in one embodiment;

[0041] Figure 2 This is a flowchart illustrating a calibration method for an optical sensor in one embodiment;

[0042] Figure 3 This is a schematic diagram of a target image in an image to be detected during implementation;

[0043] Figure 4 This is a schematic diagram of the process for determining the pixel position of a marker in an image to be detected in one embodiment;

[0044] Figure 5 This is a flowchart illustrating the process of determining the pixel position of a marker in an image to be detected, as described in another embodiment.

[0045] Figure 6 This is a schematic diagram of the structure of the marker in one embodiment;

[0046] Figure 7 This is a schematic diagram of the process for extracting symmetrical feature points on a cross-shaped symmetrical marker in one embodiment;

[0047] Figure 8 This is a schematic diagram of a cross-shaped symmetrical marker in an expanded image in one embodiment;

[0048] Figure 9 This is a flowchart illustrating the process of determining the pixel positions of symmetrical feature points on a cross-shaped symmetrical marker in an augmented image, as shown in one embodiment.

[0049] Figure 10 This is a flowchart illustrating the process of determining the pixel positions of symmetrical feature points on a cross-shaped symmetrical marker in an augmented image, as described in another embodiment.

[0050] Figure 11 This is a flowchart illustrating the process of determining a target image in one embodiment;

[0051] Figure 12This is a flowchart illustrating the process of determining the target image in another embodiment;

[0052] Figure 13 This is a structural block diagram of an optical sensor calibration device in one embodiment. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] The calibration method for optical sensors provided in this application embodiment can be applied to, for example... Figure 1 The computer device shown can be a terminal, including a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the processor executes the computer program, it implements a calibration method for an optical sensor. The display screen can be an LCD screen or an e-ink display. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0055] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0056] In one embodiment, such as Figure 2 As shown, a calibration method for an optical sensor is provided, which can be applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0057] S210. Obtain the image to be detected that includes the marker, and determine the target image that includes the marker in the image to be detected.

[0058] In this embodiment, the computer device implementing the above method is a terminal equipped with an optical sensor for image acquisition. The image to be detected is the image acquired by the computer device through the optical sensor. The marker is the target object in the scene where the computer device is located. Optionally, the target object can be an actual object or a symbolic pattern; this embodiment does not impose specific restrictions on the type of target object.

[0059] Optionally, the computer device acquires an image including the marker using an optical sensor, uses this image as the image to be detected, and identifies the marker in the image to determine the target image including the marker. Figure 3 As shown in the figure, the shaded area is the target image identified in the image to be detected.

[0060] Optionally, the computer device can input the image to be detected into a marker detection model, which then identifies the markers in the image. The marker detection model is a network model trained from marker sample images to identify the corresponding markers.

[0061] S220. Extract feature points of the markers from the target image to obtain the pixel positions of the markers in the image to be detected.

[0062] Optionally, the feature points of the marker can be general feature points of the marker, such as edge pixels of the marker; or the feature points of the marker can be unique feature points of the marker itself, determined by the structural shape of the marker. For example, when the marker is a pentagram, the corresponding unique feature points are the pixels at the five corners.

[0063] Optionally, after identifying the target image in the image to be detected, the computer device extracts the feature points of the marker in the target image. First, it determines the pixel positions of the feature points in the target image, and then determines the pixel position of the marker in the image to be detected based on these pixel positions. Alternatively, it can first determine the pixel positions of the feature points in the image to be detected, and then determine the pixel position of the marker in the image to be detected based on these pixel positions.

[0064] Optionally, to reduce computational load and improve efficiency, the computer device can first determine the pixel position of the marker in the target image based on the pixel position of the feature points of the marker in the target image, and then, based on the relative positional relationship between the target image and the image to be detected, convert the pixel position of the marker in the target image to the image to be detected, thereby obtaining the pixel position of the marker in the image to be detected.

[0065] Optionally, when the marker is symmetrical, the computer device determines the pixel position of the geometric center point of the marker in the target image based on the pixel positions of the marker's feature points in the target image, and uses this pixel position as the marker's pixel position in the target image. When the marker is asymmetrical, the pixel position can be determined based on the pixel positions of the marker's feature points in the target image according to a preset strategy. For example, when the vertical dimension of the marker is large, the computer device determines the pixel position of the vertical midpoint in the target image based on the pixel positions of the marker's feature points in the target image, and uses this pixel position as the marker's pixel position in the target image.

[0066] S230. Determine the extrinsic parameters of the optical sensor based on the pixel position of the marker in the image to be detected and the intrinsic parameters of the optical sensor.

[0067] In this context, the intrinsic parameters of an optical sensor refer to parameters related to the sensor's own characteristics, such as focal length, eccentricity, and pixel size. The extrinsic parameters, on the other hand, refer to the parameters of the optical sensor in the world coordinate system, such as attitude parameters (rotation and translation, 6 degrees of freedom). In this embodiment, the process of calibrating the optical sensor is the process of determining its extrinsic parameters.

[0068] Optionally, the computer device obtains the pixel position of the marker in the image to be detected, and then combines it with the intrinsic parameters of the optical sensor to determine the extrinsic parameters of the optical sensor. Specifically, the extrinsic parameters of the optical sensor can be calibrated using methods such as Zhang's calibration method or Tsai calibration method, which will not be elaborated further here.

[0069] In this embodiment, the computer device acquires an image to be detected that includes markers, identifies a target image containing markers within the image to be detected, extracts feature points of the markers from the target image, obtains the pixel position of the markers in the image to be detected, and then determines the extrinsic parameters of the optical sensor based on the pixel position of the markers in the image to be detected and the intrinsic parameters of the optical sensor. The accurate determination of the pixel position of the markers in the image to be detected based on the feature points of the markers improves the accuracy of the extrinsic parameters of the optical sensor obtained based on the accurate pixel position of the markers, ensuring the accuracy of subsequent image localization.

[0070] Feature point extraction from small markers is difficult and the results are unreliable, leading to large calibration errors. To improve the reliability of feature point extraction results, in one embodiment, such as... Figure 4 As shown, S220 above, which involves extracting feature points of markers from the target image to obtain the pixel positions of the markers in the image to be detected, includes:

[0071] S410. Perform pixel processing on the target image to increase the number of pixels, and obtain an expanded image with increased resolution.

[0072] Optionally, the computer device performs interpolation upsampling processing on the pixels in the target image to increase the number of pixels in the target image, thereby obtaining an expanded image with increased resolution. The interpolation upsampling processing involves adding pixels between adjacent pixels.

[0073] Optionally, the upsampling processing of interpolation can be implemented based on nearest neighbor interpolation, bilinear interpolation, mean interpolation, median interpolation, etc., which will not be elaborated here.

[0074] S420. Extract feature points of the markers from the augmented image to obtain the pixel positions of the markers in the image to be detected.

[0075] Optionally, after obtaining the augmented image, the computer device extracts the feature points of the marker from the augmented image to obtain the pixel positions of the feature points of the marker in the augmented image, and then determines the pixel position of the marker in the image to be detected based on the pixel positions of the feature points of the marker in the augmented image.

[0076] Optionally, the computer device can perform pixel-level restoration processing on the augmented image to increase the pixel count, thereby obtaining the pixel positions of the marker's feature points in the image to be detected. Then, the pixel position of the marker in the image to be detected can be determined based on the pixel positions of the marker's feature points in the image to be detected. Alternatively, the pixel position of the marker in the augmented image can be determined first based on the pixel positions of the marker's feature points in the augmented image, and then the augmented image can be restored by increasing the pixel count to obtain the pixel position of the marker in the image to be detected.

[0077] In this embodiment, the computer device performs pixel processing on the target image to increase the number of pixels, resulting in an expanded image with increased resolution. Then, feature points of the marker are extracted from the expanded image to obtain the pixel position of the marker in the image to be detected. In this method, the expanded image has an increased number of pixels and increased resolution compared to the target image, allowing the marker to be clearly presented. This helps the computer device extract the feature points of the marker, improves the reliability of the feature point extraction results, and consequently improves the pixel position of the marker in the image to be detected, thus improving the overall accuracy of extrinsic parameter calibration.

[0078] Compared to asymmetric markers, symmetrical markers, due to their structural symmetry, allow for a more accurate representation of the actual pixel position of the marker in the image to be detected, determined by geometric algorithms. Therefore, in one embodiment, the marker is a symmetrical marker, such as... Figure 5As shown, S420 above, which extracts feature points of markers from the augmented image to obtain the pixel positions of markers in the image to be detected, includes:

[0079] S510. Extract symmetrical feature points on the symmetrical markers in the expanded image to obtain the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image.

[0080] Among them, symmetrical markers are markers with symmetrical structures, which can be axially symmetrical or centrally symmetrical. Symmetrical feature points are feature points on symmetrical markers that have symmetrical structures; similarly, they can be axially symmetrical or centrally symmetrical.

[0081] Optionally, such as Figure 6 As shown, when the symmetrical marker is a circle, the corresponding symmetrical feature point can be a pixel that forms the diameter of the circle; it can be one or multiple pixels. Figure 6 The diagram shows pixels a1, a2, a3, and a4 forming two diameters on a circle. When the symmetrical marker is a rectangle, the corresponding symmetrical feature points can be pixels b1, b2, b3, and b4 at the four vertices of the rectangle, or pixels at the two diagonal corners of these four vertices, such as pixels b1 and b3, or pixels b2 and b4. When the symmetrical marker is a pentagram, the corresponding symmetrical feature points can be pixels c1, c2, c3, c4, and c5 at the five outer vertices of the pentagram, or pixels c1', c2', c3', c4', and c5' at the five inner corners of the pentagram. When the symmetrical marker is a cross, the corresponding symmetrical feature points can be pixels d1, d2, d3, and d4 at the four corners closest to the center of the cross, or pixels at the two diagonal corners of these four corners, such as pixels d1 and d3, or pixels d2 and d4. In this embodiment, the shape of the symmetrical markers and the position of the symmetrical feature points are not specifically limited, as long as they meet the requirement of having a symmetrical structure.

[0082] Optionally, the computer device uses a corresponding extraction algorithm to extract corresponding symmetrical feature points based on the shape of the symmetrical marker, thereby obtaining the pixel positions of the symmetrical feature points on the symmetrical marker in the expanded image. For example, if the symmetrical marker is circular, the computer device can extract the diameter of the circle and obtain the symmetrical feature points on the circular symmetrical marker that form that diameter (in the case of extracting two diameters, the symmetrical feature points are...). Figure 6Pixels a1, a2, a3, and a4 in the image are used to obtain the pixel positions of each symmetrical feature point in the expanded image. When the symmetrical marker is cross-shaped, the computer can use an L-shaped convolution operator to extract the four corner points near the center of the cross, or the two diagonal corner points among the four corner points, to obtain the symmetrical feature points of the cross-shaped symmetrical marker (if four corner points are extracted, the symmetrical feature points are...). Figure 6 If we extract the two corner points on the diagonal of pixels d1, d2, d3, and d4, the symmetrical feature point is... Figure 6 Pixels d1 and d3, or pixels d2 and d4, are used to obtain the pixel positions of each symmetrical feature point in the expanded image.

[0083] S520. Determine the pixel position of the symmetrical marker in the expanded image based on the pixel position of the symmetrical feature point on the symmetrical marker in the expanded image.

[0084] Optionally, the computer device can use a geometric algorithm to calculate the pixel position of the symmetrical marker in the expanded image based on the pixel positions of the symmetrical feature points in the expanded image. Specifically, the computer device can directly determine the pixel position of the geometric center point of the symmetrical feature point in the expanded image based on the pixel positions of the symmetrical feature points (i.e., the pixel positions of the symmetrical feature points in the expanded image), and use this pixel position of the geometric center point as the pixel position of the symmetrical marker in the expanded image. For example, for a circular symmetrical marker, if the computer device extracts four symmetrical feature points formed by two diameters, the computer device obtains the pixel position of the intersection of the two diameters including the four symmetrical feature points in the expanded image, and uses this pixel position of the intersection as the pixel position of the circular symmetrical marker in the expanded image. For example, for a cross-shaped symmetrical marker, if the computer device extracts two diagonal points from the four corner points near the center of the cross, the computer device then obtains the pixel position of the midpoint of the line connecting the two diagonal points in the expanded image, and uses the pixel position of this midpoint in the expanded image as the pixel position of the cross-shaped symmetrical marker in the expanded image.

[0085] S530. Restore the pixel position of the symmetrical marker in the expanded image to determine the pixel position of the symmetrical marker in the image to be detected.

[0086] The restoration process is the inverse of the pixel processing that increases the number of pixels. Optionally, when the pixel processing that increases the number of pixels is an upsampling process, the restoration process is the inverse of the upsampling process, which is a downsampling process.

[0087] Optionally, the computer device performs downsampling processing on the expanded image, which is the reverse of the previous upsampling processing. That is, it restores the pixel position of the symmetrical marker in the expanded image to obtain the pixel position of the symmetrical marker in the target image. Then, according to the relative positional relationship between the target image and the image to be detected, the pixel position of the symmetrical marker in the target image is converted to the image to be detected, so that the pixel position of the symmetrical marker in the image to be detected can be obtained.

[0088] It should be noted that the process of determining the pixel position of a symmetrical marker based on the pixel position of symmetrical feature points on the symmetrical marker can also be directly applied to the target image. For details, see S510-S530 above. For the target image (i.e., without augmentation), the computer device can directly extract the pixel positions of the symmetrical marker points on the symmetrical marker in the target image, obtaining the pixel position of the symmetrical marker in the target image. Then, based on the relative positional relationship between the target image and the image to be detected, the pixel position of the symmetrical marker in the target image is converted to the image to be detected, thus obtaining the pixel position of the symmetrical marker in the image to be detected.

[0089] In this embodiment, the marker is a symmetrical marker. The computer device extracts symmetrical feature points of the symmetrical marker from the expanded image, obtaining the pixel positions of the symmetrical feature points on the symmetrical marker in the expanded image. Then, based on the pixel positions of the symmetrical feature points on the symmetrical marker in the expanded image, the pixel position of the symmetrical marker in the expanded image is determined. The pixel position of the symmetrical marker in the expanded image is then restored to determine the pixel position of the symmetrical marker in the image to be detected. By using the above method, the pixel position of the symmetrical marker in the image to be detected is determined using symmetrical feature points on the symmetrical marker. The determined pixel position more accurately represents the actual pixel position of the symmetrical marker, improving accuracy. The restoration process is the reverse process of the aforementioned pixel processing with increased pixel quantity. Through this restoration process, the pixel position of the symmetrical marker can be converted into floating-point data, further improving data accuracy.

[0090] In practical applications, convolution operators can be used to extract feature points from images. In one embodiment, when the symmetrical marker is a cross-shaped symmetrical marker, step S510, extracting symmetrical feature points on the symmetrical marker in the expanded image to obtain the pixel positions of the symmetrical feature points on the symmetrical marker in the expanded image, includes:

[0091] A preset convolution operator is used to extract symmetrical feature points from the augmented image, resulting in the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker.

[0092] The preset convolution operator is the operator matrix used to extract the four corner points near the center of the cross-shaped symmetrical marker, and the four corner points correspond to four operator matrices.

[0093] In an optional embodiment, the preset convolution operator includes a first operator matrix, a second operator matrix, a third operator matrix, and a fourth operator matrix. For example... Figure 7 As shown, a preset convolution operator is used to extract symmetrical feature points from the augmented image, obtaining the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker, including:

[0094] S710. Perform convolution operations between the first operator matrix, the second operator matrix, the third operator matrix, and the fourth operator matrix and the pixel value of each pixel in the expanded image, respectively, to obtain the first convolution value, the second convolution value, the third convolution value, and the fourth convolution value of each pixel.

[0095] Optionally, the computer device determines four symmetrical feature points (i.e., ...) on the cross-shaped symmetrical marker in the augmented image using preset first operator matrices, second operator matrices, third operator matrices, and fourth operator matrices. Figure 6 The pixel positions of pixels d1, d2, d3, and d4 in the expanded image are determined. Specifically, the first operator matrix is ​​convolved with the pixel value of each pixel in the expanded image to obtain the first convolution value for each pixel. The second operator matrix is ​​convolved with the pixel value of each pixel in the expanded image to obtain the second convolution value for each pixel. The third operator matrix is ​​convolved with the pixel value of each pixel in the expanded image to obtain the third convolution value for each pixel. Then, the fourth operator matrix is ​​convolved with the pixel value of each pixel in the expanded image to obtain the fourth convolution value for each pixel. Thus, each pixel in the expanded image corresponds to four convolution values: the first, second, third, and fourth convolution values.

[0096] S720. The pixel with the largest first convolution value is determined as the first corner point of the cross-shaped symmetrical marker, and the pixel position of the first corner point in the expanded image is obtained.

[0097] Optionally, after the computer device obtains the four convolution values ​​corresponding to each pixel in the expanded image, it compares the magnitudes of the first convolution values ​​corresponding to each pixel, determines the largest first convolution value, and identifies the pixel with the largest first convolution value as the first corner point of the cross-shaped symmetrical marker, while simultaneously obtaining the pixel position of the first corner point in the expanded image.

[0098] In practical applications, the cross-shaped symmetrical markers in the expanded image are dark, such as black, while the remaining background area is light. Optionally, the first operator matrix is ​​an L-shaped operator. When each line of the cross-shaped symmetrical marker in the expanded image occupies two pixels in width, the first operator matrix can be: 000011000 000011000 000011000 000011000 111111000 111111000 000000000 000000000 000000000

[0108] like Figure 8 As shown, the first operator matrix is ​​used to extract the first corner point d1 in the cross-shaped symmetrical marker to obtain the pixel position of the first corner point d1 in the expanded image.

[0109] S730, The pixel with the largest second convolution value is determined as the second corner point of the cross-shaped symmetrical marker, and the pixel position of the second corner point in the expanded image is obtained.

[0110] Optionally, the computer device compares the magnitudes of the second convolution values ​​corresponding to each pixel, determines the largest second convolution value, and identifies the pixel with the largest second convolution value as the second corner point of the cross-shaped symmetrical marker, while simultaneously obtaining the pixel position of the second corner point in the expanded image.

[0111] Optionally, the second operator matrix is ​​an L-shaped operator. When each line in the cross-shaped symmetrical marker in the augmented image occupies two pixels in width, the second operator matrix can be: 000110000 000110000 000110000 000110000 000111111 000111111 000000000 000000000 000000000

[0121] like Figure 8 As shown, the second operator matrix is ​​used to extract the second corner point d2 in the cross-shaped symmetrical marker to obtain the pixel position of the second corner point d2 in the expanded image.

[0122] S740. The pixel with the largest third convolution value is determined as the third corner point of the cross-shaped symmetrical marker, and the pixel position of the third corner point in the expanded image is obtained.

[0123] Optionally, the computer device compares the magnitudes of the third convolution values ​​corresponding to each pixel, determines the largest third convolution value, and identifies the pixel with the largest third convolution value as the third corner point of the cross-shaped symmetrical marker, while simultaneously obtaining the pixel position of the third corner point in the expanded image.

[0124] Optionally, the third operator matrix is ​​an L-shaped operator. When each line in the cross-shaped symmetrical marker in the augmented image occupies two pixels in width, the third operator matrix can be: 000000000 000000000 000000000 000111111 000111111 000110000 000110000 000110000 000110000

[0134] like Figure 8 As shown, the third operator matrix is ​​used to extract the third corner point d3 in the cross-shaped symmetrical marker to obtain the pixel position of the third corner point d3 in the expanded image.

[0135] S750, The pixel with the largest fourth convolution value is determined as the fourth corner point of the cross-shaped symmetrical marker, and the pixel position of the fourth corner point in the expanded image is obtained.

[0136] Optionally, the computer device compares the magnitudes of the fourth convolution values ​​corresponding to each pixel, determines the largest fourth convolution value, and identifies the pixel with the largest fourth convolution value as the fourth corner point of the cross-shaped symmetrical marker, while simultaneously obtaining the pixel position of the fourth corner point in the expanded image.

[0137] Optionally, the fourth operator matrix is ​​an L-shaped operator. When each line in the cross-shaped symmetrical marker in the augmented image occupies two pixels in width, the fourth operator matrix can be: 000000000 000000000 000000000 111111000 111111000 000011000 000011000 000011000 000011000

[0147] like Figure 8 As shown, the fourth operator matrix is ​​used to extract the fourth corner point d4 in the cross-shaped symmetrical marker to obtain the pixel position of the fourth corner point d4 in the expanded image.

[0148] In an optional embodiment, after determining the pixel positions of the four corner points of the cross-shaped symmetrical marker in the expanded image, as follows: Figure 9 As shown, S520 above, determining the pixel position of the symmetrical marker in the expanded image based on the pixel position of the symmetrical feature points on the symmetrical marker in the expanded image, includes:

[0149] S910. Determine the quadrilateral formed by the first corner point, the second corner point, the third corner point, and the fourth corner point.

[0150] Optionally, the computer device can sequentially connect the four corner points of the cross-shaped symmetrical marker—namely, the first corner point, the second corner point, the third corner point, and the fourth corner point—to form a closed quadrilateral. For example... Figure 8 As shown, the computer device connects the first corner point d1, the second corner point d2, the third corner point d3, and the fourth corner point d4 in sequence to form a quadrilateral.

[0151] S920. Obtain the pixel position of the geometric center of the quadrilateral based on the pixel positions of the first corner point, the second corner point, the third corner point, and the fourth corner point, and determine the pixel position of the geometric center of the quadrilateral as the pixel position of the symmetry marker in the expanded image.

[0152] Optionally, the computer device determines the pixel position of the geometric center of the formed quadrilateral in the expanded image based on the pixel positions of the first, second, third, and fourth corner points, as the pixel position of the cross-shaped symmetry marker in the expanded image. For example, as Figure 8 As shown, the computer device can determine the first straight line L1 where the first corner point d1 and the fourth corner point d4 are located based on the pixel positions of the first corner point d1 and the fourth corner point d4, and determine the second straight line L2 where the second corner point d1 and the third corner point d3 are located based on the pixel positions of the second corner point d2 and the third corner point d3. Then, it can calculate the pixel position of the intersection point O between the first straight line L1 and the second straight line L2 in the expanded image. This intersection point O is the geometric center of the quadrilateral formed by the four corner points.

[0153] As mentioned above, computer equipment can directly extract symmetrical feature points on symmetrical markers in a target image to determine the pixel position of the symmetrical markers in the target image. For cross-shaped symmetrical markers, the preset convolution operator in the aforementioned embodiments can be used to extract symmetrical feature points to obtain the pixel position of the cross-shaped symmetrical marker in the target image. The specific process can be combined with... Figure 8 For details, please refer to the detailed descriptions in S710-S750 and S910-S920. Simply replace the expanded image with the target image; further details will not be provided here.

[0154] In this embodiment, for a cross-shaped symmetrical marker, the computer device uses a preset convolution operator to extract symmetrical feature points from the expanded image, obtaining the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker. Specifically, the first, second, third, and fourth operator matrices corresponding to the four corner points are used to perform convolution operations on each pixel in the expanded image, and the four corner points and their respective pixel positions in the expanded image are determined based on the obtained convolution values. This method enables targeted extraction of symmetrical feature points from cross-shaped symmetrical markers, improving the accuracy of the extraction.

[0155] Downsampling is the inverse of upsampling; essentially, it involves restoring the expanded image obtained from upsampling based on the scaling factor of upsampling. Therefore, in one embodiment, such as... Figure 10 As shown, S530 above, which restores the pixel position of the symmetrical marker in the expanded image and determines the pixel position of the symmetrical marker in the image to be detected, includes:

[0156] S1010. The pixel position of the symmetrical marker in the expanded image is restored by using the pixel processing multiplier of the increased pixel quantity, so as to obtain the pixel position of the symmetrical marker in the target image.

[0157] Optionally, the computer device determines the upsampling processing factor, i.e., the pixel processing factor for increasing the number of pixels, and performs restoration processing on the pixel position of the symmetrical marker in the expanded image based on the upsampling factor, i.e., reducing the pixel position of the symmetrical marker in the expanded image by the upsampling factor to obtain the pixel position of the symmetrical marker in the target image.

[0158] S1020. Determine the pixel position of the symmetrical marker in the image to be detected based on the pixel position of the symmetrical marker in the target image and the relative positional relationship between the target image and the image to be detected.

[0159] Optionally, after the computer device obtains the pixel position of the symmetrical marker in the target image, since the target image is obtained from the image to be detected, the pixel position of the symmetrical marker in the target image can be transformed based on the relative positional relationship between the target image and the image to be detected, thus obtaining the pixel position of the symmetrical marker in the image to be detected.

[0160] In this embodiment, the computer device utilizes the increased pixel processing multiplier of the increased pixel count to restore the pixel position of the symmetrical marker in the augmented image, obtaining the pixel position of the symmetrical marker in the target image. Then, based on the pixel position of the symmetrical marker in the target image and the relative positional relationship between the target image and the image to be detected, the pixel position of the symmetrical marker in the image to be detected is determined. This method enables the restoration of the augmented image and the determination of the pixel position of the symmetrical marker in the image to be detected, providing an accurate data foundation for subsequent extrinsic parameter calibration and thus improving the accuracy of the calibration.

[0161] To reduce data computation and improve calibration efficiency, in one embodiment, such as Figure 11 As shown, in step S210 above, determining the target image containing the marker in the image to be detected includes:

[0162] S1110. A marker detection model is used to detect and segment markers in the image to be detected, thereby obtaining the initial region of the markers in the image to be detected.

[0163] The marker detection model is a network model trained based on marker samples.

[0164] Optionally, the computer device inputs the obtained image to be detected into the marker detection model, so that the marker detection model can detect and segment the markers in the image to be detected, and output the initial region of the marker in the image to be detected, such as marking the initial region including the marker in the image to be detected.

[0165] S1120. Determine the target image based on the initial region.

[0166] Optionally, after determining the initial region in the image to be detected, the computer device further determines the target image based on the initial region.

[0167] In an alternative embodiment, such as Figure 12 As shown, the above-mentioned S1120, determining the target image based on the initial region, includes:

[0168] S1210. Adjust the size of the initial region to form the target region with the smallest area including the marker.

[0169] Optionally, after determining the initial region in the image to be detected, the computer device marks the initial region with a bounding box and displays it on the screen. The user can adjust the bounding box displayed on the screen, and the computer device will adjust the size of the initial region displayed on the screen according to the adjustment instructions corresponding to the adjustment operation, so as to form the target region with the smallest possible area including the marker.

[0170] Optionally, when the adjustment operation is only for adjusting the size of the annotation box, the target area is the smallest rectangular area including the marker; when the adjustment operation includes both adjusting the size of the annotation box and adjusting the shape of the annotation box, the target area is the area with the same or similar shape as the marker.

[0171] S1220. Extract the target region from the image to be detected to obtain the target image.

[0172] Optionally, after the computer device determines the target region in the image to be detected based on the user's adjustment operation, it directly extracts the target region to obtain the target image.

[0173] In this embodiment, the computer device uses a marker detection model to detect and segment markers in the image to be detected, obtaining an initial region of the marker in the image. The target image is then determined based on this initial region. Specifically, the size of the initial region is adjusted to form a target region with the smallest possible area including the marker. This target region is then extracted from the image to be detected, resulting in the target image. The target image determined by this method removes unnecessary image regions from the image to be detected as much as possible, significantly reducing the computational load for subsequent marker identification, improving the overall efficiency of extrinsic parameter calibration, and simultaneously reducing noise interference and improving calibration accuracy.

[0174] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0175] In one embodiment, such as Figure 13As shown, a calibration device for an optical sensor is provided, comprising: an image acquisition module 1301, a feature extraction module 1302, and an extrinsic parameter calibration module 1303, wherein:

[0176] Image acquisition module 1301 is used to acquire an image to be detected including markers, and to determine a target image including markers in the image to be detected;

[0177] The feature extraction module 1302 is used to extract feature points of markers from the target image to obtain the pixel positions of the markers in the image to be detected;

[0178] The extrinsic parameter calibration module 1303 is used to determine the extrinsic parameters of the optical sensor based on the pixel position of the marker in the image to be detected and the intrinsic parameters of the optical sensor.

[0179] In one embodiment, the feature extraction module 1302 is specifically used for:

[0180] The target image is processed to increase the number of pixels, resulting in an expanded image with increased resolution. Feature points of the markers are extracted from the expanded image to obtain the pixel positions of the markers in the image to be detected.

[0181] In one embodiment, the marker is a symmetrical marker, and the feature extraction module 1302 is specifically used for:

[0182] The symmetrical feature points on the symmetrical markers in the expanded image are extracted to obtain the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; where the symmetrical feature points are feature points on the symmetrical markers that have a symmetrical structure; the pixel positions of the symmetrical markers in the expanded image are determined based on the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; the pixel positions of the symmetrical markers in the expanded image are restored to determine the pixel positions of the symmetrical markers in the image to be detected; where the restoration process is the inverse process of pixel processing with increased pixel count.

[0183] In one embodiment, the symmetry marker is a cross-shaped symmetry marker, and the feature extraction module 1302 is specifically used for:

[0184] A preset convolution operator is used to extract symmetrical feature points from the augmented image, resulting in the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker.

[0185] In one embodiment, the preset convolution operator includes a first operator matrix, a second operator matrix, a third operator matrix, and a fourth operator matrix; the feature extraction module 1302 is specifically used for:

[0186] The first, second, third, and fourth operator matrices are convolved with the pixel values ​​of each pixel in the expanded image, respectively, to obtain the first, second, third, and fourth convolution values ​​for each pixel. The pixel with the largest first convolution value is determined as the first corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest second convolution value is determined as the second corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest third convolution value is determined as the third corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest fourth convolution value is determined as the fourth corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained.

[0187] In one embodiment, the feature extraction module 1302 is specifically used for:

[0188] Determine the quadrilateral formed by the first corner point, the second corner point, the third corner point, and the fourth corner point; obtain the pixel position of the geometric center of the quadrilateral based on the pixel positions of the first corner point, the second corner point, the third corner point, and the fourth corner point, and determine the pixel position of the geometric center of the quadrilateral as the pixel position of the symmetry marker in the expanded image.

[0189] In one embodiment, the feature extraction module 1302 is specifically used for:

[0190] The pixel position of the symmetrical marker in the expanded image is restored by increasing the pixel quantity by a certain multiple, so as to obtain the pixel position of the symmetrical marker in the target image. The pixel position of the symmetrical marker in the target image is determined based on the pixel position of the symmetrical marker in the target image and the relative positional relationship between the target image and the image to be detected.

[0191] In one embodiment, the image acquisition module 1301 is specifically used for:

[0192] A marker detection model is used to detect and segment markers in the image to be detected, thereby obtaining the initial region of the marker in the image; wherein, the marker detection model is a network model trained based on marker samples; the target image is determined based on the initial region.

[0193] In one embodiment, the image acquisition module 1301 is specifically used for:

[0194] The size of the initial region is adjusted to form the target region with the smallest possible area, including the marker; the target region is then extracted from the image to be detected to obtain the target image.

[0195] Each module in the aforementioned optical sensor calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0196] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0197] The process involves acquiring an image to be detected that includes markers, identifying a target image containing markers within the image to be detected, extracting feature points of the markers from the target image to obtain the pixel positions of the markers in the image to be detected, and determining the extrinsic parameters of the optical sensor based on the pixel positions of the markers in the image to be detected and the intrinsic parameters of the optical sensor.

[0198] In one embodiment, the processor further performs the following steps when executing the computer program:

[0199] The target image is processed to increase the number of pixels, resulting in an expanded image with increased resolution. Feature points of the markers are extracted from the expanded image to obtain the pixel positions of the markers in the image to be detected.

[0200] In one embodiment, the marker is a symmetrical marker, and the processor, when executing the computer program, further performs the following steps:

[0201] The symmetrical feature points on the symmetrical markers in the expanded image are extracted to obtain the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; where the symmetrical feature points are feature points on the symmetrical markers that have a symmetrical structure; the pixel positions of the symmetrical markers in the expanded image are determined based on the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; the pixel positions of the symmetrical markers in the expanded image are restored to determine the pixel positions of the symmetrical markers in the image to be detected; where the restoration process is the inverse process of pixel processing with increased pixel count.

[0202] In one embodiment, the symmetry marker is a cross-shaped symmetry marker, and the processor, when executing the computer program, further performs the following steps:

[0203] A preset convolution operator is used to extract symmetrical feature points from the augmented image, resulting in the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker.

[0204] In one embodiment, the preset convolution operator includes a first operator matrix, a second operator matrix, a third operator matrix, and a fourth operator matrix; when the processor executes the computer program, it further implements the following steps:

[0205] The first, second, third, and fourth operator matrices are convolved with the pixel values ​​of each pixel in the expanded image, respectively, to obtain the first, second, third, and fourth convolution values ​​for each pixel. The pixel with the largest first convolution value is determined as the first corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest second convolution value is determined as the second corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest third convolution value is determined as the third corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest fourth convolution value is determined as the fourth corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained.

[0206] In one embodiment, the processor further performs the following steps when executing the computer program:

[0207] Determine the quadrilateral formed by the first corner point, the second corner point, the third corner point, and the fourth corner point; obtain the pixel position of the geometric center of the quadrilateral based on the pixel positions of the first corner point, the second corner point, the third corner point, and the fourth corner point, and determine the pixel position of the geometric center of the quadrilateral as the pixel position of the symmetry marker in the expanded image.

[0208] In one embodiment, the processor further performs the following steps when executing the computer program:

[0209] The pixel position of the symmetrical marker in the expanded image is restored by increasing the pixel quantity by a certain multiple, so as to obtain the pixel position of the symmetrical marker in the target image. The pixel position of the symmetrical marker in the target image is determined based on the pixel position of the symmetrical marker in the target image and the relative positional relationship between the target image and the image to be detected.

[0210] In one embodiment, the processor further performs the following steps when executing the computer program:

[0211] A marker detection model is used to detect and segment markers in the image to be detected, thereby obtaining the initial region of the marker in the image; wherein, the marker detection model is a network model trained based on marker samples; the target image is determined based on the initial region.

[0212] In one embodiment, the processor further performs the following steps when executing the computer program:

[0213] The size of the initial region is adjusted to form the target region with the smallest possible area, including the marker; the target region is then extracted from the image to be detected to obtain the target image.

[0214] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0215] The process involves acquiring an image to be detected that includes markers, identifying a target image containing markers within the image to be detected, extracting feature points of the markers from the target image to obtain the pixel positions of the markers in the image to be detected, and determining the extrinsic parameters of the optical sensor based on the pixel positions of the markers in the image to be detected and the intrinsic parameters of the optical sensor.

[0216] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0217] The target image is processed to increase the number of pixels, resulting in an expanded image with increased resolution. Feature points of the markers are extracted from the expanded image to obtain the pixel positions of the markers in the image to be detected.

[0218] In one embodiment, the marker is a symmetrical marker, and the computer program, when executed by the processor, further performs the following steps:

[0219] The symmetrical feature points on the symmetrical markers in the expanded image are extracted to obtain the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; where the symmetrical feature points are feature points on the symmetrical markers that have a symmetrical structure; the pixel positions of the symmetrical markers in the expanded image are determined based on the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; the pixel positions of the symmetrical markers in the expanded image are restored to determine the pixel positions of the symmetrical markers in the image to be detected; where the restoration process is the inverse process of pixel processing with increased pixel count.

[0220] In one embodiment, the symmetry marker is a cross-shaped symmetry marker, and the computer program, when executed by the processor, further performs the following steps:

[0221] A preset convolution operator is used to extract symmetrical feature points from the augmented image, resulting in the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker.

[0222] In one embodiment, the preset convolution operator includes a first operator matrix, a second operator matrix, a third operator matrix, and a fourth operator matrix; when the computer program is executed by the processor, it further implements the following steps:

[0223] The first, second, third, and fourth operator matrices are convolved with the pixel values ​​of each pixel in the expanded image, respectively, to obtain the first, second, third, and fourth convolution values ​​for each pixel. The pixel with the largest first convolution value is determined as the first corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest second convolution value is determined as the second corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest third convolution value is determined as the third corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest fourth convolution value is determined as the fourth corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained.

[0224] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0225] Determine the quadrilateral formed by the first corner point, the second corner point, the third corner point, and the fourth corner point; obtain the pixel position of the geometric center of the quadrilateral based on the pixel positions of the first corner point, the second corner point, the third corner point, and the fourth corner point, and determine the pixel position of the geometric center of the quadrilateral as the pixel position of the symmetry marker in the expanded image.

[0226] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0227] The pixel position of the symmetrical marker in the expanded image is restored by increasing the pixel quantity by a certain multiple, so as to obtain the pixel position of the symmetrical marker in the target image. The pixel position of the symmetrical marker in the target image is determined based on the pixel position of the symmetrical marker in the target image and the relative positional relationship between the target image and the image to be detected.

[0228] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0229] A marker detection model is used to detect and segment markers in the image to be detected, thereby obtaining the initial region of the marker in the image; wherein, the marker detection model is a network model trained based on marker samples; the target image is determined based on the initial region.

[0230] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0231] The size of the initial region is adjusted to form the target region with the smallest possible area, including the marker; the target region is then extracted from the image to be detected to obtain the target image.

[0232] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0233] The process involves acquiring an image to be detected that includes markers, identifying a target image containing markers within the image to be detected, extracting feature points of the markers from the target image to obtain the pixel positions of the markers in the image to be detected, and determining the extrinsic parameters of the optical sensor based on the pixel positions of the markers in the image to be detected and the intrinsic parameters of the optical sensor.

[0234] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0235] The target image is processed to increase the number of pixels, resulting in an expanded image with increased resolution. Feature points of the markers are extracted from the expanded image to obtain the pixel positions of the markers in the image to be detected.

[0236] In one embodiment, the marker is a symmetrical marker, and the computer program, when executed by the processor, further performs the following steps:

[0237] The symmetrical feature points on the symmetrical markers in the expanded image are extracted to obtain the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; where the symmetrical feature points are feature points on the symmetrical markers that have a symmetrical structure; the pixel positions of the symmetrical markers in the expanded image are determined based on the pixel positions of the symmetrical feature points on the symmetrical markers in the expanded image; the pixel positions of the symmetrical markers in the expanded image are restored to determine the pixel positions of the symmetrical markers in the image to be detected; where the restoration process is the inverse process of pixel processing with increased pixel count.

[0238] In one embodiment, the symmetry marker is a cross-shaped symmetry marker, and the computer program, when executed by the processor, further performs the following steps:

[0239] A preset convolution operator is used to extract symmetrical feature points from the augmented image, resulting in the pixel positions of the four corner points near the center of the cross-shaped symmetrical marker.

[0240] In one embodiment, the preset convolution operator includes a first operator matrix, a second operator matrix, a third operator matrix, and a fourth operator matrix; when the computer program is executed by the processor, it further implements the following steps:

[0241] The first, second, third, and fourth operator matrices are convolved with the pixel values ​​of each pixel in the expanded image, respectively, to obtain the first, second, third, and fourth convolution values ​​for each pixel. The pixel with the largest first convolution value is determined as the first corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest second convolution value is determined as the second corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest third convolution value is determined as the third corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained. The pixel with the largest fourth convolution value is determined as the fourth corner point of the cross-shaped symmetry marker, and its pixel position in the expanded image is obtained.

[0242] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0243] Determine the quadrilateral formed by the first corner point, the second corner point, the third corner point, and the fourth corner point; obtain the pixel position of the geometric center of the quadrilateral based on the pixel positions of the first corner point, the second corner point, the third corner point, and the fourth corner point, and determine the pixel position of the geometric center of the quadrilateral as the pixel position of the symmetry marker in the expanded image.

[0244] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0245] The pixel position of the symmetrical marker in the expanded image is restored by increasing the pixel quantity by a certain multiple, so as to obtain the pixel position of the symmetrical marker in the target image. The pixel position of the symmetrical marker in the target image is determined based on the pixel position of the symmetrical marker in the target image and the relative positional relationship between the target image and the image to be detected.

[0246] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0247] A marker detection model is used to detect and segment markers in the image to be detected, thereby obtaining the initial region of the marker in the image; wherein, the marker detection model is a network model trained based on marker samples; the target image is determined based on the initial region.

[0248] In one embodiment, when the computer program is executed by the processor, it further performs the following steps:

[0249] The size of the initial region is adjusted to form the target region with the smallest possible area, including the marker; the target region is then extracted from the image to be detected to obtain the target image.

[0250] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0251] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0252] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A calibration method for an optical sensor, characterized in that, The method includes: Acquire an image to be detected that includes a marker, and determine a target image that includes the marker in the image to be detected; the marker is a cross-shaped symmetrical marker. The target image is processed to increase the number of pixels, resulting in an expanded image with increased resolution; A preset convolution operator is used to extract symmetrical feature points from the augmented image to obtain the pixel positions of the four corner points near the center of the marker; wherein, the symmetrical feature points are feature points on the marker that have a symmetrical structure; The pixel position of the marker in the expanded image is determined based on the pixel position of the symmetrical feature points on the marker in the expanded image; The pixel position of the marker in the augmented image is restored to determine the pixel position of the marker in the image to be detected; wherein, the restoration process is the reverse process of the pixel processing with increased pixel quantity; The extrinsic parameters of the optical sensor are determined based on the pixel position of the marker in the image to be detected and the intrinsic parameters of the optical sensor.

2. The method according to claim 1, characterized in that, The preset convolution operator includes a first operator matrix, a second operator matrix, a third operator matrix, and a fourth operator matrix; The step of extracting symmetrical feature points from the augmented image using a preset convolution operator to obtain the pixel positions of the four corner points near the center of the marker includes: The first operator matrix, the second operator matrix, the third operator matrix, and the fourth operator matrix are respectively convolved with the pixel value of each pixel in the expanded image to obtain the first convolution value, the second convolution value, the third convolution value, and the fourth convolution value of each pixel. The pixel with the largest first convolution value is determined as the first corner point of the marker, and the pixel position of the first corner point in the augmented image is obtained; The pixel with the largest second convolution value is determined as the second corner point of the marker, and the pixel position of the second corner point in the augmented image is obtained; The pixel with the largest third convolution value is determined as the third corner point of the marker, and the pixel position of the third corner point in the augmented image is obtained; The pixel with the largest fourth convolution value is determined as the fourth corner point of the marker, and the pixel position of the fourth corner point in the augmented image is obtained.

3. The method according to claim 2, characterized in that, Determining the pixel position of the marker in the expanded image based on the pixel positions of symmetrical feature points on the marker in the expanded image includes: Determine the quadrilateral formed by the first corner point, the second corner point, the third corner point, and the fourth corner point; The pixel position of the geometric center of the quadrilateral is obtained based on the pixel positions of the first corner point, the second corner point, the third corner point, and the fourth corner point, and the pixel position of the geometric center of the quadrilateral is determined as the pixel position of the marker in the expanded image.

4. The method according to claim 1, characterized in that, The step of restoring the pixel position of the marker in the augmented image to determine the pixel position of the marker in the image to be detected includes: The pixel position of the marker in the expanded image is restored by using the pixel processing multiplier of the increased pixel quantity to obtain the pixel position of the marker in the target image; The pixel position of the marker in the image to be detected is determined based on the pixel position of the marker in the target image and the relative positional relationship between the target image and the image to be detected.

5. The method according to any one of claims 1-4, characterized in that, Determining the target image including the marker in the image to be detected includes: A marker detection model is used to detect and segment markers in the image to be detected, thereby obtaining the initial region of the marker in the image to be detected; wherein, the marker detection model is a network model trained based on marker samples; The target image is determined based on the initial region.

6. The method according to claim 5, characterized in that, Determining the target image based on the initial region includes: The size of the initial region is adjusted to form a target region with the smallest possible area, including the marker. The target region is extracted from the image to be detected to obtain the target image.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images