Visual calibration method, device, apparatus and storage medium

Abstract

The application relates to the field of machine vision, and discloses a vision calibration method, device, equipment and storage medium, comprising the following steps: setting a calibration plate based on sampler information, collecting a calibration plate image corresponding to the calibration plate, determining pixel coordinates of feature points in the calibration plate, calculating calibration parameters based on a preset calibration model and the pixel coordinates of the feature points, and performing vision calibration on a vision system of the sampler based on the calibration parameters; since the application realizes automatic calibration of the vision system through image collection, feature point determination and calibration parameter calculation, manual intervention is reduced, algorithm optimization is added in the steps of feature point determination and calibration parameter calculation, and therefore the system robustness and anti-interference ability are improved, and the work efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of machine vision, and more particularly to a visual calibration method, apparatus, device, and storage medium. Background Technology

[0002] Under the current technological background, visual calibration relies heavily on manual labor throughout the entire process, which cannot adapt to the working conditions of grain depots, ports, etc., which are characterized by high dust, strong vibration, and complex lighting. This results in low calibration accuracy, poor robustness, and the need for frequent recalibration, leading to low work efficiency and poor calibration results. Summary of the Invention

[0003] The main purpose of this application is to provide a visual calibration method, apparatus, equipment and storage medium, which aims to solve the technical problems of related visual calibration methods that rely heavily on manual labor throughout the process, cannot adapt to the working conditions of grain depots, ports and other places with high dust, strong vibration and complex lighting, resulting in low calibration accuracy, poor robustness, and the need for frequent recalibration, resulting in low work efficiency and poor calibration effect.

[0004] To achieve the above objectives, this application provides a visual calibration method, the visual calibration method comprising: The calibration plate is set based on the sampler information, and the calibration plate image corresponding to the calibration plate is acquired; Determine the pixel coordinates of feature points in the calibration board based on the calibration board image; The calibration parameters are calculated based on the preset calibration model and the pixel coordinates of the feature points, and the vision system of the sampler is visually calibrated based on the calibration parameters.

[0005] Optionally, the step of determining the pixel coordinates of feature points in the calibration board based on the calibration board image includes: Based on the calibration plate image, determine the initial pixel coordinates of the feature points; Centered on the initial pixel coordinates, collect grayscale distribution information within a preset range; The initial pixel coordinates of the feature points are corrected based on the grayscale distribution information to obtain the pixel coordinates of the feature points.

[0006] Optionally, the step of calculating calibration parameters based on a preset calibration model and the pixel coordinates of the feature points includes: Input the pixel coordinates of the feature points into the preset calibration model to obtain the initial calibration parameters and the reprojection coordinates of the feature points; The reprojection error of the feature point is obtained based on the pixel coordinates and the reprojection coordinates of the feature point. Based on the reprojection error, the initial calibration parameters are adjusted to obtain the calibration parameters.

[0007] Optionally, the step of adjusting the initial calibration parameters based on the reprojection error to obtain the calibration parameters includes: Based on the pixel coordinates of the feature points and the reprojection coordinates, obtain the position information for reprojecting the feature points back to the image plane; Based on the location information, obtain the image effect after the vision system has been corrected by applying calibration parameters; Based on the image effect, a visual interface is generated.

[0008] Optionally, after the step of visually calibrating the vision system of the sampling machine according to the calibration parameters, the method further includes: The calibration parameters are stored in a standard format as preset calibration parameters; Based on the preset calibration parameters and the sampler information, the preset calibration parameters are adjusted according to the adjustment rules to obtain the adjusted target calibration parameters. The target calibration parameters are the calibration parameters for the next calibration of the sampler vision system.

[0009] Optionally, the step of setting the calibration plate based on the sampler information and acquiring the calibration plate image corresponding to the calibration plate includes: The calibration plate is set based on the sampler information, and the sampler image is acquired after the calibration plate is set; The sampler image is preprocessed to obtain a preprocessed image; The calibration board image is obtained from the preprocessed image based on the calibration board information.

[0010] Optionally, the step of obtaining the calibration board image from the preprocessed image based on the calibration board information includes: Based on the calibration plate information, obtain the contour information and brightness information in the preprocessed image; Based on the contour information and the brightness information, the calibration board image corresponding to the calibration board is acquired.

[0011] Furthermore, to achieve the above objectives, this application also proposes a visual calibration device, which includes: The acquisition module is used to set the calibration plate based on the sampler information and acquire the calibration plate image corresponding to the calibration plate; The determination module is used to determine the pixel coordinates of feature points in the calibration board based on the calibration board image; The calibration module is used to calculate calibration parameters based on a preset calibration model and the pixel coordinates of the feature points, and to perform visual calibration of the vision system of the sampler according to the calibration parameters.

[0012] In addition, to achieve the above objectives, this application also proposes a visual calibration device, which includes a memory, a processor, and a visual calibration program stored in the memory and executable on the processor, the visual calibration program being configured to implement the visual calibration method as described above.

[0013] In addition, to achieve the above objectives, this application also proposes a storage medium storing a visual calibration program, which, when executed by a processor, implements the visual calibration method as described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a visual calibration program that, when executed by a processor, implements the visual calibration method as described above.

[0015] This application discloses a method for setting a calibration plate based on sampler information, acquiring an image of the calibration plate corresponding to the calibration plate, determining the pixel coordinates of feature points in the calibration plate, calculating calibration parameters based on a preset calibration model and the pixel coordinates of the feature points, and performing visual calibration of the sampler's vision system based on the calibration parameters. Because this application achieves automated calibration of the vision system through image acquisition, feature point determination, and calibration parameter calculation, it reduces manual intervention. Furthermore, by incorporating algorithm optimization in the feature point determination and calibration parameter calculation stages, it improves system robustness and anti-interference capabilities, thereby increasing operational efficiency. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the first embodiment of the visual calibration method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the visual calibration method of this application; Figure 3 This is a flowchart illustrating the third embodiment of the visual calibration method of this application; Figure 4 This is a flowchart illustrating an example of the visual calibration method of this application; Figure 5A schematic diagram illustrating the sub-pixel corner point extraction and distortion correction effect of a black and white checkerboard pattern; Figure 6 This is a schematic diagram of the module structure of the visual calibration device according to an embodiment of this application; Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the visual calibration method in the embodiments of this application.

[0019] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0021] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] This application provides a visual calibration method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the visual calibration method of this application.

[0023] In the first embodiment, the visual calibration method includes: Step S10: Set up the calibration plate based on the sampler information and acquire the calibration plate image corresponding to the calibration plate.

[0024] It should be understood that the execution subject of this embodiment may be a visual calibration device with data processing, network communication and program running functions, such as a sampler, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this.

[0025] It should be noted that the sampling machine information includes, but is not limited to, the location of the vehicle-mounted camera, the working environment of the sampling machine, and the sampling location in the vehicle compartment.

[0026] It should be noted that the calibration board is a standard reference object specifically used in the field of machine vision. Its surface is printed with regular, high-precision geometric patterns and is used for solving camera imaging parameters, correcting image distortion, and calibrating the pose of vision systems.

[0027] In specific implementation, a calibration plate is fixedly or temporarily placed in the working area of ​​the sampling machine. This plate can be a standard checkerboard calibration plate, a dot calibration plate, a coded calibration plate, etc., and this embodiment does not impose any restrictions on this. The sampling machine controls the mounted camera to be calibrated to take images of the calibration plate, including the complete or partially visible calibration plate, from multiple different distances and angles. At the same time, the main area of ​​the camera site should be covered during the shooting process.

[0028] Furthermore, in order to achieve the effect of adapting to complex working conditions, the step of setting a calibration plate based on the sampler information and acquiring the calibration plate image corresponding to the calibration plate includes: setting a calibration plate based on the sampler information and acquiring the sampler image after setting the calibration plate; preprocessing the sampler image to obtain a preprocessed image; and obtaining the calibration plate image from the preprocessed image based on the calibration plate information.

[0029] In the specific implementation, a calibration plate is set up in the working area of ​​the sampling machine, the sampling machine vehicle is controlled to move, the vehicle-mounted camera autonomously acquires images of the sampling machine, and the obtained sampling machine images are processed, including but not limited to mechanisms for image optimization such as integrated noise suppression, sunlight compensation, occlusion adaptation mechanism and optional RANSAC strategy. This embodiment does not limit this, and the on-site working conditions are understood based on the sampling machine information, and the calibration plate image is obtained from the preprocessed image.

[0030] Furthermore, in order to improve the accuracy of identifying the image corresponding to the calibration board, the step of obtaining the calibration board image from the preprocessed image based on the calibration board information includes: obtaining contour information and brightness information in the preprocessed image based on the calibration board information; and acquiring the calibration board image corresponding to the calibration board based on the contour information and the brightness information.

[0031] It should be noted that the outline information can be the edge lines of all objects in the image or the right angles in the image; this embodiment does not limit this.

[0032] In the specific implementation, after obtaining the preprocessed image, the on-site working conditions are understood based on the sampler information, and the bright and dark areas in the image are distinguished. Then, the brightness changes in the image are obtained as brightness information. Combined with the contour information and brightness information, the calibration plate image corresponding to the calibration plate is acquired. For example, the sampler information identifies the sampler working area as a dusty grain factory. After performing appropriate preprocessing on the image, the brightness information can be extracted using the grayscale method or the HSV color space V channel brightness extraction algorithm. This embodiment does not limit this. At the same time, the edge lines of all objects in the image are extracted as contour information. The calibration plate area is identified by combining the contour information and brightness information, and the calibration plate image corresponding to the calibration plate is acquired.

[0033] Step S20: Determine the pixel coordinates of the feature points in the calibration board based on the calibration board image.

[0034] It should be noted that feature points are special pixels in an image that exhibit significant changes in grayscale, texture, and contour.

[0035] Pixel coordinates are coordinate values ​​in a two-dimensional coordinate system with the top-left corner of the image as the origin and pixels as the unit. They can be represented by (u, v). u represents the number of pixel columns in the horizontal direction, and v represents the number of pixel rows in the vertical direction. They can describe the position of the target point within the digital image.

[0036] In the specific implementation, the acquired calibration board image is recognized, a two-dimensional coordinate system is established within the recognized calibration board area, feature points are detected, and the pixel coordinates of the feature points are extracted.

[0037] Step S30: Calculate calibration parameters based on the preset calibration model and the pixel coordinates of the feature points, and perform visual calibration on the vision system of the sampler according to the calibration parameters.

[0038] It should be noted that the calibration parameters are a set of camera imaging parameters, spatial pose parameters, and distortion parameters calculated by the calibration model in combination with the feature point pixel coordinates and the known physical dimensions of the calibration board.

[0039] Camera imaging parameters, also known as camera intrinsic parameters or simply camera intrinsic parameters, are inherent parameters that characterize the optical imaging characteristics of a device. They may include focal length, principal point coordinates, and pixel skew coefficient.

[0040] Spatial pose parameters, also known as external pose parameters, consist of rotation matrices and translation vectors. They describe the three-dimensional spatial position and orientation of the camera coordinate system relative to the mechanical world coordinate system of the prototyping machine. Once the camera moves or deflects, these parameters will change synchronously.

[0041] Distortion parameters can include radial distortion coefficients and tangential distortion coefficients, which are used to compensate for image distortion caused by lens optical deformation and assembly deviations, and restore the true image.

[0042] In the specific implementation, the imaging geometry and imaging relationship are described based on the preset calibration model, and then the calibration parameters are calculated based on the pixel coordinates of the feature points. The vision system of the sampler is then visually calibrated based on the obtained calibration parameters.

[0043] Furthermore, to facilitate the rapid loading and use of the prototyping machine's vision system upon startup, and to dynamically adjust the calibration parameters based on on-site information, thereby avoiding frequent recalibration, enhancing scene adaptability, and improving operational efficiency, the step of visually calibrating the prototyping machine's vision system based on the calibration parameters further includes: storing the calibration parameters in a standard format as preset calibration parameters; adjusting the preset calibration parameters according to adjustment rules based on the preset calibration parameters and the prototyping machine information to obtain the adjusted target calibration parameters, which are the calibration parameters for the next calibration of the prototyping machine's vision system.

[0044] It should be noted that the standard format is a structured text format commonly used in the field of machine vision, including XML, YAML, JSON, etc.

[0045] In practice, the obtained calibration parameters are stored as preset calibration parameters in a standard format (such as YAML). When the sampler is started again, sampler information is collected, and the preset calibration parameters are adjusted according to preset rules based on the sampler information. For example, if the camera position changes, the calibration parameters are adjusted through the corresponding algorithm to obtain the adjusted target calibration parameters, which are used as the calibration parameters for the next calibration of the sampler vision system.

[0046] This embodiment automates the entire process from image acquisition and feature point determination to calibration parameter calculation, reducing manual intervention. Furthermore, by incorporating algorithm optimizations in the feature point determination and calibration parameter calculation stages, it enhances the system's robustness and anti-interference capabilities, thereby improving operational efficiency.

[0047] Reference Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the visual calibration method of this application, based on the above. Figure 1 The first embodiment shown presents a second embodiment of the visual calibration method of this application.

[0048] In the second embodiment, step S20 includes: Step S201: Determine the initial pixel coordinates of the feature points based on the calibration plate image.

[0049] Step S202: Using the initial pixel coordinates as the center, collect grayscale distribution information within a preset range.

[0050] Step S203: Correct the initial pixel coordinates of the feature points based on the grayscale distribution information to obtain the pixel coordinates of the feature points.

[0051] It should be noted that an image is composed of countless pixels, and grayscale represents the brightness of a pixel. Grayscale distribution information is a collection of the overall distribution characteristics, gradient changes, and regional differences in grayscale values ​​of each pixel in a digital image. Grayscale values ​​typically range from 0 to 255 (0 represents pure black, and 255 represents pure white), and can intuitively reflect the contrast, edge, and contour features of different areas of the image.

[0052] It should be understood that the correction of pixel coordinates based on grayscale distribution information can be achieved through grayscale moment subpixel algorithms, edge fitting subpixel algorithms, or other information and algorithms that can achieve fine correction of pixel coordinates. This embodiment does not limit this.

[0053] In the specific implementation, the acquired calibration board image is identified, a two-dimensional coordinate system is established within the identified calibration board area, the initial pixel coordinates of the feature points are extracted, and gray-level distribution information within a preset range is collected with the obtained initial pixel coordinates as the center. The gray-level distribution information, especially the gray-level values, is analyzed. The position where the gray-level value changes sharply and forms a clear gradient is used as the pixel coordinates of the corrected feature point.

[0054] This embodiment collects grayscale distribution information and performs fine-tuning on the pixel coordinates of feature points within the calibration board, breaking through the integer pixel-level positioning accuracy and obtaining high-precision sub-pixel-level feature point coordinates, thus achieving a high-precision effect.

[0055] Reference Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the visual calibration method of this application. Based on the above embodiments, the third embodiment of the visual calibration method of this application is proposed.

[0056] In the third embodiment, the calculation of calibration parameters based on the preset calibration model and the pixel coordinates of the feature points includes: Step S301: Input the pixel coordinates of the feature points into the preset calibration model to obtain the initial calibration parameters and the reprojection coordinates of the feature points.

[0057] It should be noted that reprojection coordinates are the theoretical pixel positions obtained by mapping the real physical coordinates of feature points to the pixel coordinate system based on camera imaging parameters, spatial pose parameters, and distortion parameters.

[0058] In the specific implementation, based on Zhang Zhengyou's calibration method, the imaging geometry is described using a pinhole camera model, and the specific formula is as follows:

[0059] In the formula, the intrinsic parameter matrix K includes the horizontal and vertical equivalent focal lengths. , and principal pixel coordinates , .

[0060] Under ideal distortion-free conditions, the imaging relationship is as follows:

[0061] In the formula, u = [u, v, 1] represents the pixel coordinates of a feature point on the image, R is the rotation matrix, and t is the translation vector. These are the actual physical coordinates of the feature points on the calibration board.

[0062] The normalized camera coordinates (x, y) of the pixel coordinates under ideal conditions are calculated.

[0063] Actual lenses exhibit distortion. The Brown-Conrady distortion model is used to fit the lens distortion. First, the true physical coordinates of the feature points on the calibration board are used. The homography matrix H is calculated using the measured pixel coordinates, as shown in the following formula:

[0064] By using matrix decomposition, the rotation matrix R and translation vector t of the camera relative to the calibration board at the current viewpoint are calculated sequentially, obtaining the initial values ​​of the camera's intrinsic and extrinsic parameters, and the distortion coefficients are calculated using the following formulas:

[0065]

[0066] In the formula, , ( , () are the distorted coordinates. The distortion coefficient is denoted as .

[0067] The optimal calibration parameters are then obtained through nonlinear optimization. Finally, based on the complete set of calibration parameters, forward projection calculation is performed to obtain the reprojection coordinates of the feature points.

[0068] Step S302: Obtain the reprojection error of the feature point based on the pixel coordinates of the feature point and the reprojection coordinates.

[0069] Step S303: Based on the reprojection error, adjust the initial calibration parameters to obtain the calibration parameters.

[0070] Step S304: Perform visual calibration on the vision system of the sampler according to the calibration parameters.

[0071] In the specific implementation, based on the pixel coordinates (u,v) and reprojection coordinates of the feature points (... , Reprojection error rooted Then, a reprojection error objective function is constructed for all detected corner points in all images. The Levenberg-Marquardt (LM) algorithm or a similar iterative optimization algorithm can be used for global optimization. This embodiment does not limit this approach. The goal is to minimize the root mean square value of the overall reprojection error, obtain the optimal calibration parameters, and perform visual calibration on the vision system of the sampler.

[0072] Furthermore, in order to facilitate intuitive verification of calibration quality by users, after the step of adjusting the initial calibration parameters based on the reprojection error to obtain the calibration parameters, the method further includes: obtaining position information for reprojecting the feature points back to the image plane based on the pixel coordinates of the feature points and the reprojection coordinates; obtaining the image effect after the vision system applies the calibration parameters for correction based on the position information; and generating a visualization interface based on the image effect.

[0073] It should be noted that reprojection error refers to the deviation in the distribution of reprojection coordinates and feature point pixel coordinates on the image.

[0074] In the specific implementation, based on the pixel coordinates and reprojection coordinates of the feature points, the position information of the feature points being reprojected back to the image plane is obtained, and the image effect after the vision system applies calibration parameters is obtained, and a visualization interface is generated.

[0075] This embodiment calculates the reprojection error by comparing the reprojection coordinates with the extracted pixel coordinates, thereby verifying the accuracy of the calibration results and optimizing the parameters, thus improving the accuracy of the calibration parameters.

[0076] For ease of understanding, please refer to Figure 4 and Figure 5 Examples are provided for illustration only, but are not intended to limit this application. Figure 4 This is a flowchart illustrating an example of the visual calibration method of this application. Figure 5 This is a schematic diagram illustrating the corner point extraction and distortion correction effect of sub-pixel grids in a black and white checkerboard pattern.

[0077] As an example, firstly, an 8×6 standard black and white checkerboard calibration plate with 30mm individual squares is fixed in place. The sampling management software sends a start signal to the sampling machine. Subsequently, the sampling machine autonomously controls the vehicle-mounted camera to change its pitch, yaw angles, and altitude within a range of 1–3 meters, automatically acquiring 15–20 images. Then, the determination module automatically loads the images and preprocesses them. Next, a robust algorithm is used to complete checkerboard detection and extract sub-pixel corner points. Then, the initial values ​​of the camera parameters are solved using the Zhang calibration method, and then nonlinear optimization is performed using the LM algorithm to control the root mean square value of the corner reprojection error to within 0.25 pixels, obtaining the optimized calibration parameters. Figure 5As shown, the white cross-marked points in the image are the sub-pixel corner points of the checkerboard grid extracted by the algorithm. The white lines in the image are divided into horizontal lines connecting corner points in the same row and diagonal lines contrasting across rows. Due to the inherent distortion of the lens, the parallel horizontal lines of the checkerboard grid in the real object will bend and shift when directly imaged. After distortion correction by calibration parameters, all fitted lines are restored to straight lines that are parallel to each other in the image, which can intuitively verify the algorithm's correction effect on radial and tangential distortion of the lens. The system combines the extracted sub-pixel corner point coordinates, the corner point reprojection position coordinates calculated by calibration parameters, and the distortion-corrected image to generate a visualization interface. After completion, staff can view the corner detection and distortion correction effects through a visual interface. Then, the calibration parameters (camera intrinsics, distortion coefficients) are saved as a YAML file for use by the sampling locomotive navigation and positioning modules, as well as for the next vision system calibration. The output vehicle information is then returned to the sample management software for sampling processing. During subsequent sampling operations, images of fixed markers with known spacing on the robotic arm can be taken, and the calibration parameters can be fine-tuned by calling the refinement interface to compensate for parameter drift caused by vibration and temperature, ensuring the long-term visual positioning accuracy of the sampling machine and achieving parameter persistence.

[0078] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the visual labeling method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0079] This application also provides a visual calibration device, please refer to... Figure 6 The visual calibration device includes: The acquisition module 10 is used to set the calibration plate based on the sampler information and acquire the calibration plate image corresponding to the calibration plate; The determination module 20 is used to determine the pixel coordinates of feature points in the calibration board based on the calibration board image; The calibration module 30 is used to calculate calibration parameters based on a preset calibration model and the pixel coordinates of the feature points, and to perform visual calibration on the vision system of the sampler according to the calibration parameters.

[0080] The visual calibration device provided in this application, employing the visual calibration method described in the above embodiments, solves the technical problems of related visual calibration methods being highly dependent on manual labor throughout the process. These methods are unsuitable for working conditions such as grain depots and ports, which involve high dust levels, strong vibrations, and complex lighting, resulting in low calibration accuracy, poor robustness, and the need for frequent re-calibration, leading to low operational efficiency and poor calibration results. Compared with the prior art, the beneficial effects of the visual calibration device provided in this application are the same as those of the visual calibration method provided in the above embodiments, and other technical features in the visual calibration device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0081] This application provides a visual calibration device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the visual calibration method in Embodiment 1 above.

[0082] The following is for reference. Figure 7 The diagram illustrates a structural schematic suitable for implementing the visual calibration device in the embodiments of this application. The visual calibration device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The visual calibration device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0083] like Figure 7 As shown, the vision calibration device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into RAM (Random Access Memory) 1004. RAM 1004 also stores various programs and data required for the operation of the vision calibration device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touch screens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, LCDs (Liquid Crystal Displays), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the vision calibration device to communicate wirelessly or wiredly with other devices to exchange data. Although vision calibration devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0084] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0085] The visual calibration device provided in this application, employing the visual calibration method described in the above embodiments, solves the technical problems of related visual calibration methods being highly dependent on manual labor throughout the process. These methods are unsuitable for working conditions such as grain depots and ports, which involve high dust levels, strong vibrations, and complex lighting, resulting in low calibration accuracy, poor robustness, and the need for frequent re-calibration, leading to low operational efficiency and poor calibration results. Compared with the prior art, the beneficial effects of the visual calibration device provided in this application are the same as those of the visual calibration method provided in the above embodiments, and other technical features of this visual calibration device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0086] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0087] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0088] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the visual calibration method in the above embodiments.

[0089] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), or flash memory, optical fiber, CD-ROM (CD-Read Only Memory), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0090] The aforementioned computer-readable storage medium may be included in the vision calibration device; or it may exist independently and not assembled into the vision calibration device.

[0091] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the vision calibration device, cause the vision calibration device to perform the aforementioned vision calibration method.

[0092] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including LAN (Local Area Network) or WAN (Wide Area Network)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0093] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0094] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0095] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the above-described visual calibration method. This solves the technical problems of related visual calibration methods, which are highly dependent on manual labor throughout the process, cannot adapt to complex working conditions such as dust, strong vibration, and lighting in grain depots and ports, resulting in low calibration accuracy, poor robustness, and the need for frequent re-calibration, leading to low work efficiency and poor calibration results. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the visual calibration method provided in the above embodiments, and will not be repeated here.

[0096] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the visual calibration method as described above.

[0097] The computer program product provided in this application solves the technical problems of related visual calibration methods, which rely heavily on manual labor throughout the process, cannot adapt to the complex working conditions of grain depots, ports, etc., with high dust levels, strong vibrations, and complex lighting, resulting in low calibration accuracy, poor robustness, and the need for frequent recalibration, leading to low work efficiency and poor calibration results. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the visual calibration method provided in the above embodiments, and will not be repeated here.

[0098] The above description is only a part of the embodiments of this application and does not limit the scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.

[0099] It should be noted that the data collection, tag management, rule setting, and push decision-making processes involved in this application are designed to work with other technical features to solve technical problems. They do not involve or support any illegal activities. Any data processing that may violate laws and regulations (such as unauthorized collection of privacy data, generation of discriminatory tags, setting unfair rules, or pushing illegal information) is not within the scope of protection of this application's technical solution. Of course, the user data in this application will be encrypted, anonymized, or de-identified before storage to ensure user data security.

Claims

1. A visual calibration method, characterized in that, The visual calibration method includes: The calibration plate is set based on the sampler information, and the calibration plate image corresponding to the calibration plate is acquired; Determine the pixel coordinates of feature points in the calibration board based on the calibration board image; The calibration parameters are calculated based on the preset calibration model and the pixel coordinates of the feature points, and the vision system of the sampler is visually calibrated according to the calibration parameters.

2. The visual calibration method as described in claim 1, characterized in that, The step of determining the pixel coordinates of feature points in the calibration board based on the calibration board image includes: Based on the calibration plate image, determine the initial pixel coordinates of the feature points; Centered on the initial pixel coordinates, collect grayscale distribution information within a preset range; The initial pixel coordinates of the feature points are corrected based on the grayscale distribution information to obtain the pixel coordinates of the feature points.

3. The visual calibration method as described in claim 1, characterized in that, The step of calculating calibration parameters based on the preset calibration model and the pixel coordinates of the feature points includes: Input the pixel coordinates of the feature points into the preset calibration model to obtain the initial calibration parameters and the reprojection coordinates of the feature points; The reprojection error of the feature point is obtained based on the pixel coordinates and the reprojection coordinates of the feature point. Based on the reprojection error, the initial calibration parameters are adjusted to obtain the calibration parameters.

4. The visual calibration method as described in claim 3, characterized in that, After the step of adjusting the initial calibration parameters based on the reprojection error to obtain the calibration parameters, the method further includes: Based on the pixel coordinates of the feature points and the reprojection coordinates, obtain the position information for reprojecting the feature points back to the image plane; Based on the location information, obtain the image effect after the vision system has been corrected by applying calibration parameters; Based on the image effect, a visual interface is generated.

5. The visual calibration method as described in claim 1, characterized in that, After the step of visually calibrating the vision system of the sampling machine according to the calibration parameters, the method further includes: The calibration parameters are stored in a standard format as preset calibration parameters; Based on the preset calibration parameters and the sampler information, the preset calibration parameters are adjusted according to the adjustment rules to obtain the adjusted target calibration parameters. The target calibration parameters are the calibration parameters for the next calibration of the sampler vision system.

6. The visual calibration method as described in claim 1, characterized in that, The steps of setting a calibration plate based on the sampler information and acquiring the corresponding calibration plate image include: The calibration plate is set based on the sampler information, and the sampler image is acquired after the calibration plate is set; The sampler image is preprocessed to obtain a preprocessed image; The calibration board image is obtained from the preprocessed image based on the calibration board information.

7. The visual calibration method as described in claim 6, characterized in that, The step of obtaining the calibration board image from the preprocessed image based on the calibration board information includes: Based on the calibration plate information, obtain the contour information and brightness information in the preprocessed image; Based on the contour information and the brightness information, the calibration board image corresponding to the calibration board is acquired.

8. A visual calibration device, characterized in that, The visual calibration device includes: The acquisition module is used to set the calibration plate based on the sampler information and acquire the calibration plate image corresponding to the calibration plate; The determination module is used to determine the pixel coordinates of feature points in the calibration board based on the calibration board image; The calibration module is used to calculate calibration parameters based on a preset calibration model and the pixel coordinates of the feature points, and to perform visual calibration of the vision system of the sampler according to the calibration parameters.

9. A visual calibration device, characterized in that, The visual calibration device includes: a memory, a processor, and a visual calibration program stored in the memory and executable on the processor, wherein the visual calibration program, when executed by the processor, implements the visual calibration method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a visual calibration program, which, when executed by a processor, implements the visual calibration method as described in any one of claims 1 to 7.

Images