Gray board region identification method and device, electronic equipment and storage medium

Abstract

The present application relates to the technical field of multispectral camera, and discloses a gray board area recognition method and device, electronic equipment and storage medium, comprising obtaining the first position relationship between the first image corresponding to the first camera, the marker plate coordinate system and the gray board coordinate system, and the first position information of the gray board in the gray board coordinate system, identifying the identification information and pixel information of the candidate positioning code through the pre-constructed positioning code recognition algorithm, and then obtaining the physical coordinates of the positioning code in the marker plate coordinate system, determining the second position relationship between the first camera coordinate system and the marker plate coordinate system according to the pixel coordinate information and the physical coordinate information corresponding to the candidate positioning code, and determining the pixel area of the gray board in the first image according to the first position information, the first position relationship and the second position relationship. In this way, the recognition of the gray board area can be automatically completed, the design of multiple positioning codes makes the application have strong anti-shielding ability, and the gray board recognition efficiency and recognition robustness are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method, apparatus, electronic device, and storage medium for identifying grayscale areas. Background Technology

[0002] In industrial vision and automated inspection, such as in agricultural and environmental remote sensing, surveying and mapping, to ensure the radiometric calibration accuracy of images and calculate the true reflectance, it is usually necessary to place a standard gray card in the shooting area and accurately identify and extract the gray card region to establish a stable radiometric reference benchmark. In related technologies, obtaining the gray card region generally relies on manual selection, but this manual operation is cumbersome, resulting in low recognition efficiency, and manual boundary selection is prone to introducing significant errors. Summary of the Invention

[0003] This invention provides a method, apparatus, electronic device, storage medium, and program product for identifying gray areas, in order to solve the problem of low efficiency in identifying gray areas.

[0004] In a first aspect, the present invention provides a method for identifying a gray area. This method is applied to an image processing system, which includes a rigid plate, at least one camera, and an image processing device. The rigid plate is provided with a gray area and a marker plate, and the marker plate is provided with multiple positioning codes. The method is executed by the image processing device and includes: Obtain the first positional relationship between the first image corresponding to the first camera, the marker coordinate system, and the gray plate coordinate system, as well as the first position information of the gray plate in the gray plate coordinate system, wherein the first camera is any one of at least one cameras; A pre-built location code recognition algorithm is used to identify at least one candidate location code from the first image and obtain the identification information and pixel coordinate information corresponding to the at least one candidate location code respectively. The candidate location code is the location code that is not occluded in the first image. Based on the identification information of at least one candidate positioning code, obtain the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system; Based on the pixel coordinate information and physical coordinate information corresponding to at least one candidate positioning code, a second positional relationship between the first camera coordinate system and the marker plate coordinate system is determined, wherein the first camera coordinate system is the camera coordinate system of the first camera; Based on the first position information, the first position relationship, and the second position relationship, the pixel region of the gray card in the first image is determined.

[0005] The gray area identification method provided in this invention allows for automatic gray area identification by an image processing device, eliminating the need for manual identification, resulting in high efficiency and accuracy. Furthermore, the marking plate in this solution is equipped with multiple positioning codes. Even if one positioning code is obscured, other positioning codes can be used for gray area identification, demonstrating strong anti-occlusion capabilities and improving the efficiency and accuracy of gray area identification.

[0006] In one optional implementation, determining the pixel region of the gray card in the first image based on the first position information, the first positional relationship, and the second positional relationship includes: Based on the first position information and the first position relationship, determine the second position information of the gray board in the coordinate system of the marking board; Based on the second position information and the second position relationship, determine the position information of the gray board in the camera coordinate system; After performing a projection operation on the position information of the gray board in the first camera coordinate system, the pixel area of ​​the gray board in the first image is obtained.

[0007] The gray area recognition method provided in this invention determines the position of the gray card in the camera coordinate system by using its position information in the marker coordinate system, and then projects it to identify the pixel area of ​​the gray card in the image. This process only requires simple coordinate system transformations to automatically identify the gray card area, eliminating the need for manual selection and improving recognition accuracy and efficiency.

[0008] In one optional implementation, acquiring the first image corresponding to the first camera includes: Acquire the initial image captured by the first camera, and the shooting angle of the first camera when capturing the initial image; When the shooting angle is determined to be greater than a preset angle threshold, perspective transformation is performed on the initial image to obtain the first image. Alternatively, if the shooting angle is determined to be less than or equal to a preset angle threshold, the initial image is determined as the first image.

[0009] The gray area recognition method provided in this embodiment of the invention can avoid perspective distortion caused by large-angle shooting by selectively performing perspective transformation on large-angle captured images, thereby improving recognition efficiency and accuracy.

[0010] In one optional implementation, the pixel coordinate information includes the pixel coordinates of multiple corner points, and the physical coordinate information includes the physical coordinates of multiple corner points. Based on the pixel coordinate information and physical coordinate information corresponding to at least one candidate positioning code, a second positional relationship between the first camera coordinate system and the marker plate coordinate system is determined, including: By employing a pre-constructed multi-point optimal fitting algorithm, the second positional relationship is obtained after performing fitting operations on the pixel coordinates and physical coordinates of multiple corner points.

[0011] The gray area identification method provided in this embodiment of the invention can use a multi-point optimal fitting algorithm to fit the coordinates of multiple corner points when they are identified. This can make full use of redundant information, reduce the impact of noise, and improve the reliability and robustness of coordinate system transformation results.

[0012] In an alternative implementation, when there are multiple cameras, the method further includes: Determine the actual number of candidate location codes; If the actual quantity is greater than or equal to the preset quantity threshold, then based on the identification information of at least one candidate positioning code, obtain the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system.

[0013] The gray area recognition method provided in this embodiment of the invention can quickly filter out valid images in multi-camera scenarios by judging the number threshold of candidate positioning codes, avoiding the problem of gray area recognition failure due to insufficient valid values, improving the efficiency and reliability of the multi-camera recognition process, making the obtained coordinate information more reliable, and thus improving the recognition accuracy of gray area.

[0014] In one alternative implementation, the method further includes: If the actual number is less than a preset threshold, the first image is discarded.

[0015] The gray area recognition method provided in this embodiment of the invention automatically discards images when the number of valid positioning codes is insufficient, preventing them from entering subsequent processes. This can improve recognition efficiency, as well as the accuracy and reliability of the recognition results.

[0016] In one alternative implementation, the method further includes: After determining the pixel region of the gray card in the second image, based on the second positional relationship and the third positional relationship between the second camera coordinate system and the marker coordinate system, the fourth positional relationship between the first camera coordinate system and the second camera coordinate system is determined. Here, the second camera is any one of the cameras other than the first camera among a plurality of cameras, the second image is the image corresponding to the second camera, and the second camera coordinate system is the camera coordinate system of the second camera. The target error is determined based on the fourth positional relationship, the pixel area of ​​the gray card in the first image, and the pixel area in the second image; If the target error is determined to be less than a preset error threshold, the pixel region of the gray card in the first image or the pixel region of the second image is determined as the final pixel region of the gray card.

[0017] The gray area recognition method provided in this embodiment of the invention can recognize gray areas from multiple perspectives by associating coordinate systems and controlling errors between multiple cameras. Within the allowable error range, it further improves the accuracy and reliability of gray area recognition.

[0018] Secondly, the present invention provides a gray area identification device, which is applied to an image processing system. The image processing system includes a rigid plate, at least one camera, and an image processing device. A gray plate and a marking plate are disposed on the rigid plate, and multiple positioning codes are disposed on the marking plate. The device is configured in the image processing device and includes: The acquisition module is used to acquire the first image corresponding to the first camera, the first positional relationship between the marker coordinate system and the gray plate coordinate system, and the first position information of the gray plate in the gray plate coordinate system, wherein the first camera is any one of at least one camera; The recognition module is used to identify at least one candidate location code from the first image using a pre-built location code recognition algorithm, and to obtain the identification information and pixel coordinate information corresponding to the at least one candidate location code respectively, wherein the candidate location code is the location code in the first image that is not occluded. The acquisition module is also used to acquire the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system based on the identification information of at least one candidate positioning code; The determining module is used to determine a second positional relationship between the first camera coordinate system and the marker coordinate system based on the pixel coordinate information and physical coordinate information corresponding to at least one candidate positioning code, wherein the first camera coordinate system is the camera coordinate system of the first camera; and to determine the pixel region of the gray card in the first image based on the first position information, the first positional relationship, and the second positional relationship.

[0019] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the gray area identification method of the first aspect or any corresponding embodiment described above.

[0020] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the gray area identification method of the first aspect or any corresponding embodiment thereof.

[0021] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the gray area identification method of the first aspect or any corresponding embodiment described above. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the first structure of the gray board area according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a second structure of the gray board area according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a third structure of the gray board area according to an embodiment of the present invention; Figure 5 This is a schematic flowchart of a first method for identifying gray area according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a second process for identifying gray area according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the third process of the gray board area identification method according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the fourth process of the gray board area identification method according to an embodiment of the present invention; Figure 9 This is a structural block diagram of a gray board area identification device according to an embodiment of the present invention; Figure 10 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] As an optional application scenario of this invention, such as Figure 1 As shown, the recognition system for the gray area may include a rigid plate, an image processing device, and at least one camera.

[0028] The image processing device can be a terminal device, server, etc. The camera can be a multispectral camera. The camera can be mounted on a mobile device to capture images of the rigid plate; the mobile device can be a drone. The camera is used to capture images of the rigid plate and transmit them to the image processing device. The image processing device is used to perform grayscale region recognition based on the images captured by the camera.

[0029] like Figure 2 As shown, a marking plate 101 and a gray plate 102 can be provided on the rigid plate 10. Their arrangement can be such that the marking plate 101 and the gray plate 102 are arranged side-by-side horizontally; or as shown in the diagram... Figure 3 As shown, the marker plate 101 can be offset relative to the gray plate 102; it can also be set as follows: Figure 4 As shown, the marker plate 101 and the gray plate 102 are arranged vertically. The only requirement is that the horizontal and vertical reference edges of the marker plate and the gray plate are parallel; their specific placement and relative spacing are not uniquely restricted. The three layouts described above are merely illustrative examples. The marker plate can be provided with multiple uniquely identifiable positioning codes 101a, meaning the marker plate and positioning codes 101a form a single unit, and the positioning codes 101a can be an array combination of ArUco codes, such as... Figure 2 As shown, the marking plate 101 is provided with positioning codes 101a. The number of positioning codes 101a is not limited, and can be as follows: Figure 2 The quantity distribution shown is 3 rows and 3 columns, but other types of quantity distributions (e.g., 4 rows and 4 columns) can also be used, depending on the actual needs. The marker board can be of different types, such as ArUco marker board, AprilTag marker board, ARTag marker board, or ChArUco hybrid marker board; no specific limitation is made here.

[0030] When performing radiometric calibration or parameter calibration of multispectral cameras, it is usually necessary to place a gray card in the shooting scene. By accurately extracting the effective pixel area of ​​the gray card in the image, a reliable reflectance reference value can be obtained, thereby improving the radiometric calibration accuracy of multispectral camera imaging and ensuring the authenticity and accuracy of subsequent imaging data. In related technologies, the gray card area is usually selected manually. This method is not only inefficient, but also susceptible to various factors, easily leading to large errors. Therefore, related technologies use images, or independent QR codes or barcodes for auxiliary calibration. However, this recognition method is easily affected by partial occlusion, shooting angle, and multispectral lens distortion. For example, if a QR code is partially obscured, recognition will fail. Based on this, this invention provides a gray card area recognition method that can automatically identify the gray card area, improving recognition efficiency, and has strong anti-occlusion capabilities and high robustness.

[0031] The gray area identification method in this embodiment of the invention can be applied to industrial vision and automated inspection, as well as to agriculture or environmental remote sensing, surveying and mapping, etc., and its application scenarios are not limited here.

[0032] According to an embodiment of the present invention, an embodiment of a method for identifying gray area is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] This embodiment provides a method for identifying gray area regions, which can be used in image processing devices. Figure 5 This is a flowchart of a gray area identification method according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps: Step S501: Obtain the first positional relationship between the first image corresponding to the first camera, the marker coordinate system, and the gray plate coordinate system, as well as the first position information of the gray plate in the gray plate coordinate system.

[0034] The first camera can be any one of at least one cameras.

[0035] Specifically, the cameras in the image processing system can be mounted on a mobile device, such as a drone. After the mobile device moves to a certain position above the rigid plate, the cameras mounted on it can take pictures of the rigid plate, acquiring images including the rigid plate. Taking the first camera as an example, after completing the shooting operation, the acquired first image can be transmitted to the image processing device for subsequent processing.

[0036] Referring to the above Figures 2 to 4 The positions between the marker plate and the gray plate can be preset. Therefore, the image processing device can pre-obtain the first positional relationship between the marker plate coordinate system and the gray plate coordinate system, such as the offset direction and offset amount of the two in one or more directions like x, y, and z. Figure 2 For example, there is a certain distance offset between the coordinate systems of the two in the x-direction. Taking the gray board as the reference, the offset direction of the marking board is the positive direction.

[0037] The first position information of the gray card in the gray card coordinate system can be pre-configured, and the image processing device can directly obtain the pre-stored first position information.

[0038] Step S502: Using a pre-built location code recognition algorithm, at least one candidate location code is identified from the first image, and the identification information and pixel coordinate information corresponding to the at least one candidate location code are obtained respectively.

[0039] The candidate localization code can be an unoccluded localization code in the first image. The localization code recognition algorithm can be an open-source computer vision library (OpenCV).

[0040] Specifically, the image processing device can input a first image and a predefined location code dictionary (e.g., an ArUco dictionary, consistent with the location codes on the marker board) into a location code recognition algorithm. The algorithm then uses the location code dictionary to recognize the location codes in the first image, identifying candidate location codes that can completely detect all four corner points, and outputting the corresponding identification information (ID) and pixel coordinate information for each candidate location code. The number of candidate location codes can be one or more, and the pixel coordinate information can be a list of corner points, with one identification information corresponding to one corner point in the list.

[0041] Step S503: Based on the identification information of at least one candidate positioning code, obtain the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system.

[0042] Specifically, referring to the aforementioned Figures 2 to 4 The position of each positioning code on the marker plate is preset. Generally, the image processing device can pre-store the physical coordinate information of each positioning code in the marker plate coordinate system. Therefore, after identifying a candidate positioning code, the physical coordinate information matching the identification information of the candidate positioning code can be retrieved.

[0043] Step S504: Determine the second positional relationship between the first camera coordinate system and the marker plate coordinate system based on the pixel coordinate information and physical coordinate information corresponding to at least one candidate positioning code.

[0044] Wherein, the first camera coordinate system is the camera coordinate system of the first camera.

[0045] Specifically, pixel coordinate information indicates the position of the candidate positioning code in the image, while physical coordinate information indicates the position of the candidate positioning code in the marker board. The image is captured by a camera under preset shooting parameters (e.g., camera intrinsic and extrinsic parameters), which are related to the camera coordinate system. Therefore, by calculating the relationship between the pixel coordinate information and physical coordinate information of the candidate positioning code, a second positional relationship between the first camera coordinate system and the marker board coordinate system can be determined. This second positional relationship can include a rotation matrix and a translation vector. For example, in determining the second positional relationship, algorithms such as Perspective-n-Points (PnP), bundle adjustment, iterative least squares fitting, Efficient Perspective-n-Point (EPnP), and Unified Perspective-n-Point (UPnP) can be used.

[0046] Step S505: Determine the pixel region of the gray card in the first image based on the first position information, the first position relationship, and the second position relationship.

[0047] Specifically, since the first position information is the position information of the gray board in the gray board coordinate system, the first position relationship is used to transform the coordinate system of the marker board and the coordinate system of the gray board, and the second position relationship is used to transform the coordinate system of the first camera and the coordinate system of the marker board. The ultimate goal of this scheme is to determine the pixel area of ​​the gray board in the first image. Therefore, the first position relationship and the second position relationship can be used to perform coordinate transformation operation on the first position information, transform the physical position information of the gray board into pixel coordinates in the first image, and then determine the pixel area of ​​the gray board in the first image based on the pixel coordinates.

[0048] The gray-board region recognition method provided in this embodiment first acquires a first image corresponding to a first camera, as well as a first positional relationship between the marker board coordinate system and the gray-board coordinate system, and the first positional information of the gray-board in the gray-board coordinate system. Then, using a pre-constructed positioning code recognition algorithm, at least one candidate positioning code is identified from the first image, and the identification information and pixel coordinate information corresponding to each candidate positioning code are acquired. Based on the identification information of each candidate positioning code, the physical coordinate information of each candidate positioning code in the pre-constructed marker board coordinate system is acquired. Based on the pixel coordinate information and physical coordinate information corresponding to each candidate positioning code, a second positional relationship between the first camera coordinate system and the marker board coordinate system is determined. Based on the first and second positional relationships, the pixel region of the gray-board in the first image is determined. Thus, during gray-board recognition, the image processing device can automatically identify the gray-board without manual identification, resulting in high efficiency and accuracy. Furthermore, multiple positioning codes are set on the marker board in this solution. Even if one positioning code is occluded, other positioning codes can be used to identify the gray-board region, meaning that this solution has strong anti-occlusion capabilities, improving the efficiency and accuracy of gray-board recognition.

[0049] This embodiment provides a method for identifying gray area regions, which can be used in electronic devices. Figure 6 This is a flowchart of a gray area identification method according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps: Step S601: Obtain the first positional relationship between the first image corresponding to the first camera, the marker coordinate system, and the gray plate coordinate system, as well as the first position information of the gray plate in the gray plate coordinate system.

[0050] The first camera is any one of at least one cameras.

[0051] Optionally, step S601 above includes: Step S6011: Obtain the initial image captured by the first camera, and the shooting angle of the first camera when capturing the initial image.

[0052] Specifically, the initial image captured by the first camera is obtained; see details below. Figure 5 Step S501 of the illustrated embodiment will not be described again here.

[0053] Additionally, when acquiring the initial image, the shooting angle of the first camera can be obtained. Specifically, this can be achieved by mounting an inertial measurement unit, tilt sensor, or gimbal encoder on the first camera. Alternatively, the camera's intrinsic parameters can be obtained through camera calibration, thereby calculating the camera's shooting angle, which can be the pitch angle.

[0054] Step S6012: If the shooting angle is determined to be greater than the preset angle threshold, the initial image is subjected to perspective transformation to obtain the first image.

[0055] Specifically, after obtaining the aforementioned shooting angle, the shooting angle can be compared with a preset angle threshold. If the shooting angle is greater than the preset angle threshold, it is determined that the initial image has obvious perspective distortion, and the initial image needs to be perspective transformed. The perspective transformation can be performed by using the positioning code corner points in the initial image as reference points, constructing a perspective transformation matrix based on the coordinates of the reference points, and using this transformation matrix to correct the perspective distortion of the initial image. The corrected image is then used as the first image.

[0056] Alternatively, in step S6013, if the shooting angle is determined to be less than or equal to a preset angle threshold, the initial image is determined as the first image.

[0057] Specifically, if the shooting angle is less than or equal to a preset angle threshold, the initial image is directly used as the first image.

[0058] In this way, by identifying the corner points of the positioning code and then correcting the images taken by the camera at large angles or not directly facing the rigid plate based on perspective changes, the recognition efficiency and reliability are improved.

[0059] Step S602: Using a pre-built location code recognition algorithm, at least one candidate location code is identified from the first image, and the identification information and pixel coordinate information corresponding to the at least one candidate location code are obtained respectively, wherein the candidate location code is the location code in the first image that is not occluded.

[0060] Step S603: Based on the identification information of at least one candidate positioning code, obtain the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system.

[0061] Step S604: Determine the second positional relationship between the first camera coordinate system and the marker plate coordinate system based on the pixel coordinate information and physical coordinate information corresponding to at least one candidate positioning code, wherein the first camera coordinate system is the camera coordinate system of the first camera.

[0062] For details of the above steps, please refer to [link / reference]. Figure 5 Steps S502 to S504 of the illustrated embodiment will not be described again here.

[0063] In some optional implementations, the pixel coordinate information may include the pixel coordinates of multiple corner points in the positioning code, and the physical coordinate information may include the physical coordinates of multiple corner points. Accordingly, when determining the second positional relationship, a pre-constructed multi-point optimal fitting algorithm is used to perform a fitting operation on the pixel coordinates and physical coordinates of multiple corner points to obtain the second positional relationship.

[0064] Specifically, after identifying the corner coordinates of multiple positioning codes, a multi-point optimal fitting algorithm, such as PnP or iterative least squares, can be used to fit the coordinates of multiple corner points to reduce the error until the error meets the preset convergence condition. Based on the converged rotation matrix and translation vector, the second positional relationship is obtained.

[0065] In this way, the pixel coordinates of multiple corner points are used to obtain the second positional relationship through the optimal fitting algorithm, which improves the recognition accuracy of the gray area.

[0066] Step S605: Determine the pixel region of the gray card in the first image based on the first position information, the first position relationship, and the second position relationship.

[0067] Optionally, step S605 above may include: Step S6051: Determine the second position information of the gray board in the coordinate system of the marking board based on the first position information and the first position relationship.

[0068] Specifically, in the gray plate coordinate system, the coordinates of each point on the gray plate in its own coordinate system have been preset. The first positional relationship can indicate the transformation relationship between the gray plate coordinate system and the marker plate coordinate system. Therefore, by using the first positional relationship to perform a transformation operation on the coordinates of one or more specified points on the gray plate in the gray plate coordinate system, the position information of the gray plate in the marker plate coordinate system can be determined.

[0069] For example, by using the first positional relationship, the preset three-dimensional coordinates of the gray board outline vertices in the gray board coordinate system are converted into coordinates in the marker board coordinate system, which is the second positional information.

[0070] Step S6052: Determine the position information of the gray board in the camera coordinate system based on the second position information and the second position relationship.

[0071] Specifically, the second positional relationship can indicate the transformation relationship between the first camera coordinate system and the marker plate coordinate system. Therefore, the coordinates of the gray board in the marker plate coordinate system can be transformed to the first camera coordinate system through the second positional information and the second positional relationship, so as to obtain the position information of the gray board in the camera coordinate system.

[0072] Step S6053: After performing a projection operation on the position information of the gray board in the first camera coordinate system, the pixel area of ​​the gray board in the first image is obtained.

[0073] Specifically, since the position information of the gray card in the first camera coordinate system is three-dimensional, while the image is two-dimensional, a projection operation can be performed on the position information of the gray card in the first camera coordinate system to obtain the pixel region of the gray card in the first image. The pixel region can include the row range and column range of pixels. During the projection operation, combined with the camera's inherent internal parameters and distortion coefficients, an imaging model, such as a pinhole imaging model or a polynomial distortion imaging model, can be used to map the three-dimensional coordinates of the gray card vertices in the camera coordinate system to two-dimensional pixel coordinates. The region enclosed by the closed contour formed by sequentially connecting the pixel coordinates of each vertex is the pixel region of the gray card in the first image.

[0074] The gray area recognition method provided in this embodiment is adaptable to camera shooting at large angles. Specifically, when the shooting angle is large, perspective transformation is performed in advance to reduce errors caused by camera lens distortion. It has a wide range of applications, simple recognition logic, and high efficiency. In addition, by transforming coordinates in different coordinate systems and combining the positional relationship between the marker, gray card, and camera, the pixel area of ​​the gray card is calculated, which improves the reliability of the recognition results, reduces recognition errors, and improves recognition efficiency.

[0075] When there are multiple cameras, it indicates a large number of images are captured. To ensure the accuracy of subsequent gray area recognition, images with fewer obstructed location codes can be selected from the images captured by the multiple cameras for gray area recognition. Accordingly, this embodiment provides a gray area recognition method that can be used in electronic devices. Figure 7 This is a flowchart of a gray area identification method according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps: Step S701: Obtain the first image corresponding to the first camera, and the first positional relationship between the marker coordinate system and the grayboard coordinate system.

[0076] Wherein, the first camera is any one of at least one cameras; Step S702: Using a pre-built location code recognition algorithm, at least one candidate location code is identified from the first image, and the identification information and pixel coordinate information corresponding to the at least one candidate location code are obtained respectively.

[0077] Among them, the candidate location code is the location code that is not occluded in the first image; Step S703: Determine the actual number of candidate location codes.

[0078] Specifically, after identifying the candidate location codes, that is, after performing the above step S201, the number of candidate location codes can be counted first.

[0079] Step S704: If the actual quantity is greater than or equal to the preset quantity threshold, then based on the identification information of at least one candidate positioning code, obtain the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system.

[0080] The preset quantity threshold can be one or multiple, depending on the actual needs.

[0081] Specifically, if the number of candidate location codes is greater than or equal to a preset threshold, then according to, for example Figure 5 Step S503, shown, determines the physical coordinate information of the candidate positioning code in the marker plate coordinate system, which will not be elaborated here.

[0082] Step S705: Determine the second positional relationship between the first camera coordinate system and the marker plate coordinate system based on the pixel coordinate information and physical coordinate information corresponding to at least one candidate positioning code.

[0083] Wherein, the first camera coordinate system is the camera coordinate system of the first camera. For details, please refer to [link to details]. Figure 5 Step S504 of the illustrated embodiment will not be described again here.

[0084] Step S706: Determine the pixel region of the gray card in the first image based on the first positional relationship and the second positional relationship.

[0085] Please see details Figure 5 Step S505 of the illustrated embodiment will not be described again here.

[0086] Alternatively, in step S707, if the actual number is determined to be less than a preset number threshold, the first image is discarded.

[0087] Specifically, if the number of candidate location codes is less than a preset threshold, it means that there are too many location codes that are occluded in the camera's viewpoint, which is not accurate enough for subsequent gray area identification. Therefore, the image taken in the camera's viewpoint is discarded and not used as a reference for subsequent gray area identification.

[0088] The gray area recognition method provided in this embodiment improves the efficiency and anti-occlusion ability of gray area recognition by selecting images that meet the requirements based on the number of candidate location codes identified when multiple cameras are shooting, and also improves the reliability and accuracy of recognition.

[0089] In some optional implementations, when the image processing system includes multiple cameras, grayscale regions can be identified in images captured by each camera. The differences between the identified grayscale regions in different images are then compared. If the difference is small, the identified grayscale region is considered relatively accurate, and any grayscale region can be directly determined as the final pixel region of the grayscale. If the difference is large, the image can be recaptured, and the grayscale region identification operation can be performed again. Accordingly, the image processing device can also perform the following specific steps: Step 1: After determining the pixel area of ​​the gray card in the second image, determine the fourth positional relationship between the first camera coordinate system and the second camera coordinate system based on the second positional relationship and the third positional relationship between the second camera coordinate system and the marker plate coordinate system.

[0090] In this context, the second camera is any one of the cameras other than the first camera, the second image is the image corresponding to the second camera, and the second camera coordinate system is the camera coordinate system of the second camera. The second image can be the initial image captured by the second camera, or it can be an image obtained after the analysis operations in steps S6011 to S6012. For example, if the shooting angle of the second camera is greater than a preset angle threshold, the image obtained after performing a perspective transformation on the initial image captured by the second camera.

[0091] Specifically, the method for determining the pixel region of the gray card in the second image is the same as that for determining the pixel region in the first image, and will not be elaborated here. The image processing device can determine the third positional relationship between the second camera coordinate system and the marker plate coordinate system in a manner similar to step S504 above. Further, using the marker plate coordinate system as a transformation medium, the first camera coordinate system and the second camera coordinate system are analyzed to obtain the fourth positional relationship between the first camera coordinate system and the second camera coordinate system. Here, the transformation matrices corresponding to the second positional relationship and the third positional relationship can be simultaneously inversely operated to remove intermediate variables, and the rotation matrix and translation vector between the first camera coordinate system and the second camera coordinate system can be derived, denoted as the fourth positional relationship.

[0092] Step two: Determine the target error based on the fourth positional relationship, the pixel area of ​​the gray card in the first image, and the pixel area in the second image.

[0093] Specifically, the image processing device performs a three-dimensional coordinate transformation between the first camera coordinate system and the second camera coordinate system based on the fourth positional relationship. The recognition error can be determined by the overlap of the grayscale pixel region contours under the same coordinate system. For example, the pixel coordinates of the contour vertices of the grayscale pixel region in the second image are projected onto the three-dimensional coordinate system of the second camera, transformed to the three-dimensional coordinate system of the first camera through the fourth positional relationship, and then projected back onto the first camera coordinate system to obtain the simulated grayscale pixel region contour from the first image perspective. The deviation of the vertex coordinates of the simulated grayscale pixel region contour from the grayscale pixel region contour in the original first image, or the deviation of the region area overlap, is calculated. A comparison of one type of deviation value is made, or a weighted calculation of multiple types of deviation values ​​is performed to obtain a comprehensive value. No specific deviation form is limited here; it can be selected according to actual needs to determine the target error.

[0094] Step 3: If the target error is less than the preset error threshold, determine one pixel region of the gray card in the first image or in the second image as the final pixel region of the gray card.

[0095] Specifically, the image processing device can compare the determined target error with a preset error threshold, wherein the preset error threshold can be set according to the accuracy required for the work. If the target error is within the preset error threshold, the pixel region in any image can be determined as the final grayscale pixel region.

[0096] In the case of a multi-camera array, the grayscale region identification method provided in this embodiment may be different because the images captured by different cameras are different. Therefore, in order to ensure the accuracy of the final identified grayscale region, this solution compares the identified grayscale regions in pairs of images captured by different cameras. When the error between the two is small, it indicates that the grayscale region identification is accurate. Only then will the pixel region in any one image be determined as the final grayscale pixel region.

[0097] This embodiment provides a method for identifying gray area regions, which can be used in electronic devices. Figure 8 This is a flowchart of a gray area identification method according to an embodiment of the present invention, such as... Figure 8As shown, a mobile device (such as a drone, a lifting platform, etc.) equipped with a multispectral camera moves to an altitude of 0.5 to 1.2 meters to photograph a rigid plate. The rigid plate has a gray board and a marker board with a fixed and known positional relationship. Taking an ArUco marker board as an example, after successful image capture, the target image can be transmitted to an electronic device or the image processing module in the camera to determine if the ArUco marker board has been recognized. If the ArUco marker board is not recognized, gray board recognition fails. If the ArUco marker board is recognized, the system continues to determine if at least one ArUco marker code on the marker board has been recognized. Each ArUco marker code has four corner points. If at least one complete ArUco marker code with all four corner points can be recognized, recognition is successful; otherwise, gray board recognition fails. After recognizing at least one ArUco marker code with all four corner points, the pixel coordinates and physical coordinates of the ArUco marker code are obtained. Based on these coordinates and the pre-defined fixed positional relationship between the gray board and the marker board, the position of the gray board area can be determined.

[0098] The gray area identification method provided in this embodiment uses ArUco markers and ArUco codes to identify gray areas. It has strong identification stability, provides a large number of corner points, has high identification accuracy, and stronger robustness. At the same time, it supports complex calibration requirements such as nonlinearity, non-uniformity, and multimodality, and has a wide range of applications.

[0099] This embodiment also provides a gray area identification device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] This embodiment provides a device for identifying gray area, such as... Figure 9 As shown, it includes: The acquisition module 910 is used to acquire the first positional relationship between the first image corresponding to the first camera, the marker coordinate system and the grayboard coordinate system, wherein the first camera is any one of at least one cameras.

[0101] The recognition module 920 is used to identify at least one candidate positioning code from the first image using a pre-built positioning code recognition algorithm, and to obtain the identification information and pixel coordinate information corresponding to the at least one candidate positioning code respectively, wherein the candidate positioning code is the positioning code in the first image that is not occluded.

[0102] The acquisition module 910 is also used to acquire the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system based on the identification information of at least one candidate positioning code.

[0103] The determining module 930 is used to determine a second positional relationship between the first camera coordinate system and the marker coordinate system based on the pixel coordinate information and physical coordinate information corresponding to at least one candidate positioning code, wherein the first camera coordinate system is the camera coordinate system of the first camera; and to determine the pixel region of the gray card in the first image based on the first position information, the first positional relationship, and the second positional relationship.

[0104] In some optional implementations, the acquisition module 910 includes: The acquisition unit is used to acquire the initial image captured by the first camera, and the shooting angle of the first camera when capturing the initial image.

[0105] The perspective transformation unit is used to perform perspective transformation on the initial image to obtain the first image when the shooting angle is greater than a preset angle threshold.

[0106] The image determination unit is used to determine the initial image as the first image when the shooting angle is less than or equal to a preset angle threshold.

[0107] In some alternative implementations, the determining module 930 includes: The multi-point fitting unit is used to perform fitting operations on the pixel coordinates and physical coordinates of multiple corner points using a pre-built multi-point optimal fitting algorithm to obtain the second positional relationship.

[0108] The first determining unit is used to determine the second position information of the gray board in the coordinate system of the marking board based on the first position information and the first position relationship.

[0109] The second determining unit is used to determine the position information of the gray board in the camera coordinate system based on the second position information and the second position relationship.

[0110] The projection unit is used to perform a projection operation on the position information of the gray board in the first camera coordinate system to obtain the pixel area of ​​the gray board in the first image.

[0111] In some alternative implementations, the gray area identification device further includes: The quantity statistics module is used to determine the actual number of candidate location codes.

[0112] The quantity determination processing module is used to obtain the physical coordinate information of at least one candidate positioning code in the marker plate coordinate system based on the identification information of at least one candidate positioning code when the actual quantity is determined to be greater than or equal to a preset quantity threshold.

[0113] The image removal module is used to remove the first image when the actual number is less than a preset threshold.

[0114] The cross-camera pose solving module is used to determine the fourth positional relationship between the first camera coordinate system and the second camera coordinate system after determining the pixel region of the gray card in the second image, based on the second positional relationship and the third positional relationship between the second camera coordinate system and the marker plate coordinate system. Here, the second camera is any one of the other cameras among multiple cameras except the first camera, the second image is the image corresponding to the second camera, and the second camera coordinate system is the camera coordinate system of the second camera.

[0115] The target error calculation module is used to determine the target error based on the fourth positional relationship, the pixel area of ​​the gray card in the first image, and the pixel area in the second image.

[0116] The final region determination module is used to determine one of the pixel regions of the gray card in the first image or the pixel region of the second image as the final pixel region of the gray card, provided that the target error is less than a preset error threshold.

[0117] The gray area identification device provided in this embodiment of the invention can execute the gray area identification method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0118] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0119] The following is a detailed reference. Figure 10 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in ROM 1002 or a program loaded from memory 1008 into RAM 1003. RAM 1003 also stores various programs and data required for the operation of the electronic device. The processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. An input / output (I / O) interface 1005 is also connected to bus 1004; wherein ROM is a read-only memory and RAM is a random access memory.

[0120] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0121] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1009, or installed from a memory 1008, or installed from a ROM 1002. When the computer program is executed by the processor 1001, it performs the functions defined in the gray area identification method of the embodiments of the present invention.

[0122] Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0123] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded via a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the gray area identification method shown in the above embodiments is implemented.

[0124] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0125] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for identifying gray area, characterized in that, The method is applied to an image processing system, which includes a rigid plate, at least one camera, and an image processing device. The rigid plate is provided with a gray plate and a marking plate, and the marking plate is provided with multiple positioning codes. The method is executed by the image processing device and includes: Obtain the first positional relationship between the first image corresponding to the first camera, the marker coordinate system, and the grayboard coordinate system, as well as the first position information of the grayboard in the grayboard coordinate system, wherein the first camera is any one of at least one of the cameras; Using a pre-built location code recognition algorithm, at least one candidate location code is identified from the first image, and the identification information and pixel coordinate information corresponding to at least one candidate location code are obtained respectively, wherein the candidate location code is the location code that is not occluded in the first image; Based on the identification information of at least one of the candidate positioning codes, obtain the physical coordinate information of at least one of the candidate positioning codes in the coordinate system of the marker plate; Based on the pixel coordinate information and physical coordinate information corresponding to at least one of the candidate positioning codes, a second positional relationship between the first camera coordinate system and the marker plate coordinate system is determined, wherein the first camera coordinate system is the camera coordinate system of the first camera; Based on the first location information, the first location relationship, and the second location relationship, the pixel region of the gray card in the first image is determined.

2. The method according to claim 1, characterized in that, Determining the pixel region of the gray card in the first image based on the first location information, the first location relationship, and the second location relationship includes: Based on the first position information and the first position relationship, determine the second position information of the gray board in the coordinate system of the marking board; Based on the second position information and the second position relationship, the position information of the gray board in the camera coordinate system is determined; After performing a projection operation on the position information of the gray board in the first camera coordinate system, the pixel region of the gray board in the first image is obtained.

3. The method according to claim 1 or 2, characterized in that, Acquiring the first image corresponding to the first camera includes: Acquire the initial image captured by the first camera, and the shooting angle of the first camera when capturing the initial image; If the shooting angle is determined to be greater than a preset angle threshold, the initial image is subjected to perspective transformation to obtain the first image. Alternatively, if the shooting angle is determined to be less than or equal to the preset angle threshold, the initial image may be determined as the first image.

4. The method according to claim 1 or 2, characterized in that, The pixel coordinate information includes the pixel coordinates of multiple corner points, and the physical coordinate information includes the physical coordinates of multiple corner points; The step of determining the second positional relationship between the first camera coordinate system and the marker plate coordinate system based on the pixel coordinate information and physical coordinate information corresponding to at least one of the candidate positioning codes includes: The second positional relationship is obtained by performing a fitting operation on the pixel coordinates and physical coordinates of multiple corner points using a pre-constructed multi-point optimal fitting algorithm.

5. The method according to claim 1 or 2, characterized in that, When the number of cameras is multiple, the method further includes: Determine the actual number of the candidate location codes; If the actual quantity is determined to be greater than or equal to a preset quantity threshold, then the physical coordinate information of at least one candidate positioning code in the coordinate system of the marker board is obtained according to the identification information of at least one candidate positioning code.

6. The method according to claim 5, characterized in that, The method further includes: If the actual number is determined to be less than the preset number threshold, the first image is discarded.

7. The method according to claim 5, characterized in that, The method further includes: After determining the pixel region of the gray card in the second image, a fourth positional relationship between the first camera coordinate system and the second camera coordinate system is determined based on the second positional relationship and the third positional relationship between the second camera coordinate system and the marker plate coordinate system. Here, the second camera is any one of the other cameras among the plurality of cameras except the first camera, the second image is the image corresponding to the second camera, and the second camera coordinate system is the camera coordinate system of the second camera. The target error is determined based on the fourth positional relationship, the pixel region of the gray card in the first image, and the pixel region in the second image; If the target error is determined to be less than a preset error threshold, the pixel region of the gray card in the first image or one pixel region in the second image is determined as the final pixel region of the gray card.

8. A device for identifying gray board areas, characterized in that, The device is applied to an image processing system, which includes a rigid plate, at least one camera, and an image processing apparatus. The rigid plate is provided with a gray plate and a marking plate, and the marking plate is provided with multiple positioning codes. The device is configured within the image processing apparatus. The device includes: The acquisition module is used to acquire a first image corresponding to the first camera, a first positional relationship between the marker coordinate system and the grayboard coordinate system, and the first position information of the grayboard in the grayboard coordinate system, wherein the first camera is any one of at least one of the cameras; The identification module is used to identify at least one candidate location code from the first image using a pre-built location code identification algorithm, and to obtain the identification information and pixel coordinate information corresponding to at least one of the candidate location codes respectively, wherein the candidate location code is the location code in the first image that is not occluded. The acquisition module is further configured to acquire, based on the identification information of at least one of the candidate positioning codes, the physical coordinate information of at least one candidate positioning code in the coordinate system of the marker board; The determining module is configured to determine a second positional relationship between the first camera coordinate system and the marker coordinate system based on pixel coordinate information and physical coordinate information corresponding to at least one of the candidate positioning codes, wherein the first camera coordinate system is the camera coordinate system of the first camera; and to determine the pixel region of the gray card in the first image based on the first position information, the first positional relationship, and the second positional relationship.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the gray area identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the gray area identification method according to any one of claims 1 to 7.

Images