A target detection method, device and robot

Abstract

This application provides a target detection method, apparatus, and robot, relating to the field of photovoltaic technology. The method includes: acquiring a target image of a photovoltaic module; inputting the target image into a segmentation model, which segments the target image into multiple target bounding boxes, each target bounding box including a mask image of a target object, the target object including the boundary and seam of the photovoltaic module, and each mask image having a label corresponding to the target object; integrating each mask image to obtain a mask image set; the mask image set includes mask images with two types of labels, each label corresponding to a target object. Therefore, this method can simultaneously detect the boundary and seam in the target image.

Description

[0001] This application claims priority to Chinese Patent Application No. 2026103604190, filed on March 23, 2026, entitled "A Target Detection Method, Device and Robot", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of photovoltaic technology, and in particular to a target detection method, device and robot. Background Technology

[0003] With the rapid development of the new energy industry, photovoltaic power plants, as an important form of clean energy utilization, are constantly expanding in scale, typically consisting of tens of thousands of photovoltaic modules arranged in an orderly manner according to specific rules. The power generation efficiency of photovoltaic modules directly affects the overall revenue of photovoltaic power plants, therefore, photovoltaic robots (hereinafter referred to as "robots") are needed to perform regular cleaning and inspection of photovoltaic modules.

[0004] During the operation, the robot must strictly follow the preset operation route to avoid problems such as missed sweeps and repeated sweeps. This requires the robot to be able to detect the boundaries and seams in the collected images at the same time.

[0005] Therefore, how to detect both boundaries and seams in an image simultaneously has become a pressing technical problem that needs to be solved. Summary of the Invention

[0006] To address the aforementioned issues, this application provides a target detection method, apparatus, and robot capable of simultaneously detecting boundaries and seams in an image to be detected.

[0007] The embodiments of this application disclose the following technical solutions: In a first aspect, this application discloses a target detection method, the method comprising: Acquire the image of the photovoltaic module to be inspected; The image to be detected is input into a segmentation model, which segments the image into multiple target bounding boxes. Each target bounding box includes a mask image of a target object. The target object includes the boundary and seam of the photovoltaic module. Each mask image has a label corresponding to the target object. By integrating each of the mask images, a mask image set is obtained; the mask image set includes mask images with two types of labels, each label corresponding to a target object.

[0008] Optionally, the method further includes: For the first mask image labeled as a boundary in the mask image set, perform the following steps: obtain the set of contour points in the first mask image; perform line fitting on the set of contour points to determine the boundary detection result; For the second mask image labeled as a seam in the mask image set, the following steps are performed: obtain the set of contour points in the second mask image; filter out a subset of contour points whose ordinate is greater than a second threshold from the set of contour points; perform line fitting on the subset of contour points to determine the seam detection result.

[0009] Optionally, the method further includes: Based on the prior information of the physical structure of the photovoltaic module, geometric consistency constraints are applied to the boundary detection results, and spatial topological relationship constraints are applied to the joint detection results, to obtain corrected boundary detection results and corrected joint detection results.

[0010] Optionally, the step of applying geometric consistency constraints to the boundary detection results based on prior information about the physical structure of the photovoltaic module includes: Based on the overall distribution pattern of the boundary detection results within the same photovoltaic module row, the reference direction parameters are determined. Based on the reference direction parameters, the orientation consistency of each boundary detection result is adjusted. Based on the preset physical size information of the photovoltaic module, spatial position constraints are applied to the relative positions between each of the adjusted boundary detection results.

[0011] Optionally, the step of constraining the spatial topological relationship of the joint detection results includes: Based on the reference direction parameters, the directional consistency of each joint detection result is adjusted; Based on the effective area defined by the corrected boundary detection results, spatial interval constraints are applied to each of the joint detection results.

[0012] Optionally, the method further includes: Based on the temporal relationship between the corrected boundary detection results and the corrected seam detection results, a boundary detection sequence and a seam detection sequence are established. For the corrected boundary detection results of consecutive frames in the boundary detection sequence and the corrected seam detection results of consecutive frames in the seam detection sequence, cross-frame consistency constraints are applied to obtain constrained boundary detection sequences and constrained seam detection sequences. Based on the robot's motion direction, the constrained boundary detection sequence and the constrained seam detection sequence are eliminated to obtain the target boundary detection sequence and the target seam detection sequence.

[0013] Optionally, the target object further includes a compressed block; the mask image set includes mask images of three types of labels; the method further includes: For the third mask image labeled as "block" in the mask image set, perform the following: Based on the corrected boundary detection result and the corrected seam detection result, obtain the block detection result in the third mask image.

[0014] Optionally, obtaining the set of contour points in the first mask image includes: A first overlay image is generated based on the pixel values ​​of each pixel in the first mask image and the image to be detected; The first superimposed image is binarized to obtain a first binarized image; Obtain the set of contour points in the first binarized image; The step of performing straight line fitting on the contour point set to determine the boundary detection result includes: A straight line is fitted to the set of contour points to obtain candidate detection results; the candidate detection results include at least one contour line. The boundary detection result is determined by filtering out contour lines with fewer than a first threshold number of pixels from the candidate detection results.

[0015] Optionally, obtaining the set of contour points in the second mask image includes: Shrink the mask region of the second mask image inward by M pixels to obtain a candidate mask image; M is a positive integer; A second overlay image is generated based on the pixel values ​​of each pixel in the candidate mask image and the image to be detected; The second overlay image is binarized to obtain a second binarized image; Obtain the set of contour points in the second binarized image.

[0016] Optionally, the method further includes: A merged image is generated based on the pixel value of each pixel in each mask image in the mask image set, and the number of mask images in the mask image set; The merged image is binarized to obtain a binarized merged image; Determine the area of ​​the target region in the merged image after binarization; the target region is a connected region with a pixel value of 255. If the area of ​​the region is greater than the third threshold, then by traversing the target region in each mask image in the mask image set, a subset of mask images with non-zero pixel values ​​in the target region is determined, and the subset of mask images is removed to obtain a set of mask images after removal.

[0017] Secondly, this application discloses a target detection device, the device comprising: an image acquisition module, an image segmentation module, and an image integration module; The image acquisition module is used to acquire the image of the photovoltaic module to be detected; The image segmentation module is used to input the image to be detected into a segmentation model, and segment the image to be detected into multiple target bounding boxes through the segmentation model. Each target bounding box includes a mask image of a target object. The target object includes the boundary and seam of the photovoltaic module. Each mask image has a label corresponding to the target object. The image integration module is used to integrate each of the mask images to obtain a mask image set; the mask image set includes mask images with two types of labels, each label corresponding to a target object.

[0018] Optionally, the target detection device further includes: a first determining module and a second determining module; The first determining module is configured to: for a first mask image labeled as a boundary in the mask image set, perform the following steps: obtain a set of contour points in the first mask image; perform line fitting on the set of contour points to determine the boundary detection result; The second determining module is used to perform the following steps for the second mask image labeled as a seam in the mask image set: obtaining a set of contour points in the second mask image; filtering out a subset of contour points whose ordinate is greater than a second threshold from the set of contour points; performing line fitting on the subset of contour points to determine the seam detection result.

[0019] Optionally, the target detection device further includes: a first acquisition module; The first acquisition module is used to perform geometric consistency constraints on the boundary detection results and spatial topological relationship constraints on the joint detection results based on the prior information of the physical structure of the photovoltaic module, so as to obtain the corrected boundary detection results and the corrected joint detection results.

[0020] Optionally, the first acquisition module is specifically used to: determine a reference direction parameter based on the overall distribution pattern of each boundary detection result within the same photovoltaic module row; adjust the direction consistency of each boundary detection result according to the reference direction parameter; and constrain the relative position between each adjusted boundary detection result according to the preset physical size information of the photovoltaic module.

[0021] Optionally, the first acquisition module is specifically used to: adjust the direction consistency of each seam detection result according to the reference direction parameter; and constrain the spatial interval of each seam detection result according to the effective area defined by the corrected boundary detection result.

[0022] Optionally, the target detection device further includes: a sequence establishment module, a second acquisition module, and a third acquisition module; The sequence establishment module is used to establish a boundary detection sequence and a joint detection sequence based on the temporal relationship between the corrected boundary detection results and the corrected joint detection results. The second acquisition module is used to perform cross-frame consistency constraints on the corrected boundary detection results of consecutive frames in the boundary detection sequence and the corrected seam detection results of consecutive frames in the seam detection sequence, respectively, to obtain a constrained boundary detection sequence and a constrained seam detection sequence. The third acquisition module is used to eliminate the constrained boundary detection sequence and the constrained seam detection sequence according to the robot's motion direction, so as to obtain the target boundary detection sequence and the target seam detection sequence.

[0023] Optionally, the target object further includes a block; the mask image set includes mask images with three labels; the target detection device further includes a third determination module; the third determination module is used to: for the third mask image in the mask image set with the label "block", perform: obtain the block detection result in the third mask image based on the corrected boundary detection result and the corrected seam detection result.

[0024] Optionally, the first determining module is specifically configured to: generate a first superimposed image based on the pixel values ​​of each pixel point of the first mask image and the image to be detected; perform binarization processing on the first superimposed image to obtain a first binarized image; obtain a set of contour points in the first binarized image; perform line fitting on the set of contour points to obtain candidate detection results; the candidate detection results include at least one contour line; and determine the boundary detection results by filtering out contour lines with a number of pixels less than a first threshold from the candidate detection results.

[0025] Optionally, the second determining module is specifically used to: shrink the mask region of the second mask image inward by M pixels to obtain a candidate mask image; M is a positive integer; generate a second overlay image based on the pixel values ​​of each pixel point of the candidate mask image and the image to be detected; perform binarization processing on the second overlay image to obtain a second binarized image; and obtain the set of contour points in the second binarized image.

[0026] Optionally, the target detection device further includes: an image removal module; The image removal module is used to generate a merged image based on the pixel value of each pixel in each mask image in the mask image set and the number of mask images in the mask image set; perform binarization processing on the merged image to obtain a binarized merged image; determine the area of ​​the target region in the binarized merged image; the target region is a connected region with a pixel value of 255; if the area of ​​the region is greater than a third threshold, then by traversing the target region in each mask image in the mask image set, determine the subset of mask images with non-zero pixel values ​​in the target region, and remove the subset of mask images to obtain a set of removed mask images.

[0027] Thirdly, this application discloses a robot, which includes a memory and a processor; The memory is used to store computer programs or computer instructions; The processor is configured to execute computer programs or computer instructions stored in the memory, causing the robot to perform the target detection method as described in the first aspect.

[0028] Compared with the prior art, this application has the following beneficial effects: This application provides a target detection method, apparatus, and robot. The method includes: acquiring a photovoltaic module image to be detected; inputting the image to be detected into a segmentation model, and segmenting the image into multiple target bounding boxes, each target bounding box including a mask image of a target object, the target object including the boundary and seam of the photovoltaic module, each mask image having a label corresponding to the target object; integrating each mask image to obtain a mask image set; the mask image set includes mask images with two types of labels, each label corresponding to a target object. Therefore, this method can simultaneously detect the boundary and seam in the image to be detected. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A flowchart of a target detection method provided in an embodiment of this application; Figure 2 A flowchart of another target detection method provided in the embodiments of this application; Figure 3 A schematic diagram illustrating target detection provided in an embodiment of this application; Figure 4 This is a schematic diagram of a target detection device provided in an embodiment of this application. Detailed Implementation

[0031] As described above, how to simultaneously detect boundaries and seams in an image has become a pressing technical problem that needs to be solved.

[0032] Through research, the inventors proposed a target detection method, device, and robot that can simultaneously detect boundaries and seams in the image to be detected.

[0033] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0034] See Figure 1 The figure is a flowchart of a target detection method provided in an embodiment of this application. The method includes: S101: Obtain the image of the photovoltaic module to be inspected.

[0035] In one specific implementation, an image of the photovoltaic module to be inspected can be acquired using a camera mounted on a robot.

[0036] It should be noted that, in order to ensure the accuracy of target detection, the camera's intrinsic and extrinsic parameters need to be calibrated before acquiring the image to be detected.

[0037] Intrinsic calibration, in particular, establishes a relationship between the planar coordinate system and the camera coordinate system by determining the camera's internal geometric and optical parameters, thereby correcting camera distortion based on this relationship. For example, intrinsic calibration can be performed as follows: First, acquire multiple (e.g., 20) images of the calibration board from different angles and distances to ensure sufficient coverage of the camera's field of view. Then, calculate the camera's internal parameters, such as the camera's focal length (f...). x f y ), principal point coordinates (c x c y The parameters include radial and tangential distortion coefficients. Finally, distortion correction is performed using these internal parameters, thereby effectively correcting image distortion caused by the inherent characteristics of the camera lens.

[0038] Extrinsic calibration determines the camera's position and orientation in the world coordinate system, acquiring a bird's-eye view image aligned with the photovoltaic module's plane as the image to be detected. For example, extrinsic calibration can be performed as follows: First, calculate the rotation matrix R and translation vector T required to transform from the camera coordinate system to the world coordinate system. Then, use the aforementioned rotation matrix R and translation vector T to perform an inverse perspective transformation, thereby correcting the viewing angle tilt.

[0039] S102: Input the image to be detected into the segmentation model, and the segmentation model will segment the image to be detected into multiple target bounding boxes. Each target bounding box includes a mask image of a target object. The target object includes the boundary and seam of the photovoltaic module. Each mask image has a label corresponding to the target object.

[0040] First, the training method for the segmentation model is introduced, as follows: Steps A1-A2: A1: Obtain a set of sample images of photovoltaic modules.

[0041] The sample image set includes multiple sample images (e.g., 2000 images). Each sample image shows photovoltaic modules under different lighting conditions (e.g., strong midday light, diffused light on cloudy days, low light at dusk, etc.), thereby ensuring that the segmentation model trained has good robustness.

[0042] It is understood that, since the task of this application is object detection, each sample image can be labeled in YOLO segmentation text format. For example, the labels can be "boundary" and "seam". This application does not limit this.

[0043] A2: Train a deep learning model based on the sample image set to obtain a segmentation model.

[0044] The sample image set is used as input to a deep learning model (such as YOLO-Seg, U-Net, etc.), and the label corresponding to each sample image in the sample image set is used as the output label. The deep learning model is trained with the goal of minimizing the loss function value of the deep learning model to obtain a segmentation model.

[0045] It should be noted that this application does not limit the model parameters of the segmentation model. For example, it can be: input resolution 640×640, training epochs 200~300, initial learning rate 0.001, batch size 16, optimizer AdamW, activation function ReLU, and depthwise separable convolution can be used to balance performance and computation.

[0046] It should also be noted that, to further improve the generalization ability of the segmentation model, one can first perform data augmentation processing (such as random rotation, brightness perturbation, affine transformation, etc.) on a portion of the sample images in the sample image set to obtain an augmented sample image set; then, based on the augmented sample image set, train a machine learning model to obtain the segmentation model.

[0047] It should also be noted that, considering the limited computing power, small memory, and power consumption of embedded platforms for robots, directly deploying the original segmentation model often fails to meet real-time requirements. Therefore, system-level compression and acceleration of the segmentation model are necessary. For example, the trained PyTorch model can first be converted to an intermediate representation format (e.g., ONNX), then INT8 quantization can be performed using a dedicated toolchain such as RKNN-Toolkit2, and calibration can be completed using a representative dataset to achieve an optimal balance between accuracy and inference speed. Simultaneously, model pruning can be enabled to further reduce the number of parameters and computational load. Finally, the optimized segmentation model can be deployed to run on the NPU of an embedded processor such as the RK3588, thereby achieving efficient edge inference.

[0048] In the target detection method provided in this application embodiment, the image to be detected obtained in step S101 is input into a segmentation model. The segmentation model segments the image to be detected to obtain multiple target bounding boxes. Each target bounding box includes a mask image of a target object, and each mask image has a label (boundary and seam) corresponding to the target object. It can be understood that the mask image is a binary image. Generally speaking, the pixel value of the target object (boundary and seam) is 1 (white), and the pixel value of the background is 0 (black).

[0049] S103: By integrating each mask image, a mask image set is obtained; the mask image set includes mask images with two types of labels, each label corresponding to a target object.

[0050] In step S102 above, each target object has a corresponding mask image. However, multiple target objects often exist simultaneously in a single image to be detected. For example, in a single image to be detected, the following can be detected: Target object 1: First boundary; Target object 2: Second boundary; Target object 3: First seam; Target object 4: Second seam. Therefore, it is necessary to integrate each mask image to obtain a mask image set. This mask image set includes mask images with two labels (i.e., "boundary" and "seam"), each label corresponding to a target object.

[0051] In summary, the embodiments of this application provide a target detection method that can simultaneously detect boundaries and seams in an image to be detected.

[0052] See Figure 2 The figure is a flowchart of another target detection method provided in an embodiment of this application. The method includes: S201: Obtain the image of the photovoltaic module to be inspected.

[0053] S202: Input the image to be detected into the segmentation model, and the segmentation model will segment the image to be detected into multiple target bounding boxes. Each target bounding box includes a mask image of a target object. The target objects include the boundaries, seams and pressure blocks of the photovoltaic module. Each mask image has a label corresponding to the target object.

[0054] It should be noted that this step is similar to the training method in step S102, except that the labels include not only "boundary" and "seam" but also "block". This application does not limit this aspect.

[0055] S203: By integrating each mask image, a mask image set is obtained; the mask image set includes mask images with three types of labels, each label corresponding to a target object.

[0056] In the target detection method provided in the embodiments of this application, the mask image set is typically obtained through the following methods A1-A5: A1: For each target bounding box, perform the following: Calculate the probability value of each pixel within the target bounding box belonging to the target object to obtain the segmentation probability matrix of the target bounding box.

[0057] First, the probability value of a single pixel within the target bounding box belonging to the target object can be calculated using the following formulas (1) and (2): (1) (2) Where P(h,w) is the probability value of pixel (h,w) belonging to the target object, SegSum is the weighted sum of the segmentation probabilities, N is the number of mask channels, mask[s] is the weight coefficient of the s-th mask channel, and seg[s][h][w] is the original feature value of the segmentation tensor in the s-th mask channel and pixel (h,w).

[0058] Subsequently, based on the probability value of each pixel within the target bounding box belonging to the target object, the segmentation probability matrix seg_cv_resize of the target bounding box is obtained.

[0059] A2: For each target bounding box, perform the following: Determine the absolute pixel coordinates of the target bounding box on the image to be detected based on the normalized coordinates of the target bounding box.

[0060] The four bounding box coordinates of the target bounding box are typically normalized coordinates, such as x... min =0.2, y min =0.3, x max =0.8, y max =0.7. Therefore, it is necessary to determine the absolute pixel coordinates of the target bounding box on the image to be detected using the following formulas (3)-(6) based on the normalized coordinates of the target bounding box: Left= x min ×W + 0.5 (3) Top= y min ×H+0.5 (4) Right= x max ×W + 0.5 (5) Bottom= y max ×H+0.5 (6) Where Left is the absolute pixel coordinate of the left edge of the target bounding box in the image to be detected, Top is the absolute pixel coordinate of the top edge of the target bounding box in the image to be detected, Right is the absolute pixel coordinate of the right edge of the target bounding box in the image to be detected, Bottom is the absolute pixel coordinate of the bottom edge of the target bounding box in the image to be detected, W is the image width, and H is the image height. · This is the floor operator.

[0061] It should be noted that after determining the absolute pixel coordinates of the target bounding box on the image to be detected, it is also necessary to determine whether the following conditions are met: Left≥0, Top≥0, Right≤W, Bottom≤H, right>left and bottom>top. If so, proceed with step A3 below to ensure that the target bounding box is located inside the image to be detected and is valid.

[0062] A3: For each target bounding box, perform the following: Generate a binary mask of the target object within the target bounding box based on the segmentation probability matrix of the target bounding box and the absolute pixel coordinates of the target bounding box.

[0063] Specifically, the binary mask of the target object within the target bounding box can be generated using the following formula (7), based on the segmentation probability matrix of the target bounding box and the absolute pixel coordinates of the target bounding box: (7) Among them, T min As the fourth threshold, T max Let roi(h, w) be the fifth threshold, and roi(h, w) be the binary mask of pixel (h, w). If roi(h, w) = 255, then it is white; if roi(h, w) = 0, then it is black.

[0064] For example, the fourth threshold T min It can be 0.5, the fifth threshold T max It can be 1. We choose 0.5 as the fourth threshold T. min The reason is that 0.5 is the decision boundary for binary classification problems, which can effectively filter low-confidence pixels and improve mask accuracy.

[0065] A4: For each target bounding box, perform the following: Generate a mask image based on the size information of the image to be detected and the binary mask of the target object within the target bounding box.

[0066] First, based on the size information (e.g., width and height) of the image to be detected, a pure black background image of the same size is generated, i.e., mask(i, j) = 0. Then, the binary mask of the target object within the target bounding box is combined with the pure black background image to generate a mask image, as shown in the following formula (8): (8) Where mask(rect) represents the pixel values ​​of the mask image.

[0067] It should be noted that after obtaining the mask image, a deep copy of the mask image is required, as shown in the following formula (9): (9) Here, Masks[k] is a copy of the k-th mask image, mask is the mask image, and clone(). is the deep copy function. The purpose of deep copy is to allocate new memory space, ensuring that each mask image has an independent memory lifecycle, thus avoiding data overwriting and reference errors.

[0068] A5: By integrating the mask images of each target bounding box, a set of mask images is obtained.

[0069] In the target detection method provided in this application embodiment, after integrating the mask images of each target bounding box to obtain a mask image set, the duplicate mask images in the mask image set obtained in step A5 can be deleted through the following steps B1-B4 to obtain a mask image set after deletion: B1: Generate a merged image based on the pixel value of each pixel in each mask image in the mask image set, and the number of mask images in the mask image set.

[0070] First, the average pixel value of each pixel in each mask image in the mask image set is obtained by dividing the pixel value of each pixel by the number of mask images in the set. Then, a merged image is generated based on the average pixel value of each pixel.

[0071] For example, if the mask image set includes 3 mask images, and the pixel value of pixel A in the first mask image is 255, the pixel value of pixel A in the second mask image is 0, and the pixel value of pixel A in the third mask image is 255, then the average pixel value of pixel A is determined to be (255+255+0)÷3≈170, and the value of pixel A in the merged image is 170.

[0072] B2: Perform binarization on the merged image to obtain the binarized merged image.

[0073] The truncation threshold for binarization can be 255 / K, where K is the number of mask images in the mask image set. If the pixel value of a pixel in the merged image is greater than 255 / K, then the pixel value of the merged image after binarization is 255; if the pixel value of a pixel in the merged image is less than or equal to 255 / K, then the pixel value of the merged image after binarization is 0.

[0074] B3: Determine the area of ​​the target region in the merged image after binarization.

[0075] The target region is a connected region with a pixel value of 255.

[0076] B4: If the area of ​​the region is greater than the third threshold, then by traversing the target region in each mask image in the mask image set, the subset of mask images with non-zero pixel values ​​of the target region is determined, and the subset of mask images is removed to obtain the mask image set after removal.

[0077] If the area of ​​the target region is greater than the third threshold, it is determined that the target region has significant and unreasonable mask overlap and needs to be processed. For example, a subset of mask images with non-zero pixel values ​​for the target region can be determined from the set of mask images, and only one mask image from the subset can be retained (e.g., retain the mask image with the highest confidence in the subset). Alternatively, the subset of mask images can be directly deleted to obtain a set of mask images after removal.

[0078] Therefore, by using steps B1-B4 above, the problem of mask overlap caused by false detections or noise in the segmentation model can be effectively solved.

[0079] S204: For the first mask image labeled as the boundary in the mask image set, perform the following: obtain the set of contour points in the first mask image; perform line fitting on the set of contour points to determine the boundary detection result.

[0080] For the first mask image labeled as the boundary in the set of mask images after removal, perform the following steps C1-C2. It is understood that steps B1-B4 can also be omitted, and the first mask image labeled as the boundary can be directly determined from the set of mask images, and the following steps C1-C2 can be performed. This application does not limit this.

[0081] C1: Use the cv2.findContours function of a computer vision library (such as OpenCV) to process the first mask image and obtain the set of contour points in the first mask image.

[0082] It should be noted that due to factors such as occlusion in outdoor scenes, equipment dirt, and shooting noise, the boundary areas of the mask in the first mask image may exhibit holes (inner gaps) or breaks (discontinuous boundaries), reducing the accuracy of subsequent boundary detection. Therefore, the following steps can be performed: First, a 3×3 convolution kernel is used to erode the boundary region of the mask to remove minor noise and burrs. Then, a 5×5 convolution kernel is used to dilate the broken parts of the boundary region of the mask to obtain the processed first mask image, thereby ensuring the integrity and coherence of the boundary region of the mask.

[0083] Secondly, a first overlay image is generated based on the pixel values ​​of each pixel in the processed first mask image and the image to be detected. It is understandable that the processed first mask image may lose some texture details (such as subtle metallic textures and gaps at device boundaries). Therefore, by multiplying the pixel values ​​of each pixel in the processed first mask image with the pixel values ​​of each pixel in the image to be detected to generate the first overlay image, the problem of detail loss in the segmentation model can be solved, improving the accuracy of subsequent boundary detection.

[0084] Subsequently, the first overlay image is binarized to obtain a first binarized image. For example, cv2.adaptiveThreshold can be used to binarize the first overlay image, thereby effectively overcoming common problems such as uneven lighting (dappled light and shadow) and equipment reflection that are frequently encountered by robots in outdoor inspections.

[0085] Finally, the set of contour points in the first binary image is obtained. For example, the cv2.findContours function of a computer vision library (such as OpenCV) can be used to process the first binary image to obtain the set of contour points in the first binary image.

[0086] C2: Use the least squares method or RANSAC algorithm to perform straight line fitting on the contour point set to determine the boundary detection results.

[0087] Specifically, firstly, the set of contour points is fitted with a straight line using the least squares method or the RANSAC algorithm to obtain candidate detection results; the candidate detection results include at least one contour line. For example, the formula for the least squares method is shown in formula (10) below: (10) Where (x, y) are contour points, a is the first coefficient (slope), b is the second coefficient (intercept), and N is the number of contour points in the contour point set.

[0088] Finally, the boundary detection result is determined by filtering out contour lines with fewer than a first threshold (e.g., 50 or 80 pixels) from the candidate detection results. The boundary detection result refers to a continuous, smooth boundary line. It can be understood that filtering out contour lines with fewer than the first threshold serves to eliminate "false boundaries" caused by residual noise or minute textures (e.g., small scratches on the surface of photovoltaic modules), retaining only contour lines of a certain length that conform to the characteristics of a true boundary, thereby ensuring the accuracy of the boundary detection result.

[0089] S205: For the second mask image labeled as "seam" in the mask image set, perform the following: obtain the set of contour points in the second mask image; from the set of contour points, filter to obtain a subset of contour points whose ordinate is greater than the second threshold; perform line fitting on the subset of contour points to determine the seam detection result.

[0090] For the second mask image labeled "seam" in the set of mask images after removal, perform the following steps D1-D3. It is understood that steps B1-B4 can also be omitted, and the second mask image labeled "seam" can be directly determined from the set of mask images, and the following steps D1-D3 can be performed. This application does not limit this.

[0091] D1: Use the cv2.findContours function of a computer vision library (such as OpenCV) to process the second mask image and obtain the set of contour points in the second mask image.

[0092] Specifically, firstly, the mask region in the second mask image is shrunk inward by M pixels (M is a positive integer, such as 3 or 5) to obtain a candidate mask image. It can be understood that shrinking the mask region in the second mask image by M pixels serves the purpose that the mask region in the second mask image is usually slightly wider than the actual seam location. To improve accuracy, this mask region needs to be shrunk inward by several pixels to obtain data that more closely resembles the actual seam.

[0093] Subsequently, the pixel values ​​of each pixel in the candidate mask image and the pixel values ​​of each pixel in the image to be detected are multiplied together to generate a second overlay image.

[0094] Next, the second superimposed image is binarized to obtain the second binarized image.

[0095] Subsequently, the set of contour points in the second binarized image is obtained.

[0096] D2: From the set of contour points, select those with a ordinate greater than the second threshold T. y A subset of outline points.

[0097] It is understandable that the selection process involves choosing individuals whose ordinate is greater than the second threshold T. y The purpose of the contour point subset is that the seam label area in the second mask image is relatively wide, which often leads to the extracted contour point set forming an approximately rectangular region. When directly fitting a straight line to the contour point set, multiple local optima may appear (e.g., the central axis or diagonal of the rectangle), making it impossible to stably fit the seam line. This application, by selecting contour points with larger ordinates, can lock in the contour point subset that best represents the true direction of the seam, thereby significantly improving the stability and directional accuracy of subsequent straight line fitting.

[0098] D3: Using the least squares method or RANSAC algorithm, perform linear fitting on the subset of contour points to determine the seam detection results.

[0099] The seam detection result refers to a continuous, smooth seam line. It should be noted that in practical applications, multiple seam detection results can be determined based on multiple frames of the image to be detected, and the ByteTrack multi-target tracking algorithm can be used to assign a unique and persistent ID to each seam detection result. ByteTrack is a high-performance tracker that can maintain its ID even when the target is briefly occluded or the detection result fluctuates slightly.

[0100] It should be noted that after obtaining the preliminary boundary detection results and joint detection results through steps S204 and S205, in order to improve their continuity and structural rationality in complex environments, this application can further apply geometric consistency constraints (including directional consistency adjustment and spatial position constraints) to the boundary detection results and spatial topological relationship constraints (including directional consistency adjustment and spatial interval constraints) to the joint detection results based on the prior information of the physical structure of the photovoltaic module, thereby obtaining corrected boundary detection results and corrected joint detection results. Specifically, this can include the following steps: E1: Determine the reference direction parameters based on the overall distribution pattern of the detection results at each boundary within the same photovoltaic module row.

[0101] In one specific implementation, firstly, the equation of the straight line corresponding to each boundary detection result within the same row of photovoltaic modules is determined. This equation can be expressed in the slope-intercept form y = kx + b. Subsequently, based on the slope of the straight line equation corresponding to each boundary detection result, the average slope of the photovoltaic module row is determined as a reference direction parameter.

[0102] In another specific implementation, firstly, the direction vector of the line corresponding to each boundary detection result within the same row of photovoltaic modules is determined. Then, the average direction vector of each line direction vector is used as the reference direction parameter.

[0103] It should be noted that, regardless of whether the average slope or the average direction vector is used, both can be used to characterize the overall extension direction of the boundary detection results within the same photovoltaic module row.

[0104] E2: Based on the reference direction parameters, adjust the direction consistency of the detection results for each boundary within the same photovoltaic module row.

[0105] In one specific implementation, when the reference direction parameter is the average slope, the slope of the straight line equation corresponding to each boundary detection result within the same photovoltaic module row is corrected to the average slope, so that the boundary detection results remain parallel.

[0106] In another specific implementation, when the reference direction parameter is the average direction vector, the direction vectors of all the lines corresponding to the boundary detection results within the same photovoltaic module row are corrected to the average direction vector so that the boundary detection results remain parallel.

[0107] It should be noted that the reason for making the above-mentioned directional consistency adjustment is that, based on the prior information of the physical structure of photovoltaic modules, within the same row of photovoltaic modules, the left and right boundaries should be approximately parallel in physical structure, and their directional changes should be consistent. Therefore, this application only needs to ensure that the extension direction of each boundary detection result meets the consistency constraint; this application does not limit whether to use the slope method or the direction vector method.

[0108] E3: Based on the preset physical size information of the photovoltaic module, spatial position constraints are applied to the relative positions between each adjusted boundary detection result to obtain the corrected boundary detection result.

[0109] Each boundary detection result has a corresponding label, including the left boundary line and the right boundary line. Based on the straight line equations or straight line direction vectors of the adjusted left and right boundary lines, the vertical distance between the adjusted left and right boundary lines of the photovoltaic module is calculated.

[0110] If the difference between the vertical distance and the standard width W of the photovoltaic module exceeds a preset tolerance threshold, the adjusted left boundary line and / or the adjusted right boundary line are translated in the normal direction according to the standard width W to keep the spacing between the left and right boundary lines constant, thereby obtaining the corrected boundary detection result. The standard width W is a preset physical dimension of the photovoltaic module, which can be obtained through training data, calibration data, or prior structural parameters.

[0111] It should be noted that the reason for imposing spatial constraints is that, based on prior information about the physical structure of photovoltaic modules, the distance between the left and right boundaries should remain relatively stable within the same row of photovoltaic modules, without abrupt changes or abnormal scaling. Therefore, this application introduces a standard width W to constrain the vertical distance, thereby improving the structural rationality of the boundary detection results. This application does not limit the specific spacing correction method.

[0112] It should also be noted that if only one boundary detection result of the photovoltaic module is detected, or if one boundary detection result is in good condition while the other boundary detection result is largely broken, then another boundary line parallel to it can be generated based on the detected boundary detection result and the standard width W of the photovoltaic module. Taking the detection of only the left boundary line and not the right boundary line as an example, this application can translate the standard width W along the normal direction of the left boundary line based on the standard width W of the photovoltaic module, and automatically generate another parallel right boundary line.

[0113] E4: Adjust the direction consistency of each joint inspection result according to the reference direction parameters.

[0114] In one specific implementation, when the reference direction parameter is the average slope, the slope of the straight line equation corresponding to each joint detection result is corrected to the average slope.

[0115] In another specific implementation, when the reference direction parameter is the average direction vector, the direction vector of the line corresponding to each joint detection result is corrected to the average direction vector.

[0116] By using the above method, the overall direction of the seam detection results is kept consistent with the overall direction of the boundary detection results, thereby achieving global parallelism of the seam lines.

[0117] E5: Based on the effective area defined by the corrected boundary detection results, spatial interval constraints are applied to each joint detection result.

[0118] The adjusted seam detection results are subjected to interval constraints to ensure that the horizontal projections of the two endpoints corresponding to the adjusted seam detection results are both within the effective area defined by the left and right boundary lines (corrected boundary detection results). For portions exceeding the effective area, truncation is performed to obtain the corrected seam detection results.

[0119] The reason for adjusting the direction consistency and spatial range constraints is that the overall direction of the joints of the photovoltaic module should be basically consistent with the direction of the left and right boundaries, and the boundary and the joint should meet the hierarchical relationship of "boundary outside, joint inside" in terms of spatial topology, so as to avoid situations such as joints crossing boundaries or boundaries intersecting, which do not conform to the actual module structure.

[0120] It should also be noted that if a break is detected in the joint detection result of the photovoltaic module (for example, a joint line should extend from left to right, but is broken in the middle due to dirt or reflection), this application can extend the detected joint line segment along a straight line until it intersects with the left and right boundary lines to obtain a complete joint detection result.

[0121] It should also be noted that if the detection result of the photovoltaic module seam is found to be off-center and outside the effective area enclosed by the left and right boundary lines, the seam detection result will be deleted directly.

[0122] It should also be noted that if small jagged jitter is detected in the boundary detection results or joint detection results of photovoltaic modules, this application can use a moving average or filtering algorithm to smooth the boundary detection results or joint detection results to obtain a cleaner and more stable linear expression.

[0123] It is understandable that the boundaries and seams of photovoltaic modules appear as straight lines in images. Relying solely on low-level visual features such as pixel gradients and edge detection makes it difficult to effectively distinguish between the two, easily leading to misidentification of seams as boundaries or vice versa. Such misidentification can cause the robot to misunderstand the structure of the photovoltaic module, thus affecting subsequent path planning. To address this, this application, through the processing steps E1-E5 described above, utilizes the hierarchical relationship between boundaries and seams in physical space (boundaries on the outside, seams on the inside) and directional consistency requirements to effectively eliminate category misidentification caused by similar pixel features, thereby improving the accuracy of target detection.

[0124] After the processing in steps E1-E5, the boundary detection and seam detection results of a single frame are geometrically reasonable. To further improve stability in dynamic inspection scenarios and suppress transient jitter caused by instantaneous illumination, reflection, or brief occlusion, steps E6-E8 need to be performed to introduce temporal consistency for multi-frame joint optimization. E6: Based on the temporal relationship between the corrected boundary detection results and the corrected joint detection results, establish boundary detection sequences and joint detection sequences respectively.

[0125] E7: For the corrected boundary detection results and corrected seam detection results of consecutive frames in the boundary detection sequence and seam detection sequence, cross-frame consistency constraints are applied to obtain the constrained boundary detection sequence and seam detection sequence.

[0126] Specifically, cross-frame consistency constraints include the following three aspects: Firstly, the positional and angular changes of the same corrected boundary detection result, and the positional and angular changes of the same corrected seam detection result, are determined between adjacent frames. If the positional or angular change exceeds the corresponding threshold (derived from the robot's motion speed), the corrected boundary detection result or corrected seam detection result of that adjacent frame is determined to be abnormal, and the corrected boundary detection result or corrected seam detection result of the adjacent frame is directly deleted.

[0127] Secondly, determine whether the same corrected boundary detection result, or the same corrected seam detection result, exists within N consecutive frames (N is a positive integer, such as 5 frames). If not, determine that the corrected boundary detection result or the corrected seam detection result of the adjacent frame contains random noise or false detection, and directly delete the corrected boundary detection result or the corrected seam detection result of the adjacent frame.

[0128] Thirdly, the boundary detection sequence and seam detection sequence, after being filtered by the first and second aspects, are subjected to Kalman filtering or sliding window midpoint filtering to smooth the observation value of the current frame with information from multiple previous and subsequent frames, thereby effectively suppressing instantaneous jitter.

[0129] Therefore, by applying the above three cross-frame consistency constraints to the boundary detection sequence and the seam detection sequence, the constrained boundary detection sequence and the constrained seam detection sequence can be obtained.

[0130] E8: Based on the robot's motion direction, the constrained boundary detection sequence and the constrained seam detection sequence are eliminated to obtain the target boundary detection sequence and the target seam detection sequence.

[0131] Utilizing prior knowledge of the robot's unidirectional, orderly movement along the photovoltaic array, the constrained boundary detection sequence and constrained seam detection sequence are unidirectionally verified based on the robot's movement direction. Taking the robot's movement from left to right as an example, the X-coordinate values ​​of the boundary detection results and seam detection results need to increase incrementally. Therefore, the X-coordinate of a newly appearing boundary detection result in frame t+1 must be greater than the X-coordinate of any boundary detection result in frame t. If a boundary detection result with a smaller X-coordinate suddenly appears in frame t+1, this is physically impossible; it must be a false detection (e.g., a phantom caused by reflection) and must be discarded.

[0132] It should be noted that after obtaining the target boundary detection sequence and the target seam detection sequence, the detection results of each seam in the target seam detection sequence need to be sorted {S1, S2, ..., Sn}. The area between each two adjacent seam detection results (i.e., between Si and Si+1) corresponds to an independent photovoltaic module. Therefore, based on the target seam detection sequence, a unique incremental module number (e.g., 1, 2, 3, ...) can be sequentially assigned to each photovoltaic module, completing automated module counting and supporting module-level relative positioning without RTK.

[0133] Understandably, this application, by establishing the temporal relationship between the corrected boundary detection results and the seam detection results and applying cross-frame consistency constraints, can effectively suppress instantaneous noise and occasional false detections in outdoor photovoltaic scenes, such as false detections caused by reflections, stains, brief occlusions, or camera shake, thereby improving the accuracy of target detection. Furthermore, this application also performs unidirectional verification on the constrained boundary detection sequence and the constrained seam detection sequence based on the robot's actual movement direction, which can eliminate illusory false detections that are difficult for traditional single-frame detection methods to handle, thus improving the accuracy of target detection.

[0134] S206: For the third mask image labeled as "block" in the mask image set, perform the following: Based on the corrected boundary detection results and the corrected seam detection results, obtain the block detection results in the third mask image.

[0135] The clamping block refers to the fixture for photovoltaic modules. One end of the clamping block is used to press down the boundary of the photovoltaic module, and the other end is fixedly connected to the guide rail below by bolts, thereby firmly fixing the photovoltaic module on the guide rail and preventing the photovoltaic module from sliding or falling off.

[0136] For the third mask image labeled "block" in the removed mask image set, perform the following steps. It is understood that steps B1-B4 can also be omitted, and the third mask image labeled "block" can be directly determined from the mask image set and the following steps can be performed. This application does not limit this.

[0137] Specifically, the corrected boundary detection results are used to determine the location of the outer boundary of the photovoltaic module, and the corrected seam detection results are used to determine the location of the inner seam of the photovoltaic module. Since the pressure block is usually installed at the boundary of the photovoltaic module or near the seam between adjacent photovoltaic modules, the pressure block detection results in the third mask image can be obtained using the corrected boundary detection results and the corrected seam detection results.

[0138] If, based on the block detection results, the number of blocks corresponding to a corrected boundary detection result or a corrected joint detection result is greater than the sixth threshold (e.g., 2 blocks), then the blocks at that boundary or joint are considered complete and without missing blocks, meeting safety standards. If, based on the block detection results, the number of blocks corresponding to a corrected boundary detection result or a corrected joint detection result is less than or equal to the sixth threshold, then there is a safety hazard of insufficient or missing blocks at that boundary or joint, requiring an alarm to be triggered.

[0139] See Figure 3This figure is a schematic diagram of target detection provided in an embodiment of this application. It should be noted that after determining the boundary, seam, and pressure block using the above method, an inverse perspective transformation can be performed using the rotation matrix R and translation vector T (or homography matrix H) obtained by extrinsic parameter calibration, so that the geometric results of the output boundary, seam, and pressure block are parallel to the plane of the photovoltaic module, which facilitates integration with the robot path planning module.

[0140] In summary, this application provides a target detection method. This method inputs the image to be detected into a segmentation model to obtain a set of mask images with various labels such as boundaries, seams, and bridging blocks, achieving rapid and accurate separation and extraction of boundary features, seam features, and bridging block features. Firstly, for the first mask image with boundary labels, direct linear fitting of contour points is performed, enabling rapid and accurate determination of boundary detection results. Secondly, for the second mask image with seam labels, this application first filters a subset of contour points with ordinates greater than a first threshold, and then performs linear fitting of the contour point subset, enabling rapid and accurate determination of seam detection results. These seam detection results more realistically reflect the direction and position of the seam. Thirdly, for the third mask image with bridging block labels, this application can diagnose whether the number of bridging blocks installed complies with safety regulations, thereby ensuring the safety of photovoltaic modules. Therefore, this application comprehensively improves the automation level and reliability of photovoltaic power plant operation and maintenance inspection.

[0141] See Figure 4 The figure is a schematic diagram of a target detection device provided in an embodiment of this application. The target detection device 400 includes: an image acquisition module 401, an image segmentation module 402, and an image integration module 403; Image acquisition module 401 is used to acquire the image of the photovoltaic module to be inspected; The image segmentation module 402 is used to input the image to be detected into the segmentation model, and to segment the image to be detected into multiple target bounding boxes through the segmentation model. Each target bounding box includes a mask image of a target object. The target object includes the boundary and seam of the photovoltaic module. Each mask image has a label corresponding to the target object. The image integration module 403 is used to obtain a mask image set by integrating each mask image; the mask image set includes mask images with two types of labels, each label corresponding to a target object.

[0142] In one specific implementation, the target detection device 400 further includes: a first determining module and a second determining module; The first determination module is used to: for the first mask image labeled as a boundary in the mask image set, perform the following: obtain the set of contour points in the first mask image; perform line fitting on the set of contour points to determine the boundary detection result; The second determining module is used to perform the following steps for a second mask image labeled as a seam in the mask image set: obtaining a set of contour points in the second mask image; filtering out a subset of contour points whose ordinate is greater than a second threshold from the set of contour points; and performing linear fitting on the subset of contour points to determine the seam detection result.

[0143] In one specific implementation, the target detection device 400 further includes: a first acquisition module; The first acquisition module is used to perform geometric consistency constraints on the boundary detection results and spatial topological relationship constraints on the joint detection results based on the prior information of the physical structure of the photovoltaic module, so as to obtain the corrected boundary detection results and the corrected joint detection results.

[0144] In one specific implementation, the first acquisition module is specifically used to: determine the reference direction parameter based on the overall distribution pattern of each boundary detection result within the same photovoltaic module row; adjust the direction consistency of each boundary detection result according to the reference direction parameter; and constrain the relative position between each adjusted boundary detection result according to the preset physical size information of the photovoltaic module.

[0145] In one specific implementation, the first acquisition module is specifically used to: adjust the orientation consistency of each joint detection result according to the reference orientation parameters; and constrain the spatial interval of each joint detection result according to the effective area defined by the corrected boundary detection result.

[0146] In one specific implementation, the target detection device 400 further includes: a sequence establishment module, a second acquisition module, and a third acquisition module; The sequence establishment module is used to establish boundary detection sequences and joint detection sequences based on the temporal relationship between the corrected boundary detection results and the corrected joint detection results. The second acquisition module is used to perform cross-frame consistency constraints on the corrected boundary detection results of consecutive frames in the boundary detection sequence and the corrected seam detection results of consecutive frames in the seam detection sequence, respectively, to obtain the constrained boundary detection sequence and the constrained seam detection sequence. The third acquisition module is used to eliminate the constrained boundary detection sequence and the constrained seam detection sequence according to the robot's motion direction, so as to obtain the target boundary detection sequence and the target seam detection sequence.

[0147] In one specific implementation, the target object further includes a block; the mask image set includes mask images with three labels; the target detection device 400 further includes a third determining module; the third determining module is used to: for the third mask image in the mask image set with the label "block", perform: obtain the block detection result in the third mask image based on the corrected boundary detection result and the corrected seam detection result.

[0148] In one specific implementation, the first determining module is specifically used to: generate a first superimposed image based on the pixel values ​​of each pixel in the first mask image and the image to be detected; perform binarization processing on the first superimposed image to obtain a first binarized image; obtain a set of contour points in the first binarized image; perform line fitting on the set of contour points to obtain candidate detection results; the candidate detection results include at least one contour line; and determine the boundary detection results by filtering out contour lines with a number of pixels less than a first threshold from the candidate detection results.

[0149] In one specific implementation, the second determining module is specifically used to: shrink the mask region of the second mask image inward by M pixels to obtain a candidate mask image; M is a positive integer; generate a second superimposed image based on the pixel values ​​of each pixel point of the candidate mask image and the image to be detected; perform binarization processing on the second superimposed image to obtain a second binarized image; and obtain the set of contour points in the second binarized image.

[0150] In one specific implementation, the target detection device 400 further includes: an image removal module; The image removal module is used to generate a merged image based on the pixel value of each pixel in each mask image in the mask image set and the number of mask images in the mask image set; perform binarization processing on the merged image to obtain a binarized merged image; determine the area of ​​the target region in the binarized merged image; the target region is a connected region with a pixel value of 255; if the area of ​​the region is greater than a third threshold, then by traversing the target region in each mask image in the mask image set, a subset of mask images with non-zero pixel values ​​in the target region is determined, and the subset of mask images is removed to obtain a set of removed mask images.

[0151] In summary, this application provides a target detection device. This device inputs the image to be detected into a segmentation model to obtain a set of mask images with various labels such as boundaries, seams, and blocks, achieving rapid and accurate separation and extraction of boundary features, seam features, and block features. Firstly, for the first mask image with boundary labels, direct linear fitting of contour points is performed, enabling rapid and accurate determination of boundary detection results. Secondly, for the second mask image with seam labels, this application first filters a subset of contour points with ordinates greater than a first threshold, and then performs linear fitting of the contour point subset, enabling rapid and accurate determination of seam detection results. These seam detection results more realistically reflect the direction and position of the seam. Thirdly, for the third mask image with block labels, this application can diagnose whether the number of installed blocks complies with safety regulations, thereby ensuring the safety of photovoltaic modules. Therefore, this application comprehensively improves the automation level and reliability of photovoltaic power plant operation and maintenance inspection.

[0152] This application discloses a robot, which includes a memory and a processor; the memory is used to store computer programs or computer instructions; the processor is used to execute the computer programs or computer instructions stored in the memory, causing the robot to perform the target detection method as described in the first aspect.

[0153] The robot provided in this application embodiment has the beneficial effects of the target detection method described above.

[0154] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0155] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A target detection method, characterized in that, The method includes: Acquire the image of the photovoltaic module to be inspected; The image to be detected is input into a segmentation model, which segments the image into multiple target bounding boxes. Each target bounding box includes a mask image of a target object. The target object includes the boundary and seam of the photovoltaic module. Each mask image has a label corresponding to the target object. By integrating each of the mask images, a mask image set is obtained; the mask image set includes mask images with two types of labels, each label corresponding to a target object.

2. The method according to claim 1, characterized in that, The method further includes: For the first mask image labeled as a boundary in the mask image set, perform the following steps: obtain the set of contour points in the first mask image; perform line fitting on the set of contour points to determine the boundary detection result; For the second mask image labeled as a seam in the mask image set, the following steps are performed: obtain the set of contour points in the second mask image; filter out a subset of contour points whose ordinate is greater than a second threshold from the set of contour points; perform line fitting on the subset of contour points to determine the seam detection result.

3. The method according to claim 2, characterized in that, The method further includes: Based on the prior information of the physical structure of the photovoltaic module, geometric consistency constraints are applied to the boundary detection results, and spatial topological relationship constraints are applied to the joint detection results, to obtain corrected boundary detection results and corrected joint detection results.

4. The method according to claim 3, characterized in that, The step of applying geometric consistency constraints to the boundary detection results based on prior information about the physical structure of the photovoltaic module includes: Based on the overall distribution pattern of the boundary detection results within the same photovoltaic module row, the reference direction parameters are determined. Based on the reference direction parameters, the orientation consistency of each boundary detection result is adjusted. Based on the preset physical size information of the photovoltaic module, spatial position constraints are applied to the relative positions between each of the adjusted boundary detection results.

5. The method according to claim 4, characterized in that, The spatial topological constraint on the joint detection results includes: Based on the reference direction parameters, the directional consistency of each joint detection result is adjusted; Based on the effective area defined by the corrected boundary detection results, spatial interval constraints are applied to each of the joint detection results.

6. The method according to claim 3, characterized in that, The method further includes: Based on the temporal relationship between the corrected boundary detection results and the corrected seam detection results, a boundary detection sequence and a seam detection sequence are established. For the corrected boundary detection results of consecutive frames in the boundary detection sequence and the corrected seam detection results of consecutive frames in the seam detection sequence, cross-frame consistency constraints are applied to obtain constrained boundary detection sequences and constrained seam detection sequences. Based on the robot's motion direction, the constrained boundary detection sequence and the constrained seam detection sequence are eliminated to obtain the target boundary detection sequence and the target seam detection sequence.

7. The method according to claim 3, characterized in that, The target object also includes a compressed block; the mask image set includes mask images of three types of labels; the method further includes: For the third mask image labeled as "block" in the mask image set, perform the following: Based on the corrected boundary detection result and the corrected seam detection result, obtain the block detection result in the third mask image.

8. The method according to any one of claims 2-7, characterized in that, The step of obtaining the set of contour points in the first mask image includes: A first overlay image is generated based on the pixel values ​​of each pixel in the first mask image and the image to be detected; The first superimposed image is binarized to obtain a first binarized image; Obtain the set of contour points in the first binarized image; The step of performing straight line fitting on the contour point set to determine the boundary detection result includes: A straight line is fitted to the set of contour points to obtain candidate detection results; the candidate detection results include at least one contour line. The boundary detection result is determined by filtering out contour lines with fewer than a first threshold number of pixels from the candidate detection results.

9. The method according to any one of claims 2-7, characterized in that, The step of obtaining the set of contour points in the second mask image includes: Shrink the mask region of the second mask image inward by M pixels to obtain a candidate mask image; M is a positive integer; A second overlay image is generated based on the pixel values ​​of each pixel in the candidate mask image and the image to be detected; The second overlay image is binarized to obtain a second binarized image; Obtain the set of contour points in the second binarized image.

10. The method according to any one of claims 1-7, characterized in that, The method further includes: A merged image is generated based on the pixel value of each pixel in each mask image in the mask image set, and the number of mask images in the mask image set; The merged image is binarized to obtain a binarized merged image; Determine the area of ​​the target region in the merged image after binarization; the target region is a connected region with a pixel value of 255. If the area of ​​the region is greater than the third threshold, then by traversing the target region in each mask image in the mask image set, a subset of mask images with non-zero pixel values ​​in the target region is determined, and the subset of mask images is removed to obtain a set of mask images after removal.

11. A target detection device, characterized in that, The device includes: an image acquisition module, an image segmentation module, and an image integration module; The image acquisition module is used to acquire the image of the photovoltaic module to be detected; The image segmentation module is used to input the image to be detected into a segmentation model, and segment the image to be detected into multiple target bounding boxes through the segmentation model. Each target bounding box includes a mask image of a target object. The target object includes the boundary and seam of the photovoltaic module. Each mask image has a label corresponding to the target object. The image integration module is used to integrate each of the mask images to obtain a mask image set; the mask image set includes mask images with two types of labels, each label corresponding to a target object.

12. A robot, characterized in that, The robot includes a memory and a processor; The memory is used to store computer programs or computer instructions; The processor is configured to execute a computer program or computer instructions stored in the memory, causing the robot to perform the target detection method as described in any one of claims 1 to 10.

Images