Image stitching method, computer readable storage medium and computer program product

Abstract

The present application provides an image stitching method, a computer-readable storage medium and a computer program product. A specific implementation of the method includes: obtaining at least two images to be processed; for each image to be processed, determining candidate corner points of an object to be stitched in the image to be processed; and verifying the candidate corner points, and determining target corner points of the object to be stitched according to the verification result; and performing perspective transformation on the image area surrounded by the target corner points in the image to be processed to obtain an image to be stitched; stitching at least two images to be stitched to obtain a target image. The method can improve image distortion, stitching robustness and stitching speed.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to an image stitching method, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the rapid development of visual technology, multiple partial images of an object can be stitched together to obtain a complete image of the object, so as to facilitate subsequent relevant analysis of the object based on the complete image. For example, on the production line of photovoltaic modules, it is necessary to collect images of photovoltaic modules for defect detection, but the size of photovoltaic modules is large, so multiple partial images of photovoltaic modules can be collected, and then the multiple partial images can be stitched together to obtain a complete image of the photovoltaic module, so as to facilitate defect detection through the complete image.

[0003] In the related art, the corner positions of the objects to be spliced ​​in each partial image can be detected, and then multiple partial images can be spliced ​​based on the corner position. However, the accuracy of the detected corner position is low, resulting in distortion of the obtained complete image, and the splicing process has the problems of poor robustness and slow splicing speed. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an image stitching method, a computer-readable storage medium, and a computer program product, so as to improve image distortion, enhance stitching robustness, and enhance stitching speed.

[0005] In the first aspect, the embodiment of the present application provides an image stitching method, the method comprising: obtaining at least two images to be processed; for each image to be processed, determining candidate corner points of an object to be stitched in the image to be processed; and, verifying the candidate corner points, and determining target corner points of the object to be stitched according to the verification result; and, performing perspective transformation on the image area surrounded by the target corner points in the image to be processed to obtain an image to be stitched; stitching at least two images to be stitched to obtain a target image. In this way, since the accuracy of the position of the target corner point obtained after verification is high, a more accurate image to be stitched can be obtained after perspective transformation using the target corner point, thereby improving image distortion and improving stitching robustness. Furthermore, the corresponding image to be stitched can be obtained only through the relevant information of the current image to be processed, and then stitching can be performed. In this way, there is no need to rely on the relevant information of other images to be processed. After obtaining an image to be processed, the image to be stitched corresponding to the image to be processed can be stitched with other images to be stitched. Compared with the method of stitching after all images to be processed are obtained and the corresponding images to be stitched are obtained, the stitching speed is effectively improved.

[0006] In the second aspect, the embodiment of the present application provides an image stitching device, which includes: an acquisition module for acquiring at least two images to be processed; a determination module for determining, for each image to be processed, candidate corner points of an object to be stitched in the image to be processed; a verification module for verifying the candidate corner points and determining the target corner points of the object to be stitched according to the verification results; a transformation module for performing perspective transformation on the image area surrounded by the target corner points in the image to be processed to obtain an image to be stitched; and a stitching module for stitching at least two images to be stitched to obtain a target image. In this way, since the accuracy of the position of the target corner point obtained after verification is high, a more accurate image to be stitched can be obtained after perspective transformation using the target corner point, thereby improving image distortion and improving stitching robustness. Furthermore, the corresponding image to be stitched can be obtained only through the relevant information of the current image to be processed, and then stitching can be performed. In this way, there is no need to rely on the relevant information of other images to be processed. After obtaining an image to be processed, the image to be stitched corresponding to the image to be processed can be stitched with other images to be stitched. Compared with the method of stitching after all images to be processed are obtained and the corresponding images to be stitched are obtained, the stitching speed is effectively improved.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are performed.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are performed.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, the method described in the first aspect is performed.

[0010] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 A flowchart of an image stitching method provided in an embodiment of the present application;

[0013] Figure 2 A partial image of a photovoltaic module provided in an embodiment of the present application;

[0014] Figure 3 A schematic diagram of candidate corner point positions of a photovoltaic module provided in an embodiment of the present application;

[0015] Figure 4 An image to be spliced ​​corresponding to a photovoltaic assembly provided in an embodiment of the present application;

[0016] Figure 5 A target image corresponding to a photovoltaic assembly provided in an embodiment of the present application;

[0017] Figure 6 A structural block diagram of an image stitching device provided in an embodiment of the present application;

[0018] Figure 7 A schematic diagram of the structure of an electronic device for executing an image stitching method provided in an embodiment of the present application;

[0019] 1-cell corner, 2-busbar, 3-cell, 4-cell gap, 5-frame, 6-first boundary busbar, 7-second boundary busbar, 601-acquisition module, 602-determination module, 603-verification module, 604-transformation module, 605-splicing module, 701-processor, 702-communication interface, 703-memory, 704-communication bus. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0021] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0022] It should be noted that the embodiments in this application or the technical features in the embodiments may be combined if there is no conflict.

[0023] For ease of understanding, the present application first uses photovoltaic modules as an example to provide an overview of the present application scheme. For example, in order to perform defect detection on a photovoltaic module, its image can be collected. Since the size of a photovoltaic module is generally large, multiple local images can be collected, and then the multiple local images can be stitched together to obtain a complete image of the photovoltaic module, so as to perform defect detection on the photovoltaic module based on the complete image. In the process of stitching multiple local images, the corner points existing in each local image can be first determined, that is, the positions in the image with the local maximum curvature or obvious gradient changes. These corner points can exist in different local images due to the different camera acquisition positions, so that different local images can be stitched together based on these corner points.

[0024] Furthermore, each corner point may be located at the intersection of two edges in the local image. If the two edges are the edges of the photovoltaic module to be detected area in the local image, multiple areas to be detected may be spliced ​​together after splicing the corner point with other local images. Then, after the corresponding areas to be detected in multiple local images are spliced, a complete image for detection may be obtained.

[0025] In the related art, due to the low accuracy of the detected corner point positions, the obtained complete image is distorted, and the splicing process has the problems of poor robustness and slow splicing speed. In order to solve this problem, the present application provides an image splicing method, a computer-readable storage medium and a computer program product. Furthermore, by first determining the candidate corner points of the object to be spliced ​​in each image to be processed, then verifying the candidate corner points, and determining the target corner points according to the verification results, the target corner points obtained after verification have high accuracy, so after using the target corner points for perspective transformation, a size-consistent and more accurate image to be spliced ​​can be obtained, thereby improving the image distortion and improving the robustness of splicing. In addition, the corresponding image to be spliced ​​can be obtained only through the relevant information of the current image to be processed, and then the splicing can be performed. In this way, there is no need to rely on the relevant information of other images to be processed. After obtaining an image to be processed, the image to be spliced ​​corresponding to the image to be processed can be spliced ​​with other images to be spliced. Compared with the method of performing splicing after all images to be processed are obtained and the corresponding image to be spliced ​​is obtained, the splicing speed is effectively improved.

[0026] The defects existing in the solutions in the above-mentioned related technologies are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed in the embodiments of the present invention below for the above-mentioned problems should all be the contributions made by the inventor to the present invention during the process of the present invention.

[0027] In some application scenarios, the above-mentioned image stitching method can be applied to a terminal device, which may include, for example, a PC, a tablet computer, a mobile phone, etc.; in other application scenarios, the above-mentioned image stitching method can also be applied to a server device, which may include, for example, a server, a server cluster, or a cloud platform, etc. In addition, the above-mentioned terminal device can be a physical device or a virtual device, such as a virtual machine. Exemplarily, the present application is described below by applying to a server.

[0028] Please refer to Figure 1 , which shows a flow chart of an image stitching method provided by an embodiment of the present application. Figure 1 As shown, the image stitching method includes the following steps 101 to 105.

[0029] Step 101, obtaining at least two images to be processed;

[0030] The above-mentioned image to be processed may be a partial image of an industrial product or other product, and the image to be processed may include an object to be spliced. The object to be spliced ​​may be, for example, the entirety or part of a larger component such as a photovoltaic component or a chip component, such as the entire photovoltaic component or a cell array of a photovoltaic component.

[0031] Furthermore, the image to be processed may be, for example, an image currently taken by a camera, or a historical image taken by a camera, which is not limited here.

[0032] Furthermore, the server may, for example, continuously acquire at least two images to be processed from a camera, or acquire at least two images to be processed from a preset historical image storage location.

[0033] In some application scenarios, for example, the camera can collect different partial images of the product at different positions (that is, different images to be processed), and these partial images may include images of the same partial parts of the components, or may not include images of the same partial parts of the product. That is, multiple images to be processed may overlap partially or not, and this application does not limit this.

[0034] Step 102, for each image to be processed, determining candidate corner points of the object to be spliced ​​in the image to be processed;

[0035] The candidate corner points can be regarded as the target corner points that are initially determined, and these candidate corner points to be stitched may be used to stitch out a complete image of the product. For example, if the object to be stitched is a cell array in a rectangular photovoltaic module, the candidate corner points can be, for example, the four vertices in the rectangular area where the cell array is located in the image to be processed.

[0036] In some application scenarios, for example, candidate corner points may be determined by a preset image processing algorithm.

[0037] Step 103, verifying the candidate corner points, and determining the target corner points of the object to be spliced ​​according to the verification results;

[0038] The above verification result may include passing the verification or failing the verification. Among them, for the candidate corner points that pass the verification, they can be directly determined as the target corner points; for the candidate corner points that fail the verification, they can be corrected to obtain the target corner points.

[0039] Then, the target corner points may include candidate corner points that have passed verification and / or correction procedures.

[0040] Step 104, performing perspective transformation on the image area surrounded by the target corner points in the image to be processed to obtain an image to be stitched;

[0041] The image region surrounded by the target corner points is the image region that can present the object to be spliced ​​more accurately and completely.

[0042] In some application scenarios, after performing perspective transformation on an image region surrounded by multiple target corner points, multiple images of the same size can be obtained. These images of the same size are also the images to be stitched.

[0043] In these application scenarios, for each image area surrounded by the target corner points, if the coordinates of the target corner points include (X 1 , Y 1 )、(X 2 , Y 2 )、(X 3 , Y 3 )、(X 4 , Y 4 ), the coordinates of the corresponding corner points in the image area after perspective transformation include (0, 0), (w, 0), (0, h), (w, h), and the corner transformation matrix can be obtained. Then, the coordinates of each pixel in the image area surrounded by the target corner point and the corner transformation matrix can be combined to obtain the pixel coordinates after perspective transformation. Among them, if the corner transformation matrix is , the corresponding pixel coordinates can be: . Among them, the parameters A, B, C, D, E, F, G, H, and I can be any determined values, x and X represent the horizontal coordinates of each pixel point before and after the perspective transformation, and y and Y represent the vertical coordinates of each pixel point before and after the perspective transformation. In this way, each pixel point is transformed using a unified corner point transformation matrix, so that the relationship between pixels will not be changed, and the visual effect of the spliced ​​target image can be improved to a certain extent.

[0044] Step 105 , stitching at least two of the images to be stitched together to obtain a target image.

[0045] In some application scenarios, if the sizes of multiple images to be stitched after perspective transformation are the same, the multiple images to be stitched can be directly stitched; if the sizes of multiple images to be stitched after perspective transformation are different, the multiple images to be stitched can be scaled to the same size before stitching.

[0046] Furthermore, when stitching multiple images to be stitched, they can be arranged in sequence according to a pre-marked stitching order, and then the corresponding target corner points in two adjacent images to be stitched can be overlapped to achieve stitching and obtain a complete target image. In some application scenarios, if there is only one camera, the stitching order here can be, for example, the same as the acquisition order of the images to be processed, wherein the camera sequentially captures partial images of the objects to be stitched, thereby obtaining multiple images to be processed. In other application scenarios, if there are multiple cameras, the stitching order here can be, for example, the same as the arrangement order of the cameras.

[0047] In this implementation, since the position of the target corner point obtained after verification is highly accurate, after using the target corner point for perspective transformation, a to-be-stitched image with consistent size and relatively accurate can be obtained, thereby improving the image distortion and improving the robustness of the stitching. In addition, the corresponding to-be-stitched image can be obtained only through the relevant information of the current to-be-processed image, and then the stitching can be performed, so that there is no need to rely on the relevant information of other to-be-processed images. Once an image to be processed is obtained, the image to be stitched corresponding to the image to be processed can be stitched with other to-be-stitched images, which effectively improves the stitching speed compared to the method of stitching after all the to-be-processed images are obtained and the corresponding to-be-stitched images are obtained.

[0048] In some optional implementations, the image to be processed is a local image of a photovoltaic module captured by an area array camera or a movable camera, and the object to be spliced ​​is an array of solar cells in the photovoltaic module.

[0049] The above-mentioned area array camera or movable camera can be a near-infrared monochrome industrial camera or a color industrial camera. Among them, the near-infrared monochrome industrial camera can capture the EL image of the photovoltaic module; the color industrial camera can capture the appearance image of the photovoltaic module. Then, the image to be processed can be a partial EL image or a partial appearance image of the photovoltaic module.

[0050] In some application scenarios, multiple images to be processed can be captured simultaneously by an area array camera, or multiple images to be processed can be captured separately by moving a movable camera to different positions above the photovoltaic module. For example, if a partial image of a photovoltaic module is captured by an area array camera, since the photovoltaic module usually has three steps after entering the machine, four area array cameras can be used to capture the photovoltaic module entering the capture area respectively, so that the partial image captured by each area array camera each time is 1 / 12 of the complete image of the photovoltaic module.

[0051] Furthermore, a photovoltaic module usually includes a frame and a cell array embedded in the frame. When inspecting a photovoltaic module, the quality of the cell array has a greater impact on the photovoltaic module, so defect detection of the cell array is required.

[0052] Therefore, the cell array image (i.e., the image to be stitched) can be further obtained based on the corner points in the images to be processed captured by each camera, and the cell array image can be stitched to facilitate defect detection of the entire cell array of the photovoltaic module.

[0053] See also Figure 2 , which shows a partial image of a photovoltaic module provided by an embodiment of the present application (i.e., an image to be processed). Figure 2As shown, the photovoltaic module generally includes a frame 5 and a cell array embedded in the frame 5. The cell array can be formed by splicing a plurality of cells 3, a cell gap 4 exists between two adjacent cells 3, and a cell corner 1 exists at the intersection of the edges of two adjacent cells 3. In addition, there is also a cell gap 4 between the cell 3 and the frame 5, and each cell 3 has a plurality of main grid lines 2 extending in the horizontal direction.

[0054] In some application scenarios, the server may obtain at least two partial images of the photovoltaic module. Figure 2 As shown, the server can, for example, determine a plurality of cell corner points 1 in the image to be processed, and then determine the following from the plurality of cell corner points 1: Figure 3 The four candidate corner points shown (such as Figure 3 The four candidate corner points are the corner points in the circular area, namely, the cell corner point E, the cell corner point F, the cell corner point G, and the cell corner point H). If all of these four candidate corner points pass the verification, the four candidate corner points can be determined as the corresponding four target corner points. Then, the image area surrounded by the four target corner points can be transformed into a perspective, so as to obtain the following: Figure 4 Thus, for multiple partial images, after the partial images are processed in the same or similar manner, multiple images to be stitched can be obtained, and then the multiple images to be stitched can be stitched to obtain the following: Figure 5 Then, based on the target image, the entire array of cells of the photovoltaic module can be inspected.

[0055] It should be noted that, for the sake of convenience, the present application will also be described below by taking the determination of four candidate corner points of the photovoltaic module in the image to be processed as an example, wherein the horizontal direction described is the same as Figures 2 to 5 The horizontal direction is the same as in Figures 2 to 5 The vertical direction is the same.

[0056] It should be noted that the number of candidate corner points of the photovoltaic module may not be limited to 4, but may be more or less. In addition, the direction of the photovoltaic module in the image to be processed may also be different from Figures 2 to 5 The present application does not limit the other directions.

[0057] In some optional implementations, for example, a full image search may be performed on the image to be processed to obtain candidate corner points of the object to be spliced. For example, the image to be processed may be input into a prediction model to search for candidate corner points of the object to be spliced ​​through the prediction model. The prediction model may include, for example, a deep convolutional neural network model.

[0058] In some other optional implementations, for example, the positions of the candidate corner points may be roughly determined first, and then the candidate corner points may be precisely found at the rough positions. In this way, the step 102 described above for each image to be processed, determining the candidate corner points of the object to be spliced ​​in the image to be processed, includes:

[0059] Sub-step 1021, for each image to be processed, determining a rough position of a candidate corner point in the image to be processed;

[0060] The above rough position is also the approximate position of the candidate corner point. It should be noted that if there are 4 candidate corner points in the image to be processed, each candidate corner point has a corresponding rough position. In addition, the process of determining the rough position of the candidate corner point is described in detail below in this application, which will not be repeated here.

[0061] Sub-step 1022: determining the candidate corner points according to the rough positions of the candidate corner points.

[0062] In some application scenarios, for example, the rough position of a candidate corner point may be determined as its actual position, and then the corner point of the cell located at the rough position may be directly determined as the candidate corner point.

[0063] In some other application scenarios, for example, candidate corner points may also be found near a rough position.

[0064] In this implementation, by first determining the rough position of the candidate corner point, the determination rate can be increased, so that the candidate corner point can be determined more quickly; and by determining the candidate corner point based on the rough position, the candidate corner point can be searched more finely, so that more accurate candidate corner points can be obtained. That is, this implementation can determine more accurate candidate corner points more quickly.

[0065] Furthermore, candidate corner points can be found near the rough position by following the steps below:

[0066] First, extract edge features of an image area within a preset range around the rough position of the candidate corner point;

[0067] The preset range around the rough position of the candidate corner point, that is, the preset range near the rough position, may include, for example, a range extending outward from the rough position by 1 / 16, 1 / 9 of the preset cell size, or a fixed range of 50 pixels or 100 pixels.

[0068] In some application scenarios, for example, edge features can be extracted by using a Sobel operator or a Canny operator. In these application scenarios, for example, an image area within a preset range around the rough position of the candidate corner point can be first divided, and then a threshold segmentation operation is performed on the image area to make the edge features in the image area more obvious, and then the edge features of the image area after the threshold segmentation operation is performed are extracted, which can improve the accuracy of the edge features to a certain extent. The above-mentioned threshold segmentation operation can, for example, include performing a binarization algorithm, a watershed algorithm, or other operations that can substantially distinguish between the cell array area and the non-cell array area.

[0069] Then, filtering the main grid line features on the cell in the photovoltaic module from the extracted edge features;

[0070] In some application scenarios, for example, filtering can be performed using a pre-constructed convolution kernel.

[0071] It should be noted that the main grid line (e.g. Figure 2 The area indicated by the number 2 in the middle) usually has a fixed width, so by constructing a convolution kernel of the same size as the fixed width, the busbars can be filtered out to obtain a cell array area without busbars. In this way, the probability of determining the busbars as horizontal edge lines can be reduced, and the accuracy of the candidate corner points can be improved to a certain extent.

[0072] Then, based on the filtered edge features, a horizontal edge straight line and a vertical edge straight line of the battery cell are determined;

[0073] In some application scenarios, for example, the least squares method can be used to fit multiple straight lines corresponding to the filtered edge features, and the angles of each straight line with the horizontal line can be further determined. If the angle corresponding to a straight line is 0° or near 0°, the straight line can be determined as a horizontal edge straight line, and if the angle corresponding to a straight line is 90° or near 90°, the straight line can be determined as a vertical edge straight line.

[0074] In these application scenarios, the horizontal edge straight line and the vertical edge straight line of the battery cell correspond to a battery cell gap, which may be, for example, a gap between two adjacent battery cells or a gap between a battery cell and a frame.

[0075] Finally, the intersection point of the horizontal edge straight line and the vertical edge straight line is determined as a candidate corner point.

[0076] In this implementation, the candidate corner points can be accurately found near the rough positions of the candidate corner points, and the probability of mistaking the main grid lines for horizontal edge lines is reduced, thereby improving the accuracy of the candidate corner points.

[0077] In some application scenarios, for example, the rough position of the candidate corner point may be determined by the following implementation method A, implementation method B, and implementation method C. It should be noted that the following three implementation methods are only three examples of the above image processing algorithms, and those skilled in the art may select any one of them or other implementation methods for implementation according to actual needs.

[0078] In implementation mode A, the step of determining the rough position of the candidate corner points in each image to be processed in the sub-step 1021 includes:

[0079] Step 1A, for each image to be processed, obtain the positions and types corresponding to multiple cell corner points through the prediction model;

[0080] The above prediction model may include, for example, a deep convolutional neural network model, etc. In some application scenarios, the above prediction model may be pre-trained by, for example, the following steps:

[0081] First, a plurality of sample images to be processed and sample labels corresponding to each sample image to be processed are obtained; the sample images to be processed include a plurality of sample battery cell corner points; the sample labels include the actual position and type of each sample battery cell corner point; the above types may include, for example, an upper edge corner point, a left edge corner point, a middle corner point, a bus bar corner point, etc.

[0082] Then, the sample image to be processed is used as the input of the initial prediction model, the sample label is used as the expected output of the initial prediction model, and the initial prediction model is trained to obtain a converged prediction model.

[0083] Therefore, after the image to be processed is processed by the prediction model, the position and type corresponding to each corner point of the battery cell can be obtained.

[0084] Step 2A, arranging a plurality of cell corner points in sequence according to the positions corresponding to the corner points of each cell, and obtaining the sequence information of each cell corner point;

[0085] In some application scenarios, for example, each cell corner point can be numbered in sequence to obtain the order information of each cell corner point. The above numbering can include, for example, the numbering of the cell corner points in the horizontal direction (i.e., the column numbering) and the numbering of the cell corner points in the vertical direction (i.e., the row numbering).

[0086] Step 3A: Determine the rough position of the candidate corner point according to the order information of each cell corner point and the order information and type information of the preset candidate corner points.

[0087] For example, the preset candidate corner point types are upper edge corner point, right edge corner point and bus bar corner point, which are numbered seven. Then the position of the battery cell corner point numbered seven and of the upper edge corner point, right edge corner point and bus bar corner point type can be determined as the rough position of the candidate corner point.

[0088] In this implementation, the rough positions of the candidate corner points in the image to be processed can be determined based on the preset order information and type information of the candidate corner points, thereby improving the accuracy of the candidate corner points to a certain extent.

[0089] In implementation mode B, the step of determining the rough position of the candidate corner points in each image to be processed in the sub-step 1021 includes:

[0090] Step 1B, for each image to be processed, determining a grayscale image of the image to be processed;

[0091] It should be noted that if the image to be processed is a grayscale image, the image to be processed is its grayscale image; if the image to be processed is not a grayscale image, it can be converted into a grayscale image.

[0092] Step 2B, determining a first gradient distribution sequence in the horizontal direction and a second gradient distribution sequence in the vertical direction of the pixel values ​​in the grayscale image;

[0093] In some application scenarios, for example, the grayscale mean corresponding to each row of pixels and the grayscale mean corresponding to each column of pixels in the grayscale image can be first determined, and then the difference between the grayscale means corresponding to two adjacent columns of pixels can be determined to obtain the above-mentioned first gradient distribution sequence; and then the difference between the grayscale means corresponding to two adjacent rows of pixels can be determined to obtain the above-mentioned second gradient distribution sequence;

[0094] In other application scenarios, for example, the first gradient distribution sequence and the second gradient distribution sequence can be determined respectively by the grayscale median corresponding to each row of pixels and the grayscale median corresponding to each column of pixels. The corresponding determination process is similar to the above-mentioned process of determining by grayscale mean, and will not be repeated here.

[0095] Furthermore, the first gradient distribution sequence can be used to characterize the gradient change of pixel values ​​in the horizontal direction, and the second gradient distribution sequence can be used to characterize the gradient change of pixel values ​​in the vertical direction, so that various elements existing in the cell array can be determined by the distribution of the extreme values ​​of the gradients in the first gradient distribution sequence and the second gradient distribution sequence. The various elements here can include, for example, cell gaps (e.g. Figure 2 The area indicated by number 4), the main grid line (for example Figure 2 The area indicated by the number 2), the cell area (e.g. Figure 2The area indicated by the middle label number 3), battery cell defects (such as micro cracks, black spots, contamination points, welding points, etc.), etc.

[0096] In some application scenarios, extreme values ​​can be determined from each gradient value in the first gradient distribution sequence, and the determined extreme values ​​can be used to characterize elements with large gradient changes in the horizontal direction, such as cell gaps, main grid lines, cell defects, etc. in the horizontal direction. In addition, extreme values ​​can also be determined from each gradient value in the second gradient distribution sequence, and the determined extreme values ​​are used to characterize elements with large gradient changes in the vertical direction, such as cell gaps, main grid lines, cell defects, etc. in the vertical direction.

[0097] Step 3B, determining an extreme value representing a gap between battery cells in the first gradient distribution sequence and an extreme value representing a gap between battery cells in the second gradient distribution sequence;

[0098] In some application scenarios, since the grayscale changes at the gaps of the cells in the cell array structure are usually more dramatic than the grayscale changes corresponding to other elements, the extreme values ​​corresponding to the gaps of the cells are usually greater than the extreme values ​​corresponding to other elements, and then the extreme values ​​representing the gaps of the cells can be determined by setting a threshold. Specifically, for example, a first difference threshold can be set, and when the extreme value (or the absolute value of the extreme value) in the first gradient distribution sequence is greater than the first difference threshold, the extreme value can be determined as the extreme value representing the gap of the cell. In addition, a second difference threshold can also be set, and when the extreme value (or the absolute value of the extreme value) in the second gradient distribution sequence is greater than the second difference threshold, the extreme value can be determined as the extreme value representing the gap of the cell.

[0099] In these application scenarios, since the main grid lines extend in the horizontal direction, the first gradient distribution sequence does not include extreme values ​​representing the main grid lines, while the second gradient sequence includes extreme values ​​representing the main grid lines. Since the extreme values ​​representing the main grid lines are larger than the extreme values ​​of other elements (excluding the cell gap), when determining the extreme values ​​representing the cell gap, the first difference threshold can be set to be smaller than the second difference threshold, so as to filter the extreme values ​​in the first gradient distribution sequence except the extreme values ​​representing the cell gap through the first difference threshold, and filter the other extreme values ​​in the second gradient sequence including the extreme values ​​representing the main grid lines through the second difference threshold.

[0100] In other application scenarios, of course, the first difference threshold and the second difference threshold may also be set to be the same.

[0101] Step 4B, according to the position of each extreme value representing the gap of the battery cell in the first gradient distribution sequence, a plurality of extreme values ​​representing the gap of the battery cell are arranged in sequence to obtain the order information of each extreme value representing the gap of the battery cell; according to the order information of each extreme value representing the gap of the battery cell and the horizontal order information of the preset candidate corner point, the extreme value corresponding to the candidate corner point in the horizontal direction is determined;

[0102] In some application scenarios, for example, the extreme values ​​representing the gaps between battery cells may be numbered sequentially to obtain corresponding sequence information.

[0103] Furthermore, the cell corners are usually located at the cell gap between two adjacent columns of cells, so the extreme value representing the cell gap with the corresponding number can be found in the first gradient distribution sequence according to the preset numbering of the candidate corners in the horizontal direction.

[0104] Step 5B, according to the position of each extreme value representing the gap of the battery cell in the second gradient distribution sequence, a plurality of extreme values ​​representing the gap of the battery cell are arranged in sequence to obtain the order information of each extreme value representing the gap of the battery cell; according to the order information of each extreme value representing the gap of the battery cell and the vertical order information of the preset candidate corner points, the extreme value corresponding to the candidate corner point in the vertical direction is determined;

[0105] In some application scenarios, for example, the extreme values ​​representing the gaps between the cells can be numbered in sequence to obtain the corresponding order information. Then, the extreme values ​​representing the gaps between the cells with the corresponding numbers can be found in the second gradient distribution sequence according to the numbers of the preset candidate corner points in the vertical direction.

[0106] Step 6B: Determine the rough position according to the position of the extreme value corresponding to the candidate corner point in the horizontal direction and the position of the extreme value corresponding to the candidate corner point in the vertical direction.

[0107] In the present implementation, since the candidate corner points are located at the battery cell gaps, and the grayscale changes at the battery cell gaps in the grayscale image are more dramatic, that is, there are relatively obvious gradient extremes, the rough positions of the candidate corner points are determined based on the distribution of gradient extremes in the grayscale image, making full use of the structural characteristics of the battery cell array itself, thereby improving the accuracy of the candidate corner points to a certain extent.

[0108] In implementation manner C, the step of determining the rough position of the candidate corner points in each image to be processed in the sub-step 1021 includes:

[0109] Step 1C, in the image to be processed, determining reference corner points corresponding to the candidate corner points; the positions of the reference corner points in the image to be processed are the same as the positions of the template corner points marked in the template image corresponding to the image to be processed; the template image is a partial image of the template photovoltaic module captured by an area array camera or a movable camera;

[0110] In some application scenarios, for example, a plurality of partial images of photovoltaic modules with qualified quality may be collected in advance, and the photovoltaic modules with qualified quality may be regarded as the template photovoltaic modules mentioned above.

[0111] For each partial image of the template photovoltaic module, the server can receive a marking instruction, so that the template corner point in the partial image can be marked by the marking position indicated by the marking instruction, and then based on multiple template corner points, a complete image of the cell array in the template photovoltaic module can be obtained. The above-mentioned marking instruction can be set by the operator based on experience, for example. For example, the operator can mark the position of the template corner point on the marking interface, thereby triggering the marking instruction of the corresponding position.

[0112] Furthermore, the multiple images to be processed of the photovoltaic module correspond to the multiple local images of the template photovoltaic module. For example, the image to be processed corresponding to the M region of the photovoltaic module can correspond to the local image corresponding to the M region of the template photovoltaic module. Furthermore, when the camera collects the image to be processed and the corresponding local image of the M region of the photovoltaic module, the collection method and collection position of the two are the same, for example, both are collected by the array camera of the same configuration, and both are collected after the photovoltaic module is transported to the designated position on the assembly line.

[0113] For each template corner point in the template image, its position information in the template image can be determined, and then a corner point with the same position information can be determined in the image to be processed corresponding to the template image, and the corner point is also the reference corner point corresponding to the template corner point. Therefore, the position of each reference corner point in the image to be processed is the same as the position of the corresponding template corner point in the template image.

[0114] Step 2C: determining the position of the reference corner point as the rough position of the candidate corner point.

[0115] In this implementation, the position of the reference corner point can be directly determined as the rough position of the corresponding candidate corner point. Since the reference corner point is obtained based on a relatively accurate template corner point, the position of the candidate corner point determined based on the position of the reference corner point is also more accurate.

[0116] In some optional implementations, the step of verifying the candidate corner points in step 103 and determining the target corner points of the object to be spliced ​​according to the verification result includes:

[0117] Sub-step 1031A, for each candidate corner point, determining the distance between the candidate corner point and the corresponding reference corner point; the position of the reference corner point in the image to be processed is the same as the position of the template corner point marked in the template image corresponding to the image to be processed; the template image is a partial image of the template photovoltaic module captured by an area array camera or a movable camera;

[0118] That is, after the server determines the reference corner points corresponding to the candidate corner points in the image to be processed, it can determine the distance between the candidate corner points and the corresponding reference corner points. The distance between the candidate corner points and the corresponding reference corner points can include at least one of the vertical distance, horizontal distance, and straight-line distance between the two.

[0119] Sub-step 1032A, if the distance between the candidate corner point and the reference corner point is not greater than a preset distance threshold, determining the candidate corner point as a target corner point;

[0120] In some application scenarios, if the distance between the two includes only one kind of distance, the preset distance threshold may be the preset distance threshold corresponding to the kind of distance, and in this case, the distance between the two is not greater than the preset distance threshold means that the kind of distance is not greater than the corresponding preset distance threshold. For example, if the distance between the two includes only the horizontal distance, the preset distance threshold is the preset horizontal distance threshold between the two. Then, if the horizontal distance between the two is not greater than the preset horizontal distance threshold, the candidate corner point may be determined as the target corner point.

[0121] In other application scenarios, if the distance between the two includes multiple distances, the preset distance threshold may be the preset distance thresholds corresponding to the multiple distances, respectively. In this case, the distance between the two is not greater than the preset distance threshold means that each distance is not greater than the corresponding preset distance threshold. For example, if the distance between the two includes a horizontal distance and a vertical distance, the preset distance threshold includes a preset horizontal distance threshold and a preset vertical distance threshold. Then, if the horizontal distance between the two is not greater than the preset horizontal distance threshold, and the vertical distance between the two is not greater than the preset vertical distance threshold, the candidate corner point may be determined as the target corner point.

[0122] Sub-step 1033A: if the distance between the candidate corner point and the reference corner point is greater than a preset distance threshold, the position of the candidate corner point is corrected to obtain the target corner point.

[0123] Similarly, if the distance between the two includes only one kind of distance, the preset distance threshold may be the preset distance threshold corresponding to the kind of distance, and in this case, the distance between the two being greater than the preset distance threshold means that the kind of distance is greater than the corresponding preset distance threshold. For example, if the distance between the two includes only the horizontal distance, the preset distance threshold is the preset horizontal distance threshold between the two. Then, if the horizontal distance between the two is greater than the preset horizontal distance threshold, the position of the candidate corner point may be corrected to obtain the target corner point.

[0124] If the distance between the two includes multiple distances, the preset distance threshold may be the preset distance thresholds corresponding to the multiple distances, respectively. In this case, the distance between the two is greater than the preset distance threshold means that each distance is greater than the corresponding preset distance threshold. For example, if the distance between the two includes a horizontal distance and a vertical distance, the preset distance threshold includes a preset horizontal distance threshold and a preset vertical distance threshold. Then, if the horizontal distance between the two is greater than the preset horizontal distance threshold, the horizontal position of the candidate corner point is corrected; if the vertical distance between the two is greater than the preset vertical distance threshold, the vertical position of the candidate corner point is corrected, thereby obtaining the target corner point.

[0125] In this implementation, since the reference corner points are obtained based on relatively accurate template corner points, the accuracy of the target corner points obtained after verification based on the distance between the candidate corner points and the reference corner points is relatively high.

[0126] In some optional implementations, if the distance between the candidate corner point and the reference corner point is greater than a preset distance threshold, the position of the candidate corner point is corrected to obtain the target corner point, including:

[0127] Method 1: If the server determines that the distance between the candidate corner point and the reference corner point is greater than a preset distance threshold, the server corrects the position of the candidate corner point according to the distances between other verified candidate corner points and their respective reference corner points and the position of the reference corner point corresponding to the candidate corner point to obtain the target corner point; wherein the verified candidate corner point refers to a candidate corner point whose distance to the corresponding reference corner point is not greater than the preset distance threshold;

[0128] In some application scenarios, if the distance between the candidate corner point and the reference corner point is greater than a preset distance threshold, the mean or median of the distances between other candidate corner points that have passed the verification and their respective reference corner points is determined, and the position of the candidate corner point is corrected according to the mean or median of the distances and the position of the candidate corner point to obtain the target corner point. For example, if the distance between candidate corner point a and its corresponding reference corner point is greater than the preset distance threshold among the four candidate corner points, and the distances between the other three candidate corner points and their respective corresponding reference corner points are not greater than the preset distance threshold, the mean of the distances between the other three candidate corner points that have passed the verification and their respective corresponding reference corner points can be determined, and then the position of the candidate corner point is corrected according to the mean of the distances and the position of the reference corner point corresponding to the candidate corner point to obtain the target corner point. In the correction, for example, with respect to the mean of the horizontal distance, the horizontal position of the candidate corner point can be offset by the mean of the horizontal distance to obtain the horizontal position of the target corner point.

[0129] In these application scenarios, since the accuracy of the candidate corner points that have passed the verification is relatively high, the candidate angle positions that have not passed the verification are corrected based on the positions of the candidate corner points that have passed the verification, so that relatively accurate target corner points can be obtained.

[0130] In a second method, the server may also replace the position of the candidate corner point with the position of the corresponding reference corner point to obtain the target corner point. In this way, since the reference corner point is obtained based on a relatively accurate template corner point, the accuracy of the target corner point obtained based on the reference corner point is also relatively high. In some application scenarios, for example, when the distance between multiple candidate corner points and the corresponding reference corner point is greater than a preset distance threshold, the positions of the multiple candidate corner points may be replaced with the positions of the corresponding reference corner points to improve the accuracy of the target corner point.

[0131] In this implementation, any one of the correction methods can be selected to correct the position of the candidate corner point, thereby obtaining a more accurate target corner point, which improves the robustness of the splicing to a certain extent.

[0132] In some application scenarios, for example, the selection of method 1 or method 2 can be determined based on the number of candidate corner points that fail the verification. Specifically, if the number of candidate corner points that fail the verification is small, for example, 1, method 1 can be selected to correct the position of the candidate corner points to improve the correction efficiency. If the number of candidate corner points that fail the verification is large, for example, more than 1, method 2 can be selected to correct the position of the candidate corner points to improve the accuracy of the target corner points.

[0133] In some application scenarios, the distance between two adjacent target corner points should be the same as the overall size of the battery cell preset between the two, so the candidate corner points can be verified based on this.

[0134] It should be noted that the above-mentioned overall dimensions may include, for example, at least one of the overall width, overall height, and overall diagonal length of the preset battery cells. The overall width may be the product of the preset number of battery cell columns and the horizontal width of a single battery cell, the overall height may be the product of the preset number of battery cell rows and the vertical height of a single battery cell, and the overall diagonal length may be calculated from the preset number of battery cell rows and columns and the diagonal length of a single battery cell.

[0135] In some application scenarios, if the position of the candidate corner point is Figure 3 As shown, the above-mentioned overall width can be the distance between candidate corner point E and candidate corner point F, or the distance between candidate corner point G and candidate corner point H; the above-mentioned overall height can be the distance between candidate corner point G and candidate corner point E, or the distance between candidate corner point H and candidate corner point F; the above-mentioned overall diagonal length can be the distance between candidate corner point E and candidate corner point H, or the distance between candidate corner point G and candidate corner point F.

[0136] Then, in some optional implementations, the step of verifying the candidate corner points in step 103 and determining the target corner points of the object to be spliced ​​according to the verification result includes:

[0137] Sub-step 1031B, if the distance between two adjacent candidate corner points matches the overall size of the battery cell preset between the two, then determining the candidate corner points as target corner points;

[0138] It should be noted that, since photovoltaic modules may have a variety of formats, they can be set according to the relevant size parameters of the corresponding formats to obtain the preset overall size of the battery cell.

[0139] Sub-step 1032B: if the distance between two adjacent candidate corner points does not match the overall size of the battery cell preset between the two adjacent candidate corner points, the positions of the two adjacent candidate corner points are corrected to obtain the target corner point.

[0140] In some application scenarios, if the distance between two adjacent candidate corner points includes only one distance, the preset overall size may be the overall size corresponding to the distance. In this case, the distance between the two does not match the preset overall size, which means that the distance does not match the corresponding preset overall size. For example, if the distance between the two only includes the horizontal distance, the preset overall size is the preset overall width between the two. Then, if the horizontal distance between the two is the same as the preset overall width or is within the preset error range, it can be determined that the horizontal distance between the two matches the preset overall width of the battery cell between the two. If the horizontal distance between the two is different from the preset overall width or is not within the preset error range, it can be determined that the horizontal distance between the two does not match the preset overall width of the battery cell between the two.

[0141] In other application scenarios, if the distance between the two includes multiple distances, the preset overall size can be the overall size corresponding to the multiple distances, and in this case, the distance between the two does not match the preset overall size means that each distance does not match the corresponding preset overall size. For example, if the distance between the two includes a horizontal distance and a vertical distance, the preset overall size includes a preset overall width and an overall height. Then, if the horizontal distance between two adjacent candidate corner points is the same as the overall width of the battery cell preset between the two or is within the preset error range, and the vertical distance between the two is the same as the overall height of the battery cell preset between the two or is within the preset error range, it can be determined that the distance between the two matches the overall size of the battery cell preset between the two.

[0142] Furthermore, if any distance or multiple distances are different from the corresponding overall size or are not within the preset error range, it can be determined that the distance between the two does not match the preset overall size of the battery cell between the two. Then, the positions of the two adjacent candidate corner points can be corrected to obtain the target corner point.

[0143] It should be noted that the mismatch may be caused by an error in one of the candidate corner points, or may be caused by errors in both candidate corner points. Therefore, the positions of the two adjacent candidate corner points are corrected. Specifically, a correction process is performed based on the positions of the two adjacent candidate corner points, wherein the positions of both candidate corner points may be corrected, or the position of one of the candidate corner points may be corrected.

[0144] In this implementation, since the overall size of the preset battery cell is known, the accuracy of the target corner point can be improved by verifying two adjacent candidate corner points based on the overall size of the preset battery cell.

[0145] Furthermore, in some optional implementations, the correcting the positions of the two adjacent candidate corner points to obtain the target corner point includes:

[0146] First, the distance between each candidate corner point and the corresponding reference corner point is determined, and two distances are obtained in total; the position of the reference corner point in the image to be processed is the same as the position of the template corner point marked in the template image corresponding to the image to be processed; the template image is a partial image of the template photovoltaic module captured by an area array camera or a movable camera;

[0147] The two distances obtained above are, that is, the distance between one of the two adjacent candidate corner points and its corresponding reference corner point, and the distance between the other candidate corner point and its corresponding reference corner point.

[0148] It should be noted that the distance between two adjacent candidate corner points should be the distance of the same dimension. That is, if the distance between one of the determined candidate corner points and its corresponding reference corner point is horizontal, the distance between the other candidate corner point and its corresponding reference corner point is also horizontal; if the distance between one of the determined candidate corner points and its corresponding reference corner point is vertical, the distance between the other candidate corner point and its corresponding reference corner point is also vertical.

[0149] It should be noted that if the distance between a candidate corner point and its corresponding reference corner point is small, it can be determined that the accuracy of the candidate corner point is high, while the accuracy of another candidate corner point is low. Therefore, the position of the other candidate corner point can be corrected.

[0150] Then, for the candidate corner point corresponding to the larger distance of the two distances, the position of the candidate corner point is corrected according to the position of the reference corner point corresponding to the candidate corner point and the smaller distance of the two distances.

[0151] For example, if the candidate corner point corresponds to a larger horizontal distance, a smaller horizontal distance may be offset from the horizontal position of the reference corner point of the candidate corner point to correct the horizontal position of the candidate corner point. If the candidate corner point corresponds to a larger vertical distance, a smaller vertical distance may be offset from the vertical position of the reference corner point of the candidate corner point to correct the vertical position of the candidate corner point.

[0152] In this implementation, the position of the candidate corner point can be corrected in combination with the distance between the candidate corner point and the reference corner point, thereby improving the accuracy of the target corner point and also improving the robustness of the stitching to a certain extent.

[0153] In some optional implementations, the cell array includes a plurality of main grid lines; thus, the step 103 described above of verifying the candidate corner points and determining the target corner points of the object to be spliced ​​according to the verification results includes:

[0154] Sub-step 1031C, performing edge detection on the image to be processed, and determining a plurality of busbars in the cell array according to the edge detection result;

[0155] In some application scenarios, for example, edge detection can be performed using a Sobel operator or a Canny operator to obtain edge detection results. Further, for example, based on a preset width range of the busbar, edge lines within the preset width range can be determined as busbars in the cell array.

[0156] Sub-step 1032C, determining a first boundary bus line and a second boundary bus line from a plurality of bus lines in the cell array; the first boundary bus line is a bus line closest to a candidate corner point close to the upper edge of the cell array, and the second boundary bus line is a bus line closest to a candidate corner point close to the lower edge of the cell array;

[0157] It should be noted that, based on the structural characteristics of the cell array in the photovoltaic module, the first boundary busbar should be located below the horizontal line where the candidate corner point of the upper edge of the cell array is located, and the second boundary busbar should be located above the horizontal line where the candidate corner point of the lower edge of the cell array is located. Figure 3 The main grid lines are indicated by numbers 6 and 7.

[0158] Sub-step 1033C, determining a first distance between the first boundary busbar and a candidate corner point of the upper edge of the cell array, and a second distance between the second boundary busbar and a candidate corner point of the lower edge of the cell array;

[0159] Sub-step 1034C, if the first distance and the second distance respectively match the preset spacing distances of the battery cell array, the vertical position of the candidate corner point is determined as the vertical position of the target corner point; if the first distance and / or the second distance do not match the preset spacing distances of the battery cell array, the vertical position of the unmatched candidate corner point is corrected to obtain the vertical position of the target corner point; the preset spacing distance includes a preset distance between the main grid line close to the edge of the battery cell and the edge of the battery cell.

[0160] In some application scenarios, the extension direction of the main grid line may be a horizontal direction, and the direction perpendicular to the extension direction may be a vertical direction.

[0161] In these application scenarios, the spacing distances between adjacent main grid lines are the same. Then, if the first distance is the same as the preset spacing distance or the error between the two is within the preset error range, it can be determined that the two match, otherwise the two do not match; if the second distance is the same as the preset spacing distance or the error between the two is within the preset error range, it can also be determined that the two match, otherwise the two do not match.

[0162] Furthermore, if there is an unmatched candidate corner point, the vertical position of the candidate corner point can be corrected. In some application scenarios, if the first distance does not match the preset spacing distance of the cell array, the vertical position of the first boundary main grid line is vertically offset by the preset spacing distance to obtain the vertical position of the target corner point; if the second distance does not match the preset spacing distance of the cell array, the vertical position of the second boundary main grid line is vertically offset by the preset spacing distance to obtain the vertical position of the target corner point.

[0163] In this implementation, since the preset spacing distance in the battery cell array is known, the vertical position of the candidate corner point can be verified based on this, which improves the accuracy of the target corner point to a certain extent.

[0164] In some optional implementations, in addition to determining the candidate corner points, at least one non-candidate corner point is also determined, so as to obtain a total of multiple battery cell corner points; in some application scenarios, the server can, for example, determine all battery cell corner points in the image to be processed, so as to verify the candidate corner points through all battery cell corner points.

[0165] It should be noted that the server may, for example, determine at least one non-candidate corner point when executing step 102. In addition, the server may, for example, determine at least one non-candidate corner point separately after determining the candidate corner point, which is not limited in the present application.

[0166] Thus, the step 103 of verifying the candidate corner points and determining the target corner points of the object to be spliced ​​according to the verification results includes: verifying the multiple cell corner points and determining the target corner points of the object to be spliced ​​according to the verification results. Thus, if the positions of the multiple cell corner points are correct, the positions of the candidate corner points are also correct, and then the candidate corner points can be determined as the target corner points.

[0167] In this implementation, the accuracy of the target corner point can be improved by combining multiple cell corner points to verify the candidate corner point.

[0168] In some application scenarios, since there are two cell corner points on the same edge side of a single cell, multiple cell corner points can be checked based on the preset size of the single cell, where the size of the single cell may include, for example, horizontal width, vertical height, and / or diagonal length.

[0169] Thus, the above-mentioned verification of multiple cell corner points and determination of target corner points of the object to be spliced ​​according to the verification results include:

[0170] First, for two adjacent cell corner points, if the distance between the two matches the preset size of a single cell, the two adjacent cell corner points are stored in a first set; if the distance between the two does not match the preset size of a single cell, the two adjacent cell corner points are stored in a second set;

[0171] It should be noted that the distance between two adjacent cell corners may also include at least one of the vertical distance, horizontal distance, and straight-line distance between the two. If the horizontal distance, vertical distance, and straight-line distance between the two are respectively the same as the horizontal width, vertical height, and diagonal length of a preset single cell or are within a preset error range, it can be determined that the two match. Therefore, the positions of the cell corners stored in the first set are accurate.

[0172] If any of the horizontal distance, vertical distance, and linear distance between the two is different from the corresponding preset battery cell size or is not within the preset error range, it can be determined that the two do not match. The reason for the mismatch between the two may be that both have errors, or one of them has an error. Therefore, the position of the battery cell corner point stored in the second set may have errors or may be accurate.

[0173] Then, for any cell corner point, if the cell corner point exists in the first set, the cell corner point is determined as a corner point that has passed the verification; if the cell corner point exists in the second set but not in the first set, the cell corner point is determined as a corner point that has not passed the verification;

[0174] If there is a corner point that fails to pass the verification among the candidate corner points, the position of the candidate corner point is corrected using the position of at least one corner point of the battery cell that passes the verification to obtain the target corner point.

[0175] In some application scenarios, the cell corner points that are within a preset distance range from the candidate corner points that have not passed the verification can be determined from multiple cell corner points that have passed the verification, so that correction can be made using the adjacent cell corner points. For example, if the preset distance range is the size of a single cell, the position of the cell corner point that has passed the verification can be offset by the size of a single cell to obtain the target corner point; if the preset distance range is the size of multiple cells, the position of the cell corner point that has passed the verification can be offset by the size of multiple cells to obtain the target corner point.

[0176] For ease of understanding, the horizontal distance is used as an example for explanation. That is, if the horizontal distance between two adjacent cell corner points matches the preset horizontal width of a single cell, the two adjacent cell corner points are stored in the first set; if the horizontal distance between the two does not match the horizontal width of a single cell, the two adjacent cell corner points are stored in the second set. In this way, the cell corner points in the second set may be correct or incorrect. Among them, if a cell corner point exists in the first set, it can be determined that the horizontal position of the cell corner point is correct; if a cell corner point only exists in the second set, it can be determined that there is an error in the horizontal position of the cell corner point.

[0177] Furthermore, if any of the four candidate corner points exists in the second set, the candidate corner point fails to pass the verification, and then the horizontal position of the corresponding target corner point can be obtained by offsetting the horizontal width of a single cell through the position of the adjacent cell corner point that passed the verification. If the adjacent cell corner points all fail to pass the verification, the corresponding target corner point can be obtained by offsetting the horizontal position of a cell corner point that passed the verification a little further away, by the sum of the horizontal widths of the corresponding number of cells.

[0178] In this implementation, since the size of a single battery cell is known, the position of the candidate corner point can be verified and corrected based on the size of the single battery cell, which can improve the accuracy of the target corner point and the robustness of the stitching to a certain extent.

[0179] In other application scenarios, if the position of the cell corner point is accurate, then multiple cell corner points should be relatively concentrated. Then, verification can be performed based on the position clustering result between multiple cell corner points.

[0180] Thus, the above-mentioned checking of the plurality of cell corner points and determining the target corner point of the object to be spliced ​​according to the checking result includes:

[0181] First, based on the positions of the plurality of cell corner points, a density-based clustering operation is performed to obtain a clustering result; in some application scenarios, for example, a DBSCAN algorithm (density-based spatial clustering algorithm) operation may be performed to obtain a clustering result, thereby removing outlier corner points from the clustering result to obtain non-outlier corner points;

[0182] Then, a fitting line can be obtained according to the position of the non-outlier corner point; here, for example, the position of the non-outlier corner point can be fitted by the least square method to obtain a fitting line. For example, the expression of the fitting line can be preset as , the position parameters of multiple non-outlier corner points can be brought into the expression, and the least squares method can be used to obtain the better expression coefficients. , , so the specific expression of the fitting line is .in, , It can be any determined value.

[0183] Then, for any candidate corner point, if the distance between the candidate corner point and the fitting straight line is within a preset distance range, the candidate corner point is determined as the target corner point; if the distance between the candidate corner point and the fitting straight line is not within the preset distance range, the position of the candidate corner point is corrected according to the fitting straight line to obtain the target corner point.

[0184] That is to say, by determining the distance between the candidate corner point and the fitting line, it can be determined whether the candidate corner point is outlier. If the distance between the candidate corner point and the fitting line is within the preset distance range, it can be determined that the candidate corner point is not outlier, and the candidate corner point is likely to be accurate, and then the candidate angle can be determined as the target corner point; if it is not within the preset distance range, it can be determined that the candidate corner point is outlier, and the candidate corner point is likely to be wrong. At this time, the position of the candidate corner point can be corrected by the fitting line.

[0185] In some application scenarios, for example, the horizontal position parameter of the candidate corner point can be substituted into the expression of the fitted line to obtain the corresponding vertical position. Alternatively, the vertical position parameter of the candidate corner point can be substituted into the expression of the fitted line to obtain the corresponding horizontal position.

[0186] In this implementation, since the accuracy of the positions of non-outlier corner points is relatively high, the accuracy of the fitting straight line obtained based on the positions of the non-outlier corner points is also relatively high. Subsequently, the candidate corner points are verified and corrected by the fitting straight line to obtain relatively accurate target corner points.

[0187] Those skilled in the art will appreciate that, in the above method of a specific embodiment, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0188] Please refer to Figure 6 , which shows a structural block diagram of an image stitching device provided in an embodiment of the present application. The image stitching device may be a module, a program segment or a code on an electronic device. It should be understood that the device is similar to the above Figure 1 The method embodiment corresponds to and can be executed Figure 1 The method embodiment involves various steps.

[0189] Optionally, the image stitching device comprises an acquisition module 601, a determination module 602, a verification module 603, a transformation module 604 and a stitching module 605. The acquisition module 601 is used to acquire at least two images to be processed; the determination module 602 is used to determine, for each image to be processed, candidate corner points of an object to be stitched in the image to be processed; the verification module 603 is used to verify the candidate corner points and determine the target corner points of the object to be stitched according to the verification result; the transformation module 604 is used to perform perspective transformation on the image area surrounded by the target corner points in the image to be processed to obtain the image to be stitched; the stitching module 605 is used to stitch at least two images to be stitched to obtain the target image.

[0190] Optionally, the image to be processed is a local image of a photovoltaic module captured by an area array camera or a movable camera, and the object to be spliced ​​is a cell array in the photovoltaic module.

[0191] Optionally, the verification module 603 is further used to: for each candidate corner point, determine the distance between the candidate corner point and the corresponding reference corner point; the position of the reference corner point in the image to be processed is the same as the position of the template corner point marked in the template image corresponding to the image to be processed; the template image is a local image of the template photovoltaic component captured by an array camera or a movable camera; if the distance between the candidate corner point and the reference corner point is not greater than a preset distance threshold, determine the candidate corner point as a target corner point; if the distance between the candidate corner point and the reference corner point is greater than the preset distance threshold, correct the position of the candidate corner point to obtain the target corner point.

[0192] Optionally, the verification module 603 is further used to: if the distance between the candidate corner point and the reference corner point is greater than a preset distance threshold, then the position of the candidate corner point is corrected according to the distances between other candidate corner points that have passed the verification and their respective reference corner points and the position of the reference corner point corresponding to the candidate corner point to obtain the target corner point; wherein the candidate corner point that has passed the verification refers to a candidate corner point whose distance from the corresponding reference corner point is not greater than the preset distance threshold; or, the position of the candidate corner point is replaced with the position of the corresponding reference corner point to obtain the target corner point.

[0193] Optionally, the verification module 603 is further used to: if the distance between two adjacent candidate corner points matches the overall size of the battery cell preset between the two, then the candidate corner point is determined as the target corner point; if the distance between two adjacent candidate corner points does not match the overall size of the battery cell preset between the two, then the positions of the two adjacent candidate corner points are corrected to obtain the target corner point.

[0194] Optionally, the verification module 603 is further used to: determine the distance between each candidate corner point and the corresponding reference corner point, and obtain two distances in total; the position of the reference corner point in the image to be processed is the same as the position of the template corner point marked in the template image corresponding to the image to be processed; the template image is a local image of the template photovoltaic component captured by an area array camera or a movable camera; for the candidate corner point corresponding to the larger of the two distances, the position of the candidate corner point is corrected according to the position of the reference corner point corresponding to the candidate corner point and the smaller of the two distances.

[0195] Optionally, the cell array includes a plurality of main grid lines; and the verification module 603 is further used to: perform edge detection on the image to be processed, and determine a plurality of main grid lines in the cell array according to the edge detection result; determine a first boundary main grid line and a second boundary main grid line from the plurality of main grid lines in the cell array; the first boundary main grid line is the main grid line closest to the candidate corner point close to the upper edge of the cell array, and the second boundary main grid line is the main grid line closest to the candidate corner point close to the lower edge of the cell array; determine the distance between the first boundary main grid line and the candidate corner point close to the upper edge of the cell array; A first distance between selected corner points, and a second distance between the second boundary main grid line and the candidate corner point at the lower edge of the cell array; if the first distance and the second distance respectively match the preset spacing distances of the cell array, the vertical position of the candidate corner point is determined as the vertical position of the target corner point; if the first distance and / or the second distance do not match the preset spacing distance of the cell array, the vertical position of the unmatched candidate corner point is corrected to obtain the vertical position of the target corner point; the preset spacing distance includes a preset distance between the main grid line close to the edge of the cell and the edge of the cell.

[0196] Optionally, in addition to determining the candidate corner points, at least one non-candidate corner point is also determined, so as to obtain a total of multiple battery cell corner points; and the verification module 603 is further used to: verify the multiple battery cell corner points, and determine the target corner point of the object to be spliced ​​based on the verification result.

[0197] Optionally, the verification module 603 is further used for: for two adjacent battery cell corner points, if the distance between the two matches the preset size of a single battery cell, the two adjacent battery cell corner points are stored in a first set; if the distance between the two does not match the preset size of a single battery cell, the two adjacent battery cell corner points are stored in a second set; for any battery cell corner point, if the battery cell corner point exists in the first set, the battery cell corner point is determined as a corner point that has passed the verification; if the battery cell corner point exists in the second set and does not exist in the first set, the battery cell corner point is determined as a corner point that has not passed the verification; if there are corner points that have not passed the verification among the candidate corner points, the position of at least one battery cell corner point that has passed the verification is used to correct the position of the candidate corner point to obtain the target corner point.

[0198] Optionally, the verification module 603 is further used to: perform a density-based clustering operation based on the positions of the multiple battery cell corner points to obtain a clustering result; eliminate outlier corner points from the clustering result to obtain non-outlier corner points; obtain a fitting straight line based on the positions of the non-outlier corner points; for any candidate corner point, if the distance between the candidate corner point and the fitting straight line is within a preset distance range, determine the candidate corner point as a target corner point; if the distance between the candidate corner point and the fitting straight line is not within the preset distance range, correct the position of the candidate corner point according to the fitting straight line to obtain the target corner point.

[0199] Optionally, the determination module 602 is further configured to: determine a rough position of a candidate corner point in each image to be processed; and determine the candidate corner point according to the rough position of the candidate corner point.

[0200] Optionally, the determination module 602 is further used to: for each image to be processed, obtain the positions and types corresponding to multiple battery cell corner points through the prediction model; arrange the multiple battery cell corner points in sequence according to the positions corresponding to each battery cell corner point, and obtain the order information of each battery cell corner point; determine the rough position of the candidate corner point according to the order information of each battery cell corner point, and the order information and type information of the preset candidate corner points.

[0201] Optionally, the determination module 602 is further used to: determine a grayscale image of each image to be processed; determine a first gradient distribution sequence of pixel values ​​in the horizontal direction and a second gradient distribution sequence in the vertical direction in the grayscale image; determine an extreme value representing a battery cell gap in the first gradient distribution sequence and an extreme value representing a battery cell gap in the second gradient distribution sequence; arrange multiple extreme values ​​representing battery cell gaps in sequence according to the positions of the extreme values ​​representing battery cell gaps in the first gradient distribution sequence to obtain order information of each extreme value representing a battery cell gap; and determine an extreme value representing a battery cell gap according to the positions of the extreme values ​​representing battery cell gaps in the first gradient distribution sequence. According to the order information of the extreme values ​​of the gaps and the horizontal order information of the preset candidate corner points, the extreme value corresponding to the candidate corner point in the horizontal direction is determined; according to the positions of the extreme values ​​representing the battery cell gaps in the second gradient distribution sequence, multiple extreme values ​​representing the battery cell gaps are arranged in sequence to obtain the order information of each extreme value representing the battery cell gap; according to the order information of each extreme value representing the battery cell gap and the vertical order information of the preset candidate corner points, the extreme value corresponding to the candidate corner point in the vertical direction is determined; according to the position of the extreme value corresponding to the candidate corner point in the horizontal direction and the position of the extreme value corresponding to the vertical direction, the rough position is determined.

[0202] Optionally, the determination module 602 is further used to: determine the reference corner points corresponding to each candidate corner point in the image to be processed; the positions of the reference corner points in the image to be processed are the same as the positions of the template corner points marked in the template image corresponding to the image to be processed; the template image is a partial image of the template photovoltaic component captured by an area array camera or a movable camera; and the positions of the reference corner points are determined as the rough positions of the candidate corner points.

[0203] Optionally, the determination module 602 is further used to: extract edge features of an image area within a preset range around the rough position of the candidate corner point; filter the main grid line features on the cell in the photovoltaic module from the extracted edge features; determine the horizontal edge straight line and the vertical edge straight line of the cell based on the filtered edge features; and determine the intersection of the horizontal edge straight line and the vertical edge straight line as the candidate corner point.

[0204] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system or device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0205] Please refer to Figure 7 , Figure 7A structural diagram of an electronic device for executing an image stitching method provided in an embodiment of the present application, the electronic device may include: at least one processor 701, such as a CPU, at least one communication interface 702, at least one memory 703 and at least one communication bus 704. Among them, the communication bus 704 is used to realize direct connection and communication between these components. Among them, the communication interface 702 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 703 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 703 can optionally be at least one storage device located away from the aforementioned processor. Computer-readable instructions are stored in the memory 703. When the computer-readable instructions are executed by the processor 701, the electronic device can execute the above-mentioned Figure 1 The process of any implementation of the method shown.

[0206] Understandably, Figure 7 The structure shown is for illustration only, and the electronic device may also include Figure 7 More or fewer components as shown, or with Figure 7 Different configurations are shown. Figure 7 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0207] The present application embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the following can be performed: Figure 1 Any implementation method in the method embodiment shown.

[0208] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments. For example, the method may include: acquiring at least two images to be processed; for each image to be processed, determining candidate corner points of an object to be spliced ​​in the image to be processed; and, verifying the candidate corner points, and determining target corner points of the object to be spliced ​​according to the verification results; and, performing perspective transformation on an image area surrounded by the target corner points in the image to be processed to obtain an image to be spliced; and splicing at least two of the images to be spliced ​​to obtain a target image.

[0209] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0210] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0212] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0213] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image stitching method, characterized in that: include: Acquire at least two images to be processed; For each image to be processed, determining candidate corner points of the object to be spliced ​​in the image to be processed; as well as Verifying the candidate corner points, and determining the target corner points of the object to be spliced ​​according to the verification results; as well as Performing perspective transformation on the image area surrounded by the target corner points in the image to be processed to obtain the image to be stitched; Stitching at least two of the images to be stitched together to obtain a target image; Wherein, the image to be processed is a local image of a photovoltaic module captured by an area array camera or a movable camera, and the object to be spliced ​​is a cell array in the photovoltaic module; In addition to determining the candidate corner points, at least one non-candidate corner point is also determined, so as to obtain a total of multiple cell corner points; and The verifying the candidate corner points and determining the target corner points of the object to be spliced ​​according to the verification result includes: Verifying the plurality of cell corner points, and determining the target corner point of the object to be spliced ​​according to the verification result; The step of verifying the plurality of cell corner points and determining the target corner point of the object to be spliced ​​according to the verification result includes: Based on the positions of the multiple battery cell corner points, a density-based clustering operation is performed to obtain a clustering result; From the clustering results, outlier corner points are removed to obtain non-outlier corner points; Obtaining a fitting straight line according to the positions of the non-outlier corner points; For any candidate corner point, if the distance between the candidate corner point and the fitting straight line is within a preset distance range, the candidate corner point is determined as the target corner point; if the distance between the candidate corner point and the fitting straight line is not within the preset distance range, the position of the candidate corner point is corrected according to the fitting straight line to obtain the target corner point.

2. The method according to claim 1, characterized in that The verifying the candidate corner points and determining the target corner points of the object to be spliced ​​according to the verification result includes: For each candidate corner point, determine the distance between the candidate corner point and the corresponding reference corner point; the position of the reference corner point in the image to be processed is the same as the position of the template corner point marked in the template image corresponding to the image to be processed; the template image is a partial image of the template photovoltaic module captured by an area array camera or a movable camera; If the distance between the candidate corner point and the reference corner point is not greater than a preset distance threshold, the candidate corner point is determined as a target corner point; If the distance between the candidate corner point and the reference corner point is greater than a preset distance threshold, the position of the candidate corner point is corrected according to the distances between other verified candidate corner points and their respective reference corner points and the position of the reference corner point corresponding to the candidate corner point to obtain the target corner point; wherein the verified candidate corner point refers to a candidate corner point whose distance from the corresponding reference corner point is not greater than the preset distance threshold; alternatively, the position of the candidate corner point is replaced with the position of the corresponding reference corner point to obtain the target corner point.

3. The method according to claim 1, characterized in that The verifying the candidate corner points and determining the target corner points of the object to be spliced ​​according to the verification result includes: If the distance between two adjacent candidate corner points matches the overall size of the battery cell preset between the two, the candidate corner points are determined as target corner points; If the distance between two adjacent candidate corner points does not match the overall size of the battery cell preset between the two adjacent candidate corner points, the positions of the two adjacent candidate corner points are corrected to obtain the target corner point.

4. The method according to claim 1, characterized in that: The cell array includes a plurality of main grid lines; as well as The verifying the candidate corner points and determining the target corner points of the object to be spliced ​​according to the verification result includes: Performing edge detection on the image to be processed, and determining a plurality of busbars in the cell array according to the edge detection result; Determine a first boundary bus line and a second boundary bus line from a plurality of bus lines in the cell array; the first boundary bus line is a bus line closest to a candidate corner point close to an upper edge of the cell array, and the second boundary bus line is a bus line closest to a candidate corner point close to a lower edge of the cell array; Determine a first distance between the first boundary busbar and a candidate corner point of the upper edge of the cell array, and a second distance between the second boundary busbar and a candidate corner point of the lower edge of the cell array; If the first distance and the second distance respectively match the preset spacing distance of the battery cell array, then determining the vertical position of the candidate corner point as the vertical position of the target corner point; If the first distance and / or the second distance do not match the preset spacing distance of the battery cell array, the vertical position of the unmatched candidate corner point is corrected to obtain the vertical position of the target corner point; the preset spacing distance includes the preset distance between the main grid line close to the edge of the battery cell and the edge of the battery cell.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining, for each image to be processed, candidate corner points of the object to be spliced ​​in the image to be processed includes: For each image to be processed, a rough position of a candidate corner point is determined in the image to be processed; The candidate corner points are determined according to the rough positions of the candidate corner points.

6. The method according to claim 5, characterized in that The step of determining the rough position of the candidate corner points in each image to be processed includes: For each image to be processed, the positions and types corresponding to multiple corner points of the battery cell are obtained through the prediction model; Arrange multiple cell corner points in sequence according to the positions corresponding to the corner points of each cell, and obtain the order information of each cell corner point; According to the order information of each cell corner point, and the order information and type information of the preset candidate corner points, the rough position of the candidate corner points is determined.

7. The method according to claim 5, characterized in that The step of determining the rough position of the candidate corner points in each image to be processed includes: For each image to be processed, determining a grayscale image of the image to be processed; Determine a first gradient distribution sequence in the horizontal direction and a second gradient distribution sequence in the vertical direction of pixel values ​​in the grayscale image; Determine an extreme value representing a gap between battery cells in the first gradient distribution sequence and an extreme value representing a gap between battery cells in the second gradient distribution sequence; Arrange multiple extreme values ​​representing the gap of the battery cell in sequence according to the positions of the extreme values ​​representing the gap of the battery cell in the first gradient distribution sequence to obtain the order information of each extreme value representing the gap of the battery cell; determine the extreme value corresponding to the candidate corner point in the horizontal direction according to the order information of each extreme value representing the gap of the battery cell and the preset horizontal order information of the candidate corner point; Arrange multiple extreme values ​​representing the gap of the battery cell in sequence according to the positions of the extreme values ​​representing the gap of the battery cell in the second gradient distribution sequence to obtain the order information of each extreme value representing the gap of the battery cell; determine the extreme value corresponding to the candidate corner point in the vertical direction according to the order information of each extreme value representing the gap of the battery cell and the vertical order information of the preset candidate corner point; The rough position is determined according to the position of the extreme value corresponding to the candidate corner point in the horizontal direction and the position of the extreme value corresponding to the candidate corner point in the vertical direction.

8. The method according to any one of claims 6-7, characterized in that: The step of determining the candidate corner point according to the rough position of the candidate corner point comprises: Extract edge features of an image area within a preset range around the rough position of the candidate corner point; Filtering the main grid line features on the cell in the photovoltaic module from the extracted edge features; Based on the filtered edge features, determining a horizontal edge straight line and a vertical edge straight line of the battery cell; The intersection point of the horizontal edge straight line and the vertical edge straight line is determined as a candidate corner point.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is executed.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 8 is performed.

Images