Water body image splicing method and device based on cooperation of unmanned aerial vehicle and unmanned ship

Abstract

The invention provides a water body image splicing method and device based on cooperation of an unmanned aerial vehicle and an unmanned ship, and belongs to the technical field of image processing, and the method comprises the steps: determining a first driving path of the unmanned ship based on a shooting region and a preset image splicing proportion, determining a second driving path and a plurality of shooting center points of the unmanned aerial vehicle based on the first driving path; calculating an image offset based on a plurality of water area images collected by the unmanned aerial vehicle at the plurality of shooting center points; the image offset is the offset between the position of the unmanned ship in the image and a plurality of shooting center points; and splicing the plurality of water area images based on the image offset. According to the water body image splicing method and device based on cooperation of the unmanned aerial vehicle and the unmanned ship provided by the invention, the problems that the images deviate and cannot be spliced when a large-area water area is monitored in the prior art can be solved.

Description

Technical Field

[0001] The present disclosure belongs to the field of image processing technology, and more specifically, to a method and device for stitching water body images based on the collaboration of an unmanned aerial vehicle and an unmanned boat. Background Art

[0002] In recent years, with the rapid development of my country's economy and society, more and more attention has been paid to the management and protection of water resources. Remote sensing monitoring of vast waters has become an important means of obtaining information on hydrology, water quality, ecological environment, etc. UAVs are characterized by flexibility, high efficiency, precision, accuracy, and low operating costs. They have obvious advantages in quickly acquiring high-resolution images in small areas and areas with difficult flights. Combining the two can give full play to their respective advantages, achieve the complementarity of aerial and surface remote sensing data, and improve the coverage and accuracy of remote sensing monitoring.

[0003] However, when conducting water environment monitoring in vast waters such as large lakes, reservoirs and bays, the UAV may be unable to monitor the center of the water area due to the short flight distance, and cannot detect image offset and stitch images due to the large water surface and lack of reference objects. Summary of the invention

[0004] The purpose of the present invention is to provide a method and device for stitching water images based on the collaboration of an unmanned aerial vehicle and an unmanned boat, so as to solve the problem of image offset and inability to stitch images when monitoring a large area of ​​water in the prior art.

[0005] A first aspect of the embodiments of the present disclosure provides a method for stitching water body images based on collaboration between a drone and an unmanned ship, comprising: Determine a first driving path of the unmanned boat based on the shooting area and a preset image stitching ratio, and determine a second driving path and a plurality of shooting center points of the unmanned aerial vehicle based on the first driving path; The image offset is calculated based on multiple water area images collected by the UAV at multiple shooting center points; the image offset is the offset between the position of the unmanned ship in the image and the multiple shooting center points; Multiple water area images are stitched together based on image offset.

[0006] A second aspect of the disclosed embodiment provides a water image stitching device based on the collaboration of a drone and an unmanned boat, comprising: A path calculation module, used to determine a first driving path of the unmanned ship based on the shooting area and a preset image stitching ratio, and to determine a second driving path and a plurality of shooting center points of the unmanned aerial vehicle based on the first driving path; An offset calculation module is used to calculate an image offset based on multiple water area images collected by the UAV at multiple shooting center points; the image offset is an offset between the position of the unmanned ship in the image and the multiple shooting center points; The image stitching module is used to stitch multiple water area images based on image offset.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the above-mentioned method for stitching water images based on the collaboration of an unmanned aerial vehicle and an unmanned boat are implemented.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned method for stitching water images based on the collaboration of a drone and an unmanned ship are implemented.

[0009] The beneficial effects of the water body image stitching method and device based on the cooperation of a drone and an unmanned boat provided by the embodiments of the present disclosure are: On the one hand, in monitoring vast waters, the problem of being unable to monitor the center of the waters due to the short flight distance of the drone can be effectively solved by rationally planning the first driving path of the unmanned boat and the second driving path and shooting center point of the drone in order to effectively solve the problem of being unable to monitor the center of the waters due to the short flight distance of the drone. The unmanned boat can carry the drone deep into the center of the waters to ensure that the entire waters are covered, and image information can be obtained in both the nearshore and central areas to achieve comprehensive monitoring.

[0010] On the other hand, the problem of difficulty in detecting image offset and inability to stitch images due to the lack of reference objects on the large water surface has also been solved. The unmanned boat introduced in this disclosure provides a reference for the drone, and the image correction can be accurately performed by calculating the offset between the position of the unmanned boat and the shooting center point in the image. Image stitching based on this offset can ensure accurate alignment between images, thereby generating a complete and continuous image of the water area and improving image quality.

[0011] In summary, the UAV and unmanned boat collaborative method provided by the present disclosure combines the advantages of unmanned boats and UAVs, while increasing the monitoring range and ensuring the quality of monitoring data. Whether it is the analysis of lake water quality, the study of aquatic biological distribution, or the assessment of the bay ecological environment, it can provide high-quality and highly reliable image data, providing strong technical support for water environment management and protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0013] Figure 1 A schematic diagram of a flow chart of a water image stitching method based on collaboration between a drone and an unmanned boat provided in one embodiment of the present disclosure; Figure 2 A schematic diagram of coordinate axes of a water body image based on cooperation between a drone and an unmanned boat provided in one embodiment of the present disclosure; Figure 3 A schematic diagram of an offset angle of a water body image based on cooperation between a drone and an unmanned boat provided in one embodiment of the present disclosure; Figure 4 A schematic diagram of a flow chart of another method for stitching water body images based on collaboration between a drone and an unmanned boat provided in an embodiment of the present disclosure; Figure 5 A structural block diagram of a water image stitching device based on cooperation between a drone and an unmanned boat provided in one embodiment of the present disclosure; Figure 6 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0014] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0015] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0016] The water area image in this disclosure is a water body image.

[0017] Please refer to Figure 1 , Figure 1 A schematic flow chart of a method for stitching water body images based on collaboration between a drone and an unmanned boat is provided in accordance with an embodiment of the present disclosure. The method may include S101 to S103.

[0018] S101: Determine a first driving path of the unmanned boat based on a shooting area and a preset image stitching ratio, and determine a second driving path and a plurality of shooting center points of the unmanned aerial vehicle based on the first driving path.

[0019] In this embodiment, the unmanned boat carries the drone and moves synchronously. Before determining the first driving path of the unmanned boat based on the shooting area and the preset image stitching ratio, a water image stitching method based on the cooperation of the drone and the unmanned boat also includes: constructing a water map, and determining the shooting area based on the water map.

[0020] Exemplarily, a water map is first constructed, and the water range (i.e., shooting area) to be photographed is clarified through the map. Based on the size of the shooting area and the preset image stitching ratio, the first driving path of the unmanned boat is determined. Since the unmanned boat is carrying the drone and moving synchronously, the second driving path of the drone and multiple shooting center points are further determined based on the first driving path of the unmanned boat. The drone's shooting must cover the entire shooting area and meet the requirements of image stitching, so its path and shooting points match the path of the unmanned boat.

[0021] In this embodiment, a water map can be constructed based on satellite remote sensing data and geographic information data of the target water area. The target water area is the entire water area to be photographed. When determining the preset image stitching ratio, it is necessary to ensure that the preset image stitching ratio is greater than the maximum offset, wherein the maximum offset is obtained by multiple water area image acquisition and stitching experiments and empirical calculations. The path of the drone can be generated using a path planning algorithm based on the unmanned ship path combined with its own flight performance and shooting parameters. The shooting center point can be set on the path of the drone according to a certain interval rule. When stitching images, there will be a certain degree of offset between images due to factors such as shooting angle and drone position offset caused by external environmental influences (such as strong winds), as well as lens distortion. If the preset stitching ratio is less than the maximum offset, then there will be image misalignment or incomplete overlap during the stitching process, resulting in a decrease in the quality of the stitched image. By ensuring that the stitching ratio is greater than the maximum offset, the relative position between images can be better adjusted during the stitching process, the stitching error can be reduced, and the stitching accuracy can be improved.

[0022] For example, when monitoring the water environment of large lakes, reservoirs, bays and other vast waters, the drone may fail to monitor the center of the water area due to the short flight distance, fail to detect image offset and fail to stitch images due to the large water surface and lack of reference objects. This embodiment, based on the collaboration between the drone and the unmanned boat, can correct the flight route, reduce the reserved stitching percentage of the image, adjust the offset and improve the stitching efficiency.

[0023] S102: Calculating an image offset based on a plurality of water area images collected by the UAV at a plurality of shooting center points; the image offset is an offset between a position of the unmanned boat in the image and the plurality of shooting center points.

[0024] In this embodiment, the image offset is calculated based on multiple water images collected by the drone at multiple shooting center points, including: determining the position coordinates of the unmanned ship based on the multiple water images collected by the drone at multiple shooting center points, calculating the coordinate difference between the position coordinates of the unmanned ship and the coordinates of the center point of the image, and calculating the image offset based on the coordinate difference. The position coordinates of the unmanned ship are the position information of the unmanned ship in the image. The coordinates of the center point of the image are the theoretical shooting center point, that is, the position coordinates of the unmanned ship in theory. Ideally, if there is no deviation, the position of the unmanned ship in the image should coincide with the center point of the image. After the drone collects images at multiple shooting center points, due to various factors (such as wind direction, shaking and tilting of the unmanned ship itself, etc.), the position of the unmanned ship in the image in the captured image deviates from the theoretical shooting center point. By analyzing the coordinate difference between the actual position of the unmanned ship and the shooting center point in each image, the image offset can be calculated.

[0025] Exemplarily, the position coordinates of the unmanned ship are determined based on an image recognition algorithm, the horizontal coordinate difference and the vertical coordinate difference between the position coordinates of the unmanned ship and the coordinates of the center point of the image are calculated, and the image offset is calculated based on the horizontal coordinate difference and the vertical coordinate difference. During the image acquisition process, for each water area image acquired at the shooting center point, the coordinates of the unmanned ship in the image are located using an image recognition algorithm. At the same time, the theoretical coordinates of the shooting center point are recorded. The offsets in the horizontal and vertical directions are then calculated by subtracting the coordinates. For example, the horizontal coordinate of the unmanned ship in the image is subtracted from the horizontal coordinate of the shooting center point to obtain the horizontal offset value.

[0026] Exemplarily, the image offset is calculated based on the offset direction and offset pixel value of the unmanned boat position coordinates relative to the image center point coordinates. The offset direction includes the positive or negative direction of the horizontal direction (i.e., the horizontal axis / x-axis) and the positive or negative direction of the vertical direction (i.e., the longitudinal axis / y-axis). When conducting an ecological survey of the river, the drone shoots at different shooting center points. In a certain captured water image, the position of the unmanned boat is offset by 5 pixels to the right and 3 pixels downward compared to the shooting center point. The image offset can be calculated based on this data. In subsequent data processing, the image offset can be used to correct the image to facilitate image stitching and avoid data errors or inability to stitching due to position offset.

[0027] S103: stitching the multiple water area images based on the image offset.

[0028] In this embodiment, the plurality of water area images are stitched together based on the image offset, including: determining the image overlap ratio and the offset angle based on the image offset.

[0029] Multiple water area images are stitched together based on image overlap ratio and offset angle.

[0030] In this embodiment, multiple water area images may be corrected based on the image offset and the correction model to obtain multiple corrected images, and the multiple corrected images may be spliced; the correction model may include a correction formula or an algorithm model.

[0031] Exemplarily, a correction model is established, the offset is substituted as a parameter, and the new position of each pixel of the image is calculated, so as to correct and stitch the image.

[0032] For example, an ecological monitoring of a large lake is carried out. UAVs collect water images at multiple locations. A correction matrix is ​​established for each water image. In the matrix corresponding to a certain water image, the image offset shows a deviation of 5 pixels in the horizontal direction and a deviation of 3 pixels in the vertical direction. The image offset can be used to determine that the image overlap ratio is 30% and the offset angle is 15 degrees. Using the correction formula method, the offset and angle are substituted into the formula, and the image is parameter corrected to obtain the corrected image, which is then spliced.

[0033] In this embodiment, the multiple water area images may be translated, rotated, and cropped based on the image offset to obtain multiple corrected images, and the multiple corrected images may be spliced.

[0034] For example, the image can be translated by 5 pixels (horizontally) and 3 pixels (vertically), rotated by 15 degrees, and the redundant part can be cut to obtain a corrected image according to the image offset. Finally, these corrected images are spliced ​​to form a complete lake ecological image, which clearly shows the distribution of aquatic plants in the lake and water quality changes, providing intuitive and accurate data for subsequent ecological analysis and formulation of protection measures.

[0035] In this embodiment, a water body image stitching method based on the cooperation of a drone and an unmanned boat further includes: determining a maximum image offset from image offsets corresponding to a plurality of water body images; Determine the target image stitching ratio based on the maximum image offset; the target image stitching ratio is greater than the maximum image offset; The preset image stitching ratio is updated based on the target image stitching ratio.

[0036] Exemplarily, the image offset reflects the relative deviation between each water image due to factors such as changes in the position of the drone. Correcting the image through this offset can make the image more accurately aligned when stitching. After calculating the multiple image offsets corresponding to the multiple water images, find the maximum image offset from the multiple image offsets, which represents the maximum value of the existing deviation. Determine the target image stitching ratio based on this maximum offset, and ensure that this ratio is greater than the maximum offset, so that the deviation of all images can be accommodated in subsequent stitching to ensure the integrity of the stitching. Finally, use the target image stitching ratio to update the preset image stitching ratio and use it for the next image acquisition and stitching.

[0037] For example, when counting the image offset, the offset data of all images are traversed to find the maximum value. For determining the target image stitching ratio, a certain margin (such as 10%-20%) can be added to the maximum image offset to obtain the target stitching ratio. In the image stitching software or algorithm, the preset stitching ratio is replaced with the new target stitching ratio.

[0038] For example, during the inspection of a large reservoir, the drone collects images at different shooting center points. The maximum image offset is 8 pixels in the horizontal direction and 6 pixels in the vertical direction. When determining the target image stitching ratio, it can be expanded on this basis, such as adding 2 pixels of margin in the horizontal and vertical directions.

[0039] From the above, it can be concluded that when monitoring the water environment in large lakes, reservoirs, bays and other vast waters, the drone may fail to monitor the center of the water area due to the short flight distance, fail to detect image offset and fail to stitch images due to the large water surface and lack of reference objects. The water image stitching method based on the collaboration of drones and unmanned boats provided in this embodiment can solve this problem.

[0040] On the one hand, in monitoring vast waters, the problem of being unable to monitor the center of the waters due to the short flight distance of the drone can be effectively solved by rationally planning the first driving path of the unmanned boat and the second driving path and shooting center point of the drone in order to effectively solve the problem of being unable to monitor the center of the waters due to the short flight distance of the drone. The unmanned boat can carry the drone deep into the center of the waters to ensure that the entire waters are covered, and image information can be obtained in both the nearshore and central areas to achieve comprehensive monitoring.

[0041] On the other hand, the problem of difficulty in detecting image offset and inability to stitch images due to the large water surface and lack of reference objects has also been well solved. The unmanned boat introduced in this embodiment provides a reference for the drone, and the image correction can be accurately performed by calculating the offset between the position of the unmanned boat in the image and the shooting center point. Image stitching based on this offset can ensure accurate alignment between images, thereby generating a complete and continuous image of the water area and improving image quality.

[0042] In summary, the drone and unmanned boat collaboration method provided in this embodiment combines the advantages of unmanned boats and drones, and ensures the quality of monitoring data while increasing the monitoring range. Whether it is the analysis of lake water quality, the study of aquatic biological distribution, or the assessment of the bay ecological environment, it can provide high-quality and highly reliable image data, providing strong technical support for water environment management and protection.

[0043] In one embodiment of the present disclosure, determining a first driving path of the unmanned ship based on a shooting area and a preset image stitching ratio includes: Determine the starting point and the ending point based on the shooting area; Determine multiple driving center points based on the starting point, the ending point and the preset image stitching ratio; A first driving path is determined based on a starting point, an ending point, and a plurality of driving center points.

[0044] In this embodiment, the shooting area is clearly defined, and the starting point and the ending point are determined in this shooting area. These two points define the range of the unmanned boat. Multiple driving center points are set in the water area between the starting point and the ending point. Considering that the final collected image will have deviations, a preset image stitching ratio is introduced to determine multiple driving center points. For example, the window range of the drone is preliminarily determined to be 5m*5m, and the center point of the window range is marked as the driving center point. After the preset image stitching ratio is introduced, if the preset image stitching ratio is 25%, it means that there can be a 25% overlap between two adjacent captured images. In actual operation, based on the initially determined driving center point, in order to ensure a 25% stitching ratio, the distance between each new driving center point is no longer 5m, but becomes 5m×(1-25%)=3.75m. In this way, when the unmanned boat is driving along the path, when the drone takes pictures at these newly determined driving center points, there will be a 25% overlap between adjacent images.

[0045] The purpose of this preset image stitching ratio is that when the images are subsequently stitched, the 25% overlap area can provide more image information for image correction. For example, during the image stitching process, if the image is found to be offset, the feature points in the overlapping area can be used for precise matching and adjustment, so that the stitched image is more coherent and natural, without obvious stitching marks or misalignment. Moreover, in the water monitoring environment, since the water surface may have complex conditions such as reflections and fluctuations, sufficient overlap areas can effectively reduce the impact of these factors on the image stitching quality, thereby improving the accuracy and reliability of the entire water monitoring image.

[0046] Finally, the starting point, the end point and multiple driving center points are connected to form the first driving path of the unmanned boat to ensure full coverage of the shooting area.

[0047] This embodiment ensures efficient monitoring of designated waters by setting the driving path of the unmanned boat. The shooting area is clearly defined and the starting and ending points are determined. These two points define the range of the unmanned boat. Based on the starting point, the ending point and the preset image stitching ratio, multiple driving center points are determined to ensure the overlapping area during image acquisition, thereby providing more image information for correction during image stitching, ensuring that the stitched image is coherent and natural without obvious stitching marks or misalignment.

[0048] In one embodiment of the present disclosure, multiple driving center points are determined based on the starting point, the ending point and the preset image stitching ratio, including: Determine the window range based on the flight altitude of the drone; The shooting area between the starting point and the ending point is divided into multiple sub-areas based on the window range and the preset image stitching ratio; the center of each sub-area is marked as the driving center point.

[0049] In this embodiment, each sub-area has the same shape as the window range, for example, the window range can be set to be a square, a rectangle, a diamond, etc.

[0050] In this embodiment, there is a direct correlation between the flight altitude of the drone and its window range. Generally, the higher the flight altitude, the larger the field of view. Based on the field of view and the pre-set image stitching ratio, the shooting area between the starting point and the end point can be divided. The pre-set image stitching ratio determines the degree of overlap between adjacent sub-areas. This method ensures that the collected images have sufficient overlapping information during the image stitching process. The center point of each sub-area is used as the center point of travel, so that when the unmanned ship is at these points, the drone can obtain images that meet the stitching requirements.

[0051] For example, the height of the drone can be set according to the drone's window range, the maximum spacing of the shooting range can be fixed according to the drone's height, and the window range can be fixed as a square area according to the maximum spacing. According to the square area, marking points equal to the side length of the square are delineated on the route to form equidistant square sub-areas, and the center of the square area is set as the driving center point.

[0052] Exemplarily, after determining the flight altitude of the drone, the size of the window range is obtained based on its correspondence with the window range (for example, through a preset altitude-window range function or empirical data). Then, based on the preset image stitching ratio, for example, a ratio of 20%, assuming that the window range is a square with a side length of 5 meters, then the side length of the adjacent sub-area is 5×(1-20%)=4 meters. The area between the starting point and the end point is divided into multiple sub-areas according to this size. For each sub-area, its center position is determined by mathematical calculation (such as taking the average value of the coordinates of the four vertices of the sub-area), and the position is marked as the driving center point.

[0053] This embodiment can ensure that the drone can efficiently and accurately acquire images along a predetermined path when performing a shooting task. The window range is determined by the drone's flight altitude, ensuring that the field of view coverage of the shooting area matches the flight altitude, thereby obtaining clear images with sufficient details.

[0054] At the same time, this embodiment can control the overlap between adjacent sub-areas through the pre-set image stitching ratio, which ensures the consistency and accuracy of image stitching. When the drone flies over these driving center points, it can continuously obtain images that meet the stitching requirements, thereby greatly improving the efficiency and quality of image processing. This embodiment not only improves the accuracy of drone photography, but also optimizes the image processing process, ensuring the high-quality output of the final results.

[0055] In one embodiment of the present disclosure, determining a second driving path and a plurality of shooting center points of a drone based on a first driving path includes: Using the first driving path as the second driving path of the UAV; Take each driving center point as the shooting center point.

[0056] In this embodiment, since the unmanned ship and the drone work together, the driving path of the unmanned ship determines the shooting position of the drone. Using the first driving path of the unmanned ship directly as the second driving path of the drone can ensure that the drone remains synchronized during the movement of the unmanned ship. Each driving center point is used as the shooting center point, so that the drone can shoot at these key positions, which can effectively cover the entire monitoring area, and because it is consistent with the path of the unmanned ship, it can ensure the systematic and coherent image acquisition.

[0057] For example, the unmanned ship carries the drone to the starting point, determines the drone's flight altitude and window range, and the drone moves synchronously with the unmanned ship. When passing each center mark point (i.e., the center point of the form) from the starting point, the drone hovers and shoots. During the shooting process, the unmanned ship moves synchronously to the next node to ensure that the drone can continue shooting after completing a push-scan shooting of an area if the battery is sufficient. If the battery is insufficient, it returns to the unmanned ship.

[0058] In this embodiment, the second driving path of the drone is synchronized with the first driving path of the unmanned boat, ensuring that the drone is consistent with the unmanned boat during movement, improving monitoring efficiency and image acquisition consistency. The movement of the unmanned boat provides stable endurance support for the drone, ensuring that the drone can complete a wider range of shooting tasks when the power is sufficient, and improving the flexibility and reliability of the operation.

[0059] like Figure 2 and Figure 3 As shown, in one embodiment of the present disclosure, the image offset is calculated based on multiple water area images collected by the drone at multiple shooting center points, including: Generate a pixel point plane coordinate matrix based on multiple water area images; The image offset is calculated based on the pixel plane coordinate matrix and the position of the unmanned ship in the image.

[0060] In this embodiment, generating a pixel plane coordinate matrix based on a plurality of water area images includes: generating a two-dimensional plane coordinate matrix based on a plurality of water area images; Generate a pixel point plane coordinate matrix based on the two-dimensional plane coordinate matrix.

[0061] In this embodiment, the two-dimensional plane coordinate matrix is:

[0062] in, represents a two-dimensional plane coordinate matrix, Represents the coordinates of the first row and first column point in the water area image, represents the coordinates of the point in the i-th row and i-th column in the water area image, Represents the coordinates of the point in the nth row and nth column in the water image Represents the coordinates of the point in the 1st row and the nth column in the water area image, Represents the coordinates of the point in the nth row and first column in the water area image, where n represents the number of rows and columns of the two-dimensional plane coordinate matrix.

[0063] The pixel plane coordinate matrix is:

[0064] in, Represents the pixel plane coordinate matrix.

[0065] In this embodiment, the coordinate value of each point in the m matrix is ​​subtracted from the coordinate value of the point in the i-th row and i-th column, then The coordinate value of the point in the i-th row and i-th column becomes (0, 0).

[0066] This embodiment can quantify and organize image information in an orderly manner by generating a pixel point plane coordinate matrix. The process from a two-dimensional plane coordinate matrix to a pixel point plane coordinate matrix makes the image data more accurate and detailed, which helps to more accurately splice images in water area monitoring and avoid inaccurate monitoring information caused by image dislocation. At the same time, this quantitative processing method provides a more reliable data basis for subsequent image analysis, data processing and other work, and improves the overall quality and efficiency of water area monitoring work.

[0067] In one embodiment of the present disclosure, multiple water area images are stitched together based on image offsets, including: A plurality of water area images are corrected based on the image offset, and the corrected water area images are stitched.

[0068] In this embodiment, multiple water area images are corrected based on the image offset, and the corrected water area images are spliced, including: Calculate the correction factor based on the image offset and the flight altitude of the drone; The multiple water area images are corrected based on the correction coefficients, and the corrected water area images are spliced.

[0069] In this embodiment, the image offset reflects the positional deviation between the actual water image and the theoretically desired water image, and the flight altitude of the drone is related to the image shooting angle and imaging ratio. By combining the two, a correction coefficient can be calculated, which comprehensively considers the relative position of the image in space and the geometric relationship of the shooting. Using the correction coefficient to correct the image can adjust the position and proportion of the image as a whole, so that the images can be spliced ​​together more accurately, eliminating image misalignment caused by factors such as shooting angle and height.

[0070] In this embodiment, the correction coefficient is calculated based on the image offset and the flight altitude of the drone, including: Calculate the offset angle based on the image offset and the flight altitude; Calculate a correction factor based on the deviation angle and a correction formula; The multiple water area images are corrected based on the correction coefficients, and the corrected water area images are spliced.

[0071] Exemplarily, after shooting is completed, the shooting results of each window are processed. The processing flow may include: setting the theoretical center point position to (0, 0, 0), setting the actual position to (x, y, H), and the distances of the actual position from the center point position on the x-axis and y-axis are Δx and Δy respectively, which are the offsets on the x-axis and y-axis. Since the height is fixed, when the actual position is offset relative to the center point position, an angle (i.e., offset angle) will be formed. The calculation formula of the offset angle is as follows:

[0072]

[0073] in, Indicates the flight altitude of the drone. Indicates the x-axis angle formed when the actual position is offset from the center point position. It indicates the y-axis angle formed when the actual position is offset from the center point position, Δx indicates the distance between the actual position and the center point position on the x-axis, and Δy indicates the distance between the actual position and the center point position on the y-axis.

[0074] In this embodiment, calculating the correction coefficient based on the deviation angle and the correction formula includes: calculating the correction coefficients of the x-axis and the y-axis based on the deviation angle, and the calculation formula of the correction coefficient is:

[0075]

[0076] in, represents the correction coefficient of the x-axis, Indicates the correction factor of the y-axis.

[0077] according to and Create a correction coefficient function:

[0078] Wherein, k represents the correction coefficient function.

[0079] The corrected matrix is:

[0080] Pair Matrix Reset to obtain the target matrix M and complete the image correction. The reset formula is:

[0081] in, Represents the target matrix.

[0082] This embodiment combines the image offset and the flight altitude so that the correction coefficient can accurately reflect the spatial position and shooting angle of the image, thereby effectively eliminating the image misalignment problem caused by changes in shooting angle and altitude. The offset angle and correction coefficient are calculated by mathematical models, and the coordinates of the image pixel points are accurately transformed to achieve accurate image correction. Finally, by splicing the corrected images, a complete water area image is obtained, which greatly improves the image quality and provides more accurate data support for water area monitoring and analysis.

[0083] like Figure 4 As shown, in one embodiment of the present disclosure, a process of a water image stitching method based on the collaboration of a drone and an unmanned ship may include: constructing a water map; setting a shooting area, marking a starting point and an end point; fixing the flight altitude (i.e., the flight altitude of the drone), and fixing the window range as a square area; dividing the route into segments with equal-width marking points, and marking the center point according to the square area of ​​the window range; determining the reserved stitching percentage, planning the optimal path, and setting the drone flight route; the drone moves synchronously with the unmanned ship, and when the unmanned ship and the drone move to the center point, the drone hovers for shooting; establishing x-axis and y-axis offset coefficients based on the offset of the unmanned ship position in the image compared to the center position; performing image correction based on the x-axis and y-axis offset coefficients; obtaining the maximum offset percentage based on multiple offset calculations, and setting the maximum offset; and realizing drone stitching and optimization of vast waters.

[0084] In the disclosed embodiments, the water image stitching method of the UAV and the unmanned boat has a significant effect. First, the orderliness of image acquisition is ensured by constructing a water map and setting the shooting area. The UAV shoots at a fixed height and within the window range to ensure the consistency of image quality. Through equal-width marking points and optimal path planning, the shooting efficiency and the integrity of image coverage are improved. The synchronous movement and hovering shooting of the UAV and the unmanned boat ensure the stability and accuracy of the image. Image correction and offset calculation further improve the stitching accuracy, and ultimately achieve seamless stitching and optimization of images of vast waters, providing efficient technical support for water monitoring and management.

[0085] Corresponding to the above embodiment, a water image stitching method based on the collaboration of a drone and an unmanned boat is described. Figure 5 This is a structural block diagram of a water image stitching device based on the cooperation of a drone and an unmanned boat provided by an embodiment of the present disclosure. For the sake of convenience, only the parts related to the embodiment of the present disclosure are shown. Figure 5 A water body image stitching device 20 based on the collaboration of a drone and an unmanned ship includes: a path calculation module 21, an offset calculation module 22 and an image stitching module 23.

[0086] The path calculation module 21 is used to determine a first driving path of the unmanned ship based on the shooting area and a preset image stitching ratio, and to determine a second driving path and a plurality of shooting center points of the unmanned aerial vehicle based on the first driving path; The offset calculation module 22 is used to calculate the image offset based on the multiple water area images collected by the UAV at the multiple shooting center points; the image offset is the offset between the position of the unmanned ship in the image and the multiple shooting center points; The image stitching module 23 is used to stitch multiple water area images based on the image offset.

[0087] In one embodiment of the present disclosure, the path calculation module 21 is specifically used to determine the starting point and the ending point based on the shooting area; Determine multiple driving center points based on the starting point, the ending point and the preset image stitching ratio; A first driving path is determined based on a starting point, an ending point, and a plurality of driving center points.

[0088] In one embodiment of the present disclosure, the path calculation module 21 is further configured to determine the window range based on the flight altitude of the drone; The shooting area between the starting point and the ending point is divided into multiple sub-areas based on the window range and the preset image stitching ratio; the center of each sub-area is marked as the driving center point.

[0089] In one embodiment of the present disclosure, the path calculation module 21 is further configured to use the first driving path as the second driving path of the drone; Take each driving center point as the shooting center point.

[0090] In one embodiment of the present disclosure, the offset calculation module 22 is specifically used to generate a pixel point plane coordinate matrix based on a plurality of water area images; The image offset is calculated based on the pixel plane coordinate matrix and the position of the unmanned ship in the image.

[0091] In an embodiment of the present disclosure, the image stitching module 23 is specifically configured to correct a plurality of water area images based on the image offset, and stitch the corrected water area images.

[0092] In one embodiment of the present disclosure, the image stitching module 23 is further configured to calculate a correction coefficient based on the image offset and the flight altitude of the drone; The multiple water area images are corrected based on the correction coefficients, and the corrected water area images are spliced.

[0093] See also Figure 6 , Figure 6 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 6 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules in the above-mentioned device embodiments, such as Figure 5 The functions of modules 21 to 23 are shown.

[0094] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0095] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0096] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0097] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of a water image stitching method based on collaboration between a drone and an unmanned boat provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device 300 described in the embodiments of the present disclosure, which will not be repeated here.

[0098] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0099] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0100] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0101] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0102] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0104] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A water image stitching method based on the collaboration of unmanned aerial vehicles and unmanned boats, characterized in that: include: Determine a first driving path of the unmanned boat based on the shooting area and a preset image stitching ratio, and determine a second driving path and a plurality of shooting center points of the unmanned aerial vehicle based on the first driving path; Calculating an image offset based on multiple water area images collected by the drone at the multiple shooting center points; the image offset is the offset between the position of the unmanned ship in the image and the multiple shooting center points; The multiple water area images are stitched based on the image offset.

2. The method for stitching water body images based on the collaboration of unmanned aerial vehicles and unmanned boats as claimed in claim 1, characterized in that: The method of determining a first driving path of the unmanned ship based on the shooting area and a preset image stitching ratio includes: Determine a starting point and an ending point based on the shooting area; Determine a plurality of driving center points based on the starting point, the ending point and the preset image stitching ratio; The first driving path is determined based on the starting point, the ending point, and the multiple driving center points.

3. The method for stitching water body images based on the collaboration of unmanned aerial vehicles and unmanned boats as claimed in claim 2, characterized in that: The determining of a plurality of driving center points based on the starting point, the ending point and the preset image stitching ratio includes: Determine the window range based on the flight altitude of the drone; The shooting area between the starting point and the ending point is divided into a plurality of sub-areas based on the viewing window range and the preset image stitching ratio; and the center of each sub-area is marked as the driving center point.

4. The method for stitching water body images based on the collaboration of unmanned aerial vehicles and unmanned boats as claimed in claim 2, characterized in that: Determining a second driving path and a plurality of shooting center points of the drone based on the first driving path includes: Using the first driving path as the second driving path of the UAV; Each driving center point is used as a shooting center point.

5. The method for stitching water body images based on the collaboration of unmanned aerial vehicles and unmanned boats as claimed in claim 1, characterized in that: The calculating of the image offset based on the multiple water area images collected by the drone at the multiple shooting center points includes: Generate a pixel point plane coordinate matrix based on the multiple water area images; The image offset is calculated based on the pixel point plane coordinate matrix and the position of the unmanned ship in the image.

6. The method for stitching water body images based on the collaboration of unmanned aerial vehicles and unmanned boats as claimed in claim 1, characterized in that: The step of stitching the plurality of water area images based on the image offset comprises: The multiple water area images are corrected based on the image offset, and the corrected water area images are spliced.

7. The method for stitching water body images based on the collaboration of unmanned aerial vehicles and unmanned boats as claimed in claim 6, characterized in that: The correcting the multiple water area images based on the image offset and splicing the corrected water area images comprises: Calculating a correction coefficient based on the image offset and the flight altitude of the drone; The plurality of water area images are corrected based on the correction coefficient, and the corrected water area images are spliced.

8. A water image stitching device based on the collaboration of a drone and an unmanned boat, characterized in that: include: A path calculation module, used to determine a first driving path of the unmanned ship based on a shooting area and a preset image stitching ratio, and to determine a second driving path of the unmanned aircraft and a plurality of shooting center points based on the first driving path; An offset calculation module, used to calculate an image offset based on a plurality of water area images collected by the UAV at the plurality of shooting center points; the image offset is an offset between the position of the unmanned ship in the image and the plurality of shooting center points; An image stitching module is used to stitch the multiple water area images based on the image offset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images