Multi-screen display method and system based on unmanned aerial vehicle cluster carrying
Through the multi-screen display method of the drone cluster, combined with the combination of wide-angle and high-definition image technology, high-definition wide-area images are generated, which solves the problem of inconvenient image stitching when the drone cluster is flying, and achieves higher-definition image display.
Patent Information
- Application Number
- CN202411431292.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The drone cluster cannot maintain a stable flight array under changing weather and complex terrain, resulting in the acquired images being unable to be directly spliced into complete images, making it inconvenient to view and display.
Using a multi-screen display method based on a drone cluster, a wide-angle image and high-definition image are obtained through the controller, the target field of view and the detailed field of view are determined, image combination is performed, high-definition wide-area images are generated, and displayed on the display device.
It achieves a higher-definition and complete continuous image when the drone swarm passes by, which is more convenient to observe and solves the problem of inconvenient image stitching.
Smart Images

Figure CN118972710A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a multi-screen display method and system based on a drone cluster. Background Art
[0002] The drone swarm uses high-precision sensing equipment to collect a large amount of image information in the process of performing the scheduled tasks. This image information covers a variety of different scenes and content, including but not limited to topography, traffic conditions, weather changes, ecological and environmental protection, and other fields. Through advanced remote sensing technology and image processing algorithms, the drone swarm can process and analyze these image data in real time, providing accurate and timely visual feedback to the ground command center.
[0003] In the process of image acquisition, drone swarms can achieve precise flight and image acquisition in complex environments through preset communication protocols and adaptive adjustment mechanisms. However, in the face of unfavorable factors such as changeable weather and complex terrain, drone swarms may not be able to maintain a stable flight array, and images obtained by multiple drones cannot be directly spliced into a complete image, which is inconvenient to display in the observation map. Summary of the invention
[0004] The embodiment of the present application provides a multi-screen display method based on a drone cluster to improve the above-mentioned problem.
[0005] In order to achieve the above objectives, this application adopts the following technical solutions: In the first aspect, the embodiment of the present application proposes a multi-screen display method based on a drone cluster, which is applicable to a drone system, the system includes a controller, multiple display devices and multiple drones, the multiple drones include at least one first drone equipped with a wide-angle camera and multiple second drones equipped with high-definition cameras, the method includes: The controller acquires a wide-angle image acquired by the first drone, and determines a target field of view based on the wide-angle image, where the target field of view is an area displayed by the wide-angle image; The controller acquires a plurality of high-definition images acquired by the plurality of second drones, and determines a plurality of detail fields of view according to the plurality of high-definition images, where the detail fields of view are areas displayed by the high-definition images; The controller determines a target detail field of view located within the target field of view among the plurality of detail fields of view; The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images; The controller controls the display device to display high-definition wide-area images.
[0006] In combination with the first aspect, in some embodiments, the controller determines a target detail field of view located within the target field of view among the plurality of detail fields of view, including: The controller divides the wide-angle image into a plurality of first sub-images, and determines a plurality of first target sub-images from the plurality of first sub-images, wherein the first target sub-image is a first sub-image having a unique characteristic property from the plurality of sub-images, wherein the characteristic property includes an average brightness, an average grayscale, and an average RGB characteristic value; The controller divides the high-definition image into a plurality of second sub-images, and the size of the second sub-image is the same as the size of the first sub-image; If at least one of the plurality of second sub-images formed by segmenting a high-definition image has the same characteristic properties as any first target sub-image, the controller determines that the area displayed by the high-definition image is the target detail field of view.
[0007] In combination with the first aspect, in some implementations, the controller determines a plurality of target high-definition images based on a plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images, including: If the controller determines that the area displayed by a high-definition image is a target detail field of view, the controller determines that the high-definition image is a target high-definition image; The controller superimposes and completes multiple target high-definition images to form a high-definition target image.
[0008] In combination with the first aspect, in some implementations, the controller superimposes and completes a plurality of target high-definition images to form a high-definition target image, including: The controller determines a plurality of second target sub-images from a plurality of second sub-images formed by segmenting the plurality of target high-definition images based on the plurality of first target sub-images, wherein the second target sub-images are second sub-images having the same characteristic properties as the first target sub-images; The controller maps the position corresponding to the target second sub-image to the wide-angle image, wherein the target second sub-image with the same characteristic features corresponds to the position of the target first sub-image; The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to a combination of the final plurality of second sub-images and the plurality of target second sub-images.
[0009] In combination with the first aspect, in some implementations, the controller maps a second sub-image adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to a combination of the final plurality of second sub-images and the plurality of target second sub-images, including: The controller maps the second sub-image into the wide-angle image and determines whether the second sub-image is superimposed with other second sub-images; If the second sub-image is superimposed with other second sub-images, a second sub-image is selected from the superimposed second sub-images, and a high-definition wide-area image is generated based on the second sub-image combination.
[0010] In combination with the first aspect, in some embodiments, if the second sub-image is superimposed with other second sub-images, selecting a second sub-image from the superimposed second sub-images, and generating a high-definition wide-area image based on the second sub-image combination includes: The controller obtains a plurality of high-definition images corresponding to the plurality of second sub-images superimposed on each other, and determines a field of view center point of a target detail field of view corresponding to each high-definition image; The controller obtains the position information of the second sub-image, selects the second sub-image closest to the corresponding field of view center point from a plurality of mutually superimposed second sub-images, and generates a high-definition wide-area image based on the second sub-image.
[0011] In combination with the first aspect, in some implementations, the controller maps a second sub-image adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to a combination of the final plurality of second sub-images and the plurality of target second sub-images, further comprising: The controller maps the second sub-image into the wide-angle image and determines whether the second sub-image completely fills all positions in the wide-angle image; If the second sub-image does not completely fill all positions in the wide-angle image, the controller controls the second UAV to obtain images of the unfilled positions and generate the second sub-image of the high-definition wide-area image as a combination.
[0012] In combination with the first aspect, in some implementations, a plurality of display devices are arranged in an array, and a controller controls the display device to display a high-definition wide-area image, including: The controller divides the high-definition wide-area image into a plurality of third sub-images arranged in an array, and the number of the third sub-images is the same as the number of display devices and corresponds one to one; The controller controls the multiple display devices to display the corresponding third sub-image.
[0013] A second aspect of the embodiment of the present application proposes a multi-screen display system based on a drone cluster, including a controller, multiple display devices, and multiple drones, wherein the multiple drones include at least one first drone equipped with a wide-angle camera and multiple second drones equipped with high-definition cameras, and the system is configured as follows: The controller acquires a wide-angle image acquired by the first drone, and determines a target field of view based on the wide-angle image, where the target field of view is an area displayed by the wide-angle image; The controller acquires a plurality of high-definition images acquired by the plurality of second drones, and determines a plurality of detail fields of view according to the plurality of high-definition images, where the detail fields of view are areas displayed by the high-definition images; The controller determines a target detail field of view located within the target field of view among the plurality of detail fields of view; The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images; The controller controls the display device to display high-definition wide-area images.
[0014] In conjunction with the second aspect, in some embodiments, the system is configured as follows: The controller determines a target detail field of view located within the target field of view among the multiple detail fields of view, including: The controller divides the wide-angle image into a plurality of first sub-images, and determines a plurality of first target sub-images from the plurality of first sub-images, wherein the first target sub-image is a first sub-image having a unique characteristic property from the plurality of sub-images, wherein the characteristic property includes an average brightness, an average grayscale, and an average RGB characteristic value; The controller divides the high-definition image into a plurality of second sub-images, and the size of the second sub-image is the same as the size of the first sub-image; If at least one of the plurality of second sub-images formed by segmenting a high-definition image has the same characteristic properties as any first target sub-image, the controller determines that the area displayed by the high-definition image is the target detail field of view.
[0015] In conjunction with the second aspect, in some embodiments, the system is configured as follows: The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images, including: If the controller determines that the area displayed by a high-definition image is a target detail field of view, the controller determines that the high-definition image is a target high-definition image; The controller superimposes and completes multiple target high-definition images to form a high-definition target image.
[0016] In conjunction with the second aspect, in some embodiments, the system is configured as follows: The controller superimposes and completes multiple target high-definition images to form a high-definition target image, including: The controller determines a plurality of second target sub-images from a plurality of second sub-images formed by segmenting the plurality of target high-definition images based on the plurality of first target sub-images, wherein the second target sub-images are second sub-images having the same characteristic properties as the first target sub-images; The controller maps the position corresponding to the target second sub-image to the wide-angle image, wherein the target second sub-image with the same characteristic features corresponds to the position of the target first sub-image; The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to a combination of the final plurality of second sub-images and the plurality of target second sub-images.
[0017] In conjunction with the second aspect, in some embodiments, the system is configured as follows: The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to the combination of the final plurality of second sub-images and the plurality of target second sub-images, including: The controller maps the second sub-image into the wide-angle image and determines whether the second sub-image is superimposed with other second sub-images; If the second sub-image is superimposed with other second sub-images, a second sub-image is selected from the superimposed second sub-images, and a high-definition wide-area image is generated based on the second sub-image combination.
[0018] In conjunction with the second aspect, in some embodiments, the system is configured as follows: If the second sub-image is superimposed with other second sub-images, selecting a second sub-image from the superimposed second sub-images, and generating a high-definition wide-area image based on the second sub-image combination, including: The controller obtains a plurality of high-definition images corresponding to the plurality of second sub-images superimposed on each other, and determines a field of view center point of a target detail field of view corresponding to each high-definition image; The controller obtains the position information of the second sub-image, selects the second sub-image closest to the corresponding field of view center point from a plurality of mutually superimposed second sub-images, and generates a high-definition wide-area image based on the second sub-image.
[0019] In conjunction with the second aspect, in some embodiments, the system is configured as follows: The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to the combination of the final plurality of second sub-images and the plurality of target second sub-images, further comprising: The controller maps the second sub-image into the wide-angle image and determines whether the second sub-image completely fills all positions in the wide-angle image; If the second sub-image does not completely fill all positions in the wide-angle image, the controller controls the second UAV to obtain images of the unfilled positions and generate the second sub-image of the high-definition wide-area image as a combination.
[0020] In conjunction with the second aspect, in some embodiments, the system is configured as follows: Multiple display devices are arranged in an array, and the controller controls the display devices to display high-definition wide-area images, including: The controller divides the high-definition wide-area image into a plurality of third sub-images arranged in an array, and the number of the third sub-images is the same as the number of display devices and corresponds one to one; The controller controls the multiple display devices to display the corresponding third sub-image.
[0021] A third aspect of an embodiment of the present invention provides an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method proposed in the first aspect of the embodiment of the present invention.
[0022] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the embodiment of the present invention.
[0023] In summary, the above method and device have the following technical effects: The embodiment of the present application proposes a multi-screen display method based on a drone cluster. First, the controller obtains a wide-angle image obtained by a first drone, and then the controller obtains multiple high-definition images obtained by multiple second drones. Then, the controller determines multiple target high-definition images based on multiple target detail fields of view, and establishes and displays a high-definition wide-area image based on the multiple target high-definition images. The method proposed in the present application combines images obtained by different drone devices, determines the approximate orientation by the area of the wide-angle camera, and obtains detailed images by the high-definition drone and combines them into images of the same size as the images obtained by the wide-angle camera. It can obtain higher-definition and complete continuous images when passing through the drone group, which is more convenient to observe. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A flowchart of a multi-screen display method based on a drone cluster proposed in an embodiment of the present application. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0026] This application proposes a multi-screen display method based on a drone cluster, which is applicable to a drone system. Specifically, the drone system includes a controller, multiple display devices, and multiple drones, and the multiple drones include at least one first drone equipped with a wide-angle camera and multiple second drones equipped with high-definition cameras. Figure 1 The method proposed in the embodiment of the present application includes the following steps: S101: The controller obtains a wide-angle image obtained by a first drone, and determines a target field of view based on the wide-angle image, where the target field of view is an area displayed by the wide-angle image.
[0027] It is understandable that the drone system in the present application is a cluster system, that is, it includes drones with a variety of different configurations. In the present application, it mainly includes a first drone equipped with a wide-angle camera and multiple second drones equipped with high-definition cameras. Of course, in other embodiments, it can also include drones with other specific functions. The models of drones in the drone cluster are not repeated in this application. In the present application, the number of first drones, that is, drones equipped with wide-angle cameras, is less than that of the second drones. The first drone is used to obtain a large range of images. For example, when the cluster passes through a forest farm, the first drone can obtain images of the entire forest farm at a higher point in the entire cluster. In this embodiment, the field of view of the wide-angle camera of the first drone is also the field of view of the wide-angle camera, that is, the area displayed by the wide-angle image.
[0028] S102: The controller obtains a plurality of high-definition images obtained by a plurality of second drones, and determines a plurality of detail fields of view according to the plurality of high-definition images, where the detail fields of view are areas displayed by the high-definition images.
[0029] It is understandable that in this embodiment, the wide-angle image obtained by the first drone has the largest field of view, that is, the area covered by the image is the largest. However, due to equipment conditions, the detail image obtained by the first drone has low clarity. When using a cluster to obtain clear images such as pests and diseases, using only a wide-angle camera cannot meet the needs. Therefore, in this embodiment, it is also necessary to use the high-definition camera carried by the second drone to obtain a clearer image. Similar to the wide-angle camera, the field of view of the high-definition camera is also the detail field of view, and the detail field of view is necessarily smaller than the target field of view.
[0030] S103: The controller determines a target detail field of view located within the target field of view among the multiple detail fields of view.
[0031] It can be understood that in this embodiment, the area determined by the wide-angle camera field of view is the area that needs to be scanned or imaged. In this embodiment, the wide-angle camera and the target field of view play a role in determining the area, and the high-definition camera and the detail field of view are used to supplement the detailed image in the area.
[0032] Specifically, as an implementation, step S103 includes the following steps: Since the multiple second drones cannot move in a perfect array formation during flight, the acquired images cannot be directly stitched, and images that are not of the target area may also be acquired during the flight. Therefore, it is necessary to determine the target detail field of view that is within the target field of view among the multiple detail fields of view.
[0033] S1031: The controller divides the wide-angle image into multiple first sub-images, and determines multiple first target sub-images among the multiple first sub-images, wherein the first target sub-image is the first sub-image having unique characteristic characteristics among the multiple sub-images, wherein the characteristic characteristics include average brightness, average grayscale, and average RGB characteristic values.
[0034] It can be understood that in this embodiment, a feature comparison method is adopted, that is, the wide-angle image is first divided into multiple first sub-images, each of which has its unique features. Exemplarily, the characteristic property in this embodiment can be one of the average brightness, average grayscale, and average RGB feature value. Of course, in some other embodiments, artificial intelligence can also be used to identify certain feature points in the first sub-image, which is not limited here. The characteristic attributes of the first sub-image are used to match with the high-definition image.
[0035] S1032: The controller divides the high-definition image into a plurality of second sub-images, and the size of the second sub-image is the same as the size of the first sub-image.
[0036] It can be understood that in order to match the high-definition image with the wide-angle image in the above step, in this embodiment, the multiple high-definition images acquired by the second drone need to be divided into images with the same granularity as the first sub-image to facilitate matching and comparison.
[0037] S1033: If at least one of the plurality of second sub-images formed by dividing a high-definition image has the same characteristic features as any first target sub-image, the controller determines that the area displayed by the high-definition image is the target detail field of view.
[0038] It can be understood that in this embodiment, through the processing of the controller, multiple second sub-images formed by segmenting multiple high-definition images can be matched with multiple first sub-images based on characteristic characteristics. For example, there is a tree in a first sub-image. When a high-definition camera scans the tree, among the second sub-images formed by segmenting the high-definition image, there must be a second sub-image with a tree, that is, the second sub-image is matched with the first sub-image, that is, the market in which the high-definition camera acquires the image is the target detail field of view.
[0039] S104: The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images.
[0040] It can be understood that in this embodiment, after the target detail market is determined, the image corresponding to the target detail market is also the target high-definition image, that is, the image is captured in the wide-angle image. Therefore, by combining multiple target high-definition images captured by the drone cluster, a higher-definition high-definition image that is consistent with the wide-angle image can be obtained.
[0041] During the superposition process, due to the uncertainty of the position of the drone shooting the high-definition image, a feature point may be photographed by multiple high-definition drones. In order to solve the problem of repeated superposition, only one second sub-image can be selected. Specifically, in some embodiments, step S104 may include the following steps: S1041: The controller maps the second sub-image into the wide-angle image, and determines whether the second sub-image is superimposed with other second sub-images.
[0042] It is understandable that when mapping multiple second sub-images into the wide-angle image, it is possible to determine whether superposition occurs through positional relationship or layer recognition, and the specific determination method is not limited herein.
[0043] S1042: If the second sub-image is superimposed with other second sub-images, select a second sub-image from the superimposed second sub-images, and generate a high-definition wide-area image based on the second sub-image combination.
[0044] As for how to select a second sub-image from the second sub-images cut from multiple different high-definition images, in this embodiment, multiple high-definition images corresponding to the multiple superimposed second sub-images can be obtained, and the field of view center point of the target detail field of view corresponding to each high-definition image can be determined, and then the controller obtains the position information of the second sub-image, selects the second sub-image closest to the corresponding field of view center point from the multiple superimposed second sub-images, and generates a high-definition wide-area image based on the second sub-image. In this way, according to common sense in the field, the regional distortion of the center point of the camera principle is large, while the position distortion close to the center point is small. Therefore, selecting the second sub-image closest to the corresponding field of view center point can minimize the distortion in the final result.
[0045] In some other embodiments, there may be a problem of filling holes, that is, the required area is not matched or not captured by the high-definition camera. Therefore, in this embodiment, step S104 may further include the following steps: S1043: The controller maps the second sub-image into the wide-angle image, and determines whether the second sub-image completely fills all positions in the wide-angle image.
[0046] It can be understood that the unfilled positions can be determined by finally combining the positional relationship of the multiple second sub-images. Of course, there are other determination methods, which are not limited here.
[0047] S1044: If the second sub-image does not completely fill all positions in the wide-angle image, the controller controls the second UAV to obtain images of the unfilled positions and use them as a combination to generate a second sub-image of the high-definition wide-area image.
[0048] It is understandable that when an unfilled position is determined, the image of the corresponding position is lost. At this time, the drone equipped with a high-definition camera can be controlled to return and shoot to obtain a complete image of the area.
[0049] S105: The controller controls the display device to display a high-definition wide-area image.
[0050] It is understandable that after acquiring a complete high-definition wide-area image, the controller can control the image to be displayed on a display device for easy observation by the user. In some embodiments, a display may not be able to display a high-definition wide-area image with a large frame. Therefore, as an embodiment, a plurality of display device arrays may be set up, and the controller divides the high-definition wide-area image into a plurality of third sub-images arranged in an array. The number of third sub-images is the same as the number of display devices and corresponds one to one. Then, the controller controls the plurality of display devices to display the corresponding third sub-images.
[0051] The embodiment of the present application proposes a multi-screen display method based on a drone cluster. First, the controller obtains a wide-angle image obtained by a first drone, and then the controller obtains multiple high-definition images obtained by multiple second drones. Then, the controller determines multiple target high-definition images based on multiple target detail fields of view, and establishes and displays a high-definition wide-area image based on the multiple target high-definition images. The method proposed in the present application combines images obtained by different drone devices, determines the approximate direction by the area of the wide-angle camera, and obtains detailed images by the high-definition drone and combines them into images of the same size as the images obtained by the wide-angle camera. It can obtain higher-definition and complete continuous images when the drone group passes by, which is more convenient to observe.
[0052] Based on the same inventive concept, the present application also proposes a multi-screen display system based on a drone cluster, including a controller, multiple display devices, and multiple drones, wherein the multiple drones include at least one first drone equipped with a wide-angle camera and multiple second drones equipped with high-definition cameras. The system is configured as follows: The controller acquires a wide-angle image acquired by the first drone, and determines a target field of view based on the wide-angle image, where the target field of view is an area displayed by the wide-angle image; The controller acquires a plurality of high-definition images acquired by the plurality of second drones, and determines a plurality of detail fields of view according to the plurality of high-definition images, where the detail fields of view are areas displayed by the high-definition images; The controller determines a target detail field of view located within the target field of view among the plurality of detail fields of view; The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images; The controller controls the display device to display high-definition wide-area images.
[0053] In some embodiments, the system is configured to: The controller determines a target detail field of view located within the target field of view among the multiple detail fields of view, including: The controller divides the wide-angle image into a plurality of first sub-images, and determines a plurality of first target sub-images from the plurality of first sub-images, wherein the first target sub-image is a first sub-image having a unique characteristic property from the plurality of sub-images, wherein the characteristic property includes an average brightness, an average grayscale, and an average RGB characteristic value; The controller divides the high-definition image into a plurality of second sub-images, and the size of the second sub-image is the same as the size of the first sub-image; If at least one of the plurality of second sub-images formed by segmenting a high-definition image has the same characteristic properties as any first target sub-image, the controller determines that the area displayed by the high-definition image is the target detail field of view.
[0054] In some embodiments, the system is configured to: The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images, including: If the controller determines that the area displayed by a high-definition image is a target detail field of view, the controller determines that the high-definition image is a target high-definition image; The controller superimposes and completes multiple target high-definition images to form a high-definition target image.
[0055] In some embodiments, the system is configured to: The controller superimposes and completes multiple target high-definition images to form a high-definition target image, including: The controller determines a plurality of second target sub-images from a plurality of second sub-images formed by segmenting the plurality of target high-definition images based on the plurality of first target sub-images, wherein the second target sub-images are second sub-images having the same characteristic properties as the first target sub-images; The controller maps the position corresponding to the target second sub-image to the wide-angle image, wherein the target second sub-image with the same characteristic features corresponds to the position of the target first sub-image; The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to a combination of the final plurality of second sub-images and the plurality of target second sub-images.
[0056] In some embodiments, the system is configured to: The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to the combination of the final plurality of second sub-images and the plurality of target second sub-images, including: The controller maps the second sub-image into the wide-angle image and determines whether the second sub-image is superimposed with other second sub-images; If the second sub-image is superimposed with other second sub-images, a second sub-image is selected from the superimposed second sub-images, and a high-definition wide-area image is generated based on the second sub-image combination.
[0057] In some embodiments, the system is configured to: If the second sub-image is superimposed with other second sub-images, selecting a second sub-image from the superimposed second sub-images, and generating a high-definition wide-area image based on the second sub-image combination, including: The controller obtains a plurality of high-definition images corresponding to the plurality of second sub-images superimposed on each other, and determines a field of view center point of a target detail field of view corresponding to each high-definition image; The controller obtains the position information of the second sub-image, selects the second sub-image closest to the corresponding field of view center point from a plurality of mutually superimposed second sub-images, and generates a high-definition wide-area image based on the second sub-image.
[0058] In some embodiments, the system is configured to: The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates a high-definition wide-area image according to the combination of the final plurality of second sub-images and the plurality of target second sub-images, further comprising: The controller maps the second sub-image into the wide-angle image and determines whether the second sub-image completely fills all positions in the wide-angle image; If the second sub-image does not completely fill all positions in the wide-angle image, the controller controls the second UAV to obtain images of the unfilled positions and generate the second sub-image of the high-definition wide-area image as a combination.
[0059] In some embodiments, the system is configured to: Multiple display devices are arranged in an array, and the controller controls the display devices to display high-definition wide-area images, including: The controller divides the high-definition wide-area image into a plurality of third sub-images arranged in an array, and the number of the third sub-images is the same as the number of display devices and corresponds one to one; The controller controls the multiple display devices to display the corresponding third sub-image.
[0060] The embodiment of the present application proposes a multi-screen display system based on a drone cluster. First, the controller obtains a wide-angle image obtained by a first drone, and then the controller obtains multiple high-definition images obtained by multiple second drones. Then, the controller determines multiple target high-definition images based on multiple target detail fields of view, and establishes and displays a high-definition wide-area image based on the multiple target high-definition images. The method proposed in the present application combines images obtained by different drone devices, determines the approximate direction by the area of the wide-angle camera, and obtains detailed images by the high-definition drone and combines them into images of the same size as the images obtained by the wide-angle camera. It can obtain higher-definition and complete continuous images when the drone group passes by, which is more convenient to observe.
[0061] Based on the same inventive concept, an embodiment of the present application further proposes an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the multi-screen display method based on a drone cluster according to an embodiment of the present application.
[0062] In addition, to achieve the above-mentioned purpose, an embodiment of the present application also proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the multi-screen display method based on a drone cluster of the embodiment of the present application is implemented.
[0063] The following is a detailed introduction to the various components of electronic equipment: The processor is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).
[0064] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.
[0065] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.
[0066] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor, or may exist independently and be coupled to the processor through an interface circuit of the electronic device, which is not specifically limited in the embodiments of the present invention.
[0067] A transceiver is used to communicate with a network device or a terminal device.
[0068] Optionally, the transceiver may include a receiver and a transmitter, wherein the receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.
[0069] Optionally, the transceiver may be integrated with the processor, or may exist independently and be coupled to the processor via an interface circuit of the router, which is not specifically limited in the embodiment of the present invention.
[0070] In addition, the technical effects of the electronic device can refer to the technical effects of the data transmission method in the above method embodiment, which will not be repeated here.
[0071] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0072] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0073] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0074] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.
[0075] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ad, bc, or abc, where a, b, and c can be single or multiple.
[0076] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0077] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
Claims
1. A multi-screen display method based on drone clusters, characterized in that: Applicable to a drone system, the system includes a controller, multiple display devices and multiple drones, the multiple drones include at least one first drone equipped with a wide-angle camera and multiple second drones equipped with high-definition cameras, the method includes: The controller acquires a wide-angle image acquired by the first drone, and determines a target field of view based on the wide-angle image, where the target field of view is an area displayed by the wide-angle image; The controller acquires a plurality of high-definition images acquired by the plurality of second drones, and determines a plurality of detail fields of view according to the plurality of high-definition images, wherein the detail fields of view are areas displayed by the high-definition images; The controller determines a target detail field of view located within the target field of view among the plurality of detail fields of view; The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images; The controller controls the display device to display the high-definition wide-area image.
2. According to claim 1, a multi-screen display method based on drone clusters is characterized in that: The controller determines a target detail field of view located within the target field of view among the plurality of detail fields of view, including: The controller divides the wide-angle image into a plurality of first sub-images, and determines a plurality of first target sub-images from the plurality of first sub-images, wherein the first target sub-image is a first sub-image having a unique characteristic among the plurality of sub-images, wherein the characteristic characteristic includes an average brightness, an average grayscale, and an average RGB characteristic value; The controller divides the high-definition image into a plurality of second sub-images, and the size of the second sub-image is the same as the size of the first sub-image; If at least one of the plurality of second sub-images formed by dividing a high-definition image has the same characteristic features as any of the first target sub-images, the controller determines that the area displayed by the high-definition image is the target detail field of view.
3. According to claim 2, a multi-screen display method based on drone clusters is characterized in that: The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images, including: If the controller determines that an area displayed by the high-definition image is the target detail field of view, the controller determines that the high-definition image is the target high-definition image; The controller superimposes and completes a plurality of the target high-definition images to form a high-definition target image.
4. According to claim 3, a multi-screen display method based on drone clusters is characterized in that: The controller superimposes and completes the plurality of target high-definition images to form the high-definition target image, including: The controller determines, based on the plurality of first target sub-images, a plurality of second target sub-images from a plurality of second sub-images formed by segmenting the plurality of target high-definition images, wherein the second target sub-images are second sub-images having the same characteristic features as the first target sub-images; The controller maps the position corresponding to the target second sub-image into the wide-angle image, wherein the target second sub-image having the same characteristic property corresponds to the position of the target first sub-image; The controller maps the second sub-images adjacent to the target second sub-image into the wide-angle image, and generates the high-definition wide-area image according to a combination of a final plurality of the second sub-images and a plurality of target second sub-images.
5. The multi-screen display method based on drone cluster according to claim 4 is characterized in that: The controller maps the second sub-images adjacent to the target second sub-images into the wide-angle image, and generates the high-definition wide-area image according to a combination of a final plurality of the second sub-images and a plurality of target second sub-images, including: The controller maps the second sub-image into the wide-angle image, and determines whether the second sub-image is superimposed with other second sub-images; If the second sub-image is superimposed with other second sub-images, one second sub-image is selected from the superimposed second sub-images, and the high-definition wide-area image is generated based on the combination of the second sub-images.
6. The multi-screen display method based on drone cluster according to claim 5 is characterized in that: If the second sub-image is superimposed with other second sub-images, selecting one second sub-image from the superimposed second sub-images, and generating the high-definition wide-area image based on the combination of the second sub-images, including: The controller acquires a plurality of the high-definition images corresponding to the plurality of the second sub-images superimposed on each other, and determines a field of view center point of a target detail field of view corresponding to each of the high-definition images; The controller obtains the position information of the second sub-image, selects the second sub-image closest to the corresponding center point of the field of view from a plurality of superimposed second sub-images, and generates the high-definition wide-area image based on the second sub-image.
7. The multi-screen display method based on drone cluster according to claim 5 is characterized in that: The controller maps the second sub-images adjacent to the target second sub-images into the wide-angle image, and generates the high-definition wide-area image according to a combination of the final plurality of the second sub-images and the plurality of target second sub-images, further comprising: The controller maps the second sub-image into the wide-angle image and determines whether the second sub-image completely fills all positions in the wide-angle image; If the second sub-image does not completely fill all positions in the wide-angle image, the controller controls the second UAV to obtain images of the unfilled positions and generate the second sub-image of the high-definition wide-area image as a combination.
8. A multi-screen display method based on drone cluster according to any one of claims 1 to 7, characterized in that: A plurality of display devices are arranged in an array, and the controller controls the display devices to display the high-definition wide-area image, including: The controller divides the high-definition wide-area image into a plurality of third sub-images arranged in an array, and the number of the third sub-images is the same as the number of the display devices and corresponds one to one; The controller controls the plurality of display devices to display the corresponding third sub-image.
9. A multi-screen display system based on a drone cluster, characterized in that: The system comprises a controller, a plurality of display devices and a plurality of drones, wherein the plurality of drones comprises at least one first drone equipped with a wide-angle camera and a plurality of second drones equipped with high-definition cameras, and the system is configured as follows: The controller acquires a wide-angle image acquired by the first drone, and determines a target field of view based on the wide-angle image, where the target field of view is an area displayed by the wide-angle image; The controller acquires a plurality of high-definition images acquired by the plurality of second drones, and determines a plurality of detail fields of view according to the plurality of high-definition images, wherein the detail fields of view are areas displayed by the high-definition images; The controller determines a target detail field of view located within the target field of view among the plurality of detail fields of view; The controller determines a plurality of target high-definition images based on the plurality of target detail fields of view, and establishes a high-definition wide-area image based on the plurality of target high-definition images; The controller controls the display device to display the high-definition wide-area image.
10. An electronic device, characterized in that: include: at least one processor; And, a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by at least one of the processors, and the instructions are executed by at least one of the processors to enable at least one of the processors to execute the method as claimed in claim 1.
Citation Information
Patent Citations
Image acquisition method and device, electronic equipment and computer readable storage medium
CN112087571A
Image acquisition method and device, electronic equipment and computer readable storage medium
CN112087580A
Image processing method and device and electronic equipment
CN113570617A
Camera module control method, camera module and electronic equipment
CN116567401A
Multi-screen synchronous display method and device, electronic equipment and storage medium
CN117032605A
