Image processing method and device thereof, electronic device and computer readable storage medium

Abstract

The application provides an image processing method and device, an electronic device and a computer readable storage medium, and relates to the technical field of mobile devices. The method is applied to a ground solution server and includes the following steps: receiving N images collected by a mobile device and current solution data of the N images, wherein the current solution data of the images is determined based on a solution process of the images that has been completed in the mobile device, N is a positive integer; and based on the current solution data of the N images, the N images are continuously solved to obtain target surveying and mapping data corresponding to the N images. The image processing method can perform part of the solution task during the operation of the mobile device, reduces the waste of computing resources during the operation of the mobile device, and thus achieves the purposes of improving the aerial surveying and mapping efficiency, reducing the waiting time for mapping, and reducing the solution pressure of the subsequent ground solution server.

Description

Technical Field

[0001] The present application relates to the technical field of mobile equipment, and in particular to an image processing method and apparatus thereof, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the continuous development of science and technology, mapping technology has been widely used in industries such as map surveying and agricultural plant protection. When the survey area is too large, mobile devices have difficulty quickly producing maps due to the limitations of their computing power and battery life.

[0003] Traditionally, ground-based computation servers have been used to batch compute all images collected by mobile devices. This not only wastes computing resources during mobile device operations, resulting in low image output efficiency and long wait times, but also places excessive pressure on the ground-based computation servers. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide an image processing method, an image processing device, an electronic device and a computer-readable storage medium to solve the technical problems in the prior art of low map output efficiency, long waiting time, and excessive pressure on the ground solution server.

[0005] According to a first aspect of an embodiment of the present application, there is provided an image processing method, which is applied to a ground solution server, the method comprising: receiving N images collected by a mobile device and current solution data of each of the N images, wherein the current solution data of the image is determined based on a solution process that has been completed for the image in the mobile device, and N is a positive integer; and continuing to solve the N images based on the current solution data of each of the N images to obtain target mapping data corresponding to each of the N images.

[0006] In one embodiment, based on the current solution data of each of the N images, the N images are continued to be solved to obtain digital orthoimages corresponding to each of the N images, including: determining the remaining solution process corresponding to each of the N images based on the current solution data of each of the N images; and executing the remaining solution process corresponding to each of the N images based on the current solution data of each of the N images to obtain target mapping data corresponding to each of the N images.

[0007] In one embodiment, the current solution data includes different types of image result data; wherein, based on the current solution data of each of the N images, the remaining solution processes corresponding to each of the N images are determined, including: based on the type of the image result data corresponding to each of the N images, determining the current solution process corresponding to each of the N images; based on the current solution process corresponding to each of the N images, determining the remaining solution process corresponding to each of the N images.

[0008] In one embodiment, based on the type of image result data corresponding to each of the N images, the current solution process corresponding to each of the N images is determined, including: if the type of the image result data corresponding to the image is a digital surface model image, then the current solution process of the image is determined to have completed the digital surface model solution process; if the type of the image result data corresponding to the image is a digital orthoimage, then the current solution process of the image is determined to have completed all solution processes.

[0009] In one embodiment, based on current solution data of each of N images, a remaining solution process corresponding to each of the N images is executed to obtain a digital orthoimage corresponding to each of the N images, including: if the remaining solution processes corresponding to each of M images among the N images are digital orthoimage solution processes, then for each of the M images, an orthorectification operation is performed based on the pose data and the digital surface model image in the current solution data of the image to obtain a digital orthoimage corresponding to the image, where M is a positive integer less than N.

[0010] According to a second aspect of an embodiment of the present application, an image processing method is provided, which is applied to a movable device, the movable device including an acquisition module and a processing module, the method including: acquiring N images acquired by the acquisition module, where N is a positive integer; after the image is acquired, the processing module performs real-time processing on the image, and determines the current processing data of the image based on the completed processing process of the image; and storing the N images and the current processing data of each of the N images.

[0011] In one embodiment, based on the completed image solution process, the current solution data of the image is determined, including: at a preset time point, all data generated in the completed image solution process is used as the current solution data; wherein the preset time point is the time point when the latest image in the sequence of N images has completed the digital surface model solution process, or the time point when the movable device completes the task and returns to the docking site at the airport.

[0012] According to a third aspect of an embodiment of the present application, an image processing device is provided for use in a ground solution server, wherein the image to be solved device includes: a receiving module configured to receive current solution data of N images collected by a movable device and N images to be solved, wherein the current solution data of the image to be solved is determined based on a solution process that has been completed for the image to be solved in the movable device for the image to be solved, and N is a positive integer; and a solution module configured to continue to solve the N images to be solved based on the current solution data of each of the N images to be solved, so as to obtain target surveying and mapping data corresponding to each of the N images to be solved.

[0013] According to a fourth aspect of an embodiment of the present application, an image processing device is provided, which is applied to a movable device, and the device includes: an acquisition module, configured to acquire N images, where N is a positive integer; a processing module, configured to solve the image in real time after the image is acquired, and determine the current solved data of the image based on the completed solving process of the image; a storage module, configured to store the N images and the current solved data of each of the N images.

[0014] According to a fifth aspect of an embodiment of the present application, an electronic device is provided, comprising: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the image processing method of the first aspect or the second aspect described above.

[0015] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the image processing method of the first or second aspect described above.

[0016] The image processing method provided in the embodiment of the present application is combined with the aerial survey data being solved in real time by the mobile device after being collected by the mobile device, and based on the completed solution process of the image, the current solution data of the image can be determined, which can provide a basis for subsequent continued solution. After receiving N images collected by the mobile device and the current solution data of each of the N images, the ground solution server continues to solve the N images based on the current solution data of each of the N images to obtain the target surveying and mapping data corresponding to each of the N images. This method realizes the purpose of executing part of the solution task during the operation of the mobile device, making full use of the computing power resources during the operation of the mobile device, reducing the waste of computing power resources during the operation of the mobile device, improving the drawing efficiency, reducing the waiting time, and effectively reducing the task pressure of the post-learning solution platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The figure shows a system architecture diagram of an application scenario of the image processing method provided by an embodiment of the present application.

[0018] Figure 2 FIG2 is a flow chart of an image processing method provided by an embodiment of the present disclosure.

[0019] Figure 3 The figure shows a flow chart of continuing to solve N images based on the current solved data of each of the N images to obtain target mapping data corresponding to each of the N images, provided by an embodiment of the present disclosure.

[0020] Figure 4FIG2 is a schematic diagram of the structure of an image processing device provided in one embodiment of the present application.

[0021] Figure 5 Shown is a structural schematic diagram of an image processing device provided in yet another embodiment of the present application.

[0022] Figure 6 Shown is a structural schematic diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0024] Mapping technology uses mobile devices to collect images, solves them, and constructs digital orthophoto maps (DOMs). These DOMs are then used to create topographic maps. Because topographic maps are crucial to industries like mapping and agricultural plant protection, mapping technology is widely used in these areas.

[0025] When the aerial survey area is too large, such as tens of thousands of acres, it is difficult for the mobile device to obtain digital orthophotos of all images when completing the task due to the computing power and endurance of the mobile device. In other words, it is difficult to achieve the purpose of rapid map output. In addition, due to the impact of the network speed of the mobile device on data migration and data processing efficiency, the images collected by the mobile device in real time cannot be transmitted to the ground solution server in real time and with clear image quality. In traditional technology, after the mobile device returns to the docking site, all collected images are migrated to the ground solution server, and the ground solution server is used to perform batch solution on all images. Traditional technology not only causes a waste of computing resources during the operation of the mobile device, resulting in low map output efficiency and long waiting time, but also causes excessive pressure on the ground solution server.

[0026] In order to solve the above problems, an embodiment of the present application provides an image processing method, which combines the image data with the real-time solution of the mobile device after being collected by the mobile device, and based on the completed solution process of the image, the current solution data of the image can be determined, which can provide a basis for subsequent continued solution. After receiving N images collected by the mobile device and the current solution data of each of the N images, the ground solution server continues to solve the N images based on the current solution data of each of the N images to obtain the target mapping data corresponding to each of the N images. This method realizes the purpose of executing part of the solution task during the operation of the mobile device, making full use of the computing power resources during the operation of the mobile device, reducing the waste of computing power resources during the operation of the mobile device, improving the drawing efficiency, reducing the waiting time, and effectively reducing the task pressure of the post-learning solution platform.

[0027] The following combination Figure 1 An example is given to illustrate the system architecture of the application scenario of the image processing method.

[0028] like Figure 1 As shown, the system architecture of the application scenario of the image processing method provided by the embodiment of the present disclosure involves a movable device 110, a control platform 120 communicatively connected to the movable device 110, and a ground solution server 130 capable of transmitting data with the movable device 110.

[0029] In actual use, mobile device 110 receives a task instruction from control platform 120. In response to the task instruction, mobile device 110 collects N images of the area to be surveyed, where N is a positive integer. After each image is captured, it undergoes real-time computation, and based on the image's completed computation progress, the image's current computational data is determined. After mobile device 110 completes its mission, that is, after it returns to its docking station, it transmits the N images and their respective current computational data to ground computation server 130.

[0030] After receiving the N images collected by the mobile device 110 and the current solution data of each of the N images, the ground solution server 130 continues to solve the N images based on the current solution data of each of the N images, obtains the target mapping data corresponding to each of the N images, and obtains a surface topography map based on the target mapping data corresponding to each of the N images.

[0031] Exemplarily, mobile device 110 includes, but is not limited to, drones and unmanned vehicles. Control platform 120 is a server that hosts control functions. Ground solution server 130 is a server that hosts solution functions. In one example, control platform 120 is a standalone server. In another example, control platform 120 is a server integrated with ground solution server 130.

[0032] In an embodiment of the present application, part of the solution task can be performed during the operation of the mobile device, and the computing power resources during the operation of the mobile device can be fully utilized, thereby reducing the waste of computing power resources during the operation of the mobile device, improving the efficiency of drawing, reducing waiting time, and effectively reducing the task pressure of the subsequent solution platform.

[0033] The following combination Figures 2 to 6 The image processing method, image processing method device, electronic device and computer-readable storage medium mentioned in the embodiments of the present application are introduced in detail.

[0034] Figure 2 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present disclosure. Figure 2 As shown, the image processing method mentioned in the embodiment of the present disclosure involves a ground solution server and a movable device.

[0035] For a removable device, the removable device includes a collection module and a processing module. The data processing method provided by the embodiment of the present disclosure includes the following steps.

[0036] Step S201: Acquire N images captured by the capture module.

[0037] For example, N is a positive integer. The images mentioned above are images of the area to be surveyed taken by the mobile device during operation, and the N images can be obtained by photographic imaging.

[0038] Step S202: After the image is captured, the processing module performs real-time image processing and determines the current image processing data based on the completed image processing process.

[0039] In order to make full use of the computing resources of the mobile device and reduce the waste of computing resources during the operation of the mobile device, the mobile device immediately solves the image in real time after collecting each image.

[0040] The specific implementation method of the real-time solution mentioned above is to extract the sparse point cloud and pose information corresponding to the image through the Structure From Motion (SFM) algorithm, and then obtain the corresponding digital surface model (DSM) image of the image through methods such as inverse distance weighting or triangulation based on the obtained sparse point cloud. Then, based on the obtained DSM and the corresponding pose information of the image, the corresponding digital orthophoto image DOM is constructed. In other words, the digital orthophoto image DOM is a specific form of target mapping data. It should be noted that the solution process for each image is different, and the entire real-time solution process mentioned above is not necessarily executed.

[0041] Based on the above description, the overall solution process includes the DSM solution process and the digital orthoimage solution process. The DSM solution process converts an image into its corresponding DSM image, while the digital orthoimage solution process converts the DSM image into a digital orthoimage.

[0042] Since the N images are acquired at different times, the solution processes of the N images are also different. Some images may have completed the digital orthoimage solution process and obtained the digital orthoimage corresponding to the image, while some images may have just completed the digital surface model solution process and only obtained the digital surface model image corresponding to the image.

[0043] In order for the ground-based solution server to continue solving the image based on the portion of the solution already performed by the mobile device, it is necessary to determine the current image solution process and transmit the current solution data obtained during the current solution process to the ground-based solution server. In other words, determining the current solution process and obtaining the current solution data provide the basis for the ground-based solution server to perform subsequent solutions, enabling the ground-based solution server to continue solving the image based on the portion of the solution task performed by the mobile device.

[0044] Exemplarily, for each solved data item in the N images, the current solved data item of the image is determined based on a completed solving process for the image. Determining the current solved data item of the image based on a completed solving process for the image can be implemented by, at a predetermined time point, using all data generated in the completed solving process for the image as the current solved data item.

[0045] In one example, the preset time point is the time point when the mobile device stops at the airport after completing the mission and returning. Specifically, the above-mentioned determination of the current solution data of the image based on the completed solution process of the image can be implemented as follows: when the mobile device stops at the docking site after completing the mission and returning, all data generated in the completed solution process of the image is used as the current solution data. Selecting the time point when the mobile device stops solving at the airport as the node at which the mobile device stops solving can make the best use of the computing resources of the mobile device, maximize the use of the computing power of the mobile device, and minimize the overall solution time.

[0046] In another example, the preset time point is the time point at which the latest image in the N images has completed the digital surface model solution process. Specifically, the aforementioned determination of the current solution data of an image based on the completed solution process of the image can also be implemented by using all data generated in the completed solution process of the image as the current solution data when the latest image in the N images has completed the digital surface model solution process.

[0047] Since the digital orthophoto image solution process uses the largest computing power resources from DSM to DOM, the time point when the latest image in the N images has completed the digital surface model solution process is selected as the node for stopping the solution of the mobile device. This can effectively and reasonably allocate the computing power of the mobile device and the ground settlement server, and while reducing the waiting time for map output compared with traditional technologies, it can also extend the service life of the mobile device.

[0048] From the above description, it can be seen that, for example, if a certain image has completed the digital orthophoto solution process, the pose data, point cloud data, and DSM image generated by the image during the digital surface model solution process are determined as the current solution data.

[0049] Step S203: store the N images and the current solution data of each of the N images.

[0050] Furthermore, the stored N images and the current solution data of each of the N images are transmitted to the ground solution server, so that the ground solution server continues to solve the N images based on the current solution data of each of the N images to obtain the target mapping data corresponding to each of the N images.

[0051] In one example, N images and current solution data of each of the N images are stored in a flash memory card (TF) of a removable device, and the TF card of the removable device is placed in a card slot of a ground solution server, so that the ground solution server reads the current solution data of each of the N images from the TF card and transmits it to the ground solution server.

[0052] In another example, the N images and the current solution data of each of the N images are transmitted to the ground solution server via a Universal Serial Bus (USB) connecting the mobile device and the ground solution server.

[0053] For the ground solution server, the data processing method provided by the embodiment of the present disclosure includes the following steps.

[0054] Step S204: Receive N images captured by the mobile device and current solution data of each of the N images.

[0055] Exemplarily, images are processed in real time after capture, and the image's current processed data is determined based on the image's completed processing. More specifically, the image's current processed data is determined based on the image's completed processing at a preset time point. In one example, the preset time point is the time point at which the digital surface model processing has completed for the latest image in the N images. In another example, the preset time point is the time point at which the mobile device has returned from completing a mission and docked at a docking point.

[0056] Step S205 : Based on the current solved data of each of the N images, continue to solve the N images to obtain digital orthoimages corresponding to each of the N images.

[0057] Specifically, the disclosed embodiment can skip the already completed solution process for each of the N images based on the current solution data of each of the N images and continue to solve the N images. Since there is no need to repeat the solution process that has already been completed on the mobile device, the task pressure on the subsequent solution platform can be effectively reduced.

[0058] In an embodiment of the present application, the aerial survey data is solved in real time by the mobile device after being collected by the mobile device, and the current solution data of the image can be determined based on the completed solution process of the image, which can provide a basis for subsequent continued solution. After receiving N images collected by the mobile device and the current solution data of each of the N images, the ground solution server continues to solve the N images based on the current solution data of each of the N images to obtain the target surveying and mapping data corresponding to each of the N images. This achieves the purpose of executing part of the solution task during the operation of the mobile device, making full use of the computing power resources during the operation of the mobile device, reducing the waste of computing power resources during the operation of the mobile device, improving the drawing efficiency, reducing the waiting time, and effectively reducing the task pressure of the post-learning solution platform.

[0059] The following combination Figure 3 The specific implementation method of continuing to solve N images based on the current solved data of each of the N images to obtain the digital orthophoto images corresponding to each of the N images is described in detail.

[0060] Figure 3 FIG. 1 is a flow chart of an embodiment of the present disclosure for continuing to solve N images based on the current solved data of each of the N images to obtain digital orthophoto images corresponding to each of the N images. Figure 3 As shown, based on the current solved data of each of the N images, the steps of continuing to solve the N images to obtain the target mapping data corresponding to each of the N images include the following steps.

[0061] Step S301 : determining the remaining solving process corresponding to each of the N images based on the current solving data of each of the N images.

[0062] For example, since the current image solution data is determined based on the image solution process that has been completed at a preset time point, the type of image result data obtained varies depending on the image solution process that has been completed. For example, if an image has completed the digital surface model solution process, the image result data type corresponding to the current image is a digital surface model solution image. If an image has completed the digital orthophoto image solution process, the image result data type corresponding to the current image is a digital orthophoto image.

[0063] Since the current solution data includes different types of image result data, the above-mentioned determination of the remaining solution processes corresponding to each of the N images based on the current solution data of each of the N images can be implemented as follows: determining the current solution process corresponding to each of the N images based on the type of the image result data corresponding to each of the N images; and determining the remaining solution process corresponding to each of the N images based on the current solution process corresponding to each of the N images.

[0064] In some embodiments, if the type of the image result data corresponding to the image is a digital surface model image, then the current solution process of the image is determined to have completed the digital surface model solution process; if the type of the image result data corresponding to the image is a digital orthoimage, then the current solution process of the image is determined to have completed all solution processes.

[0065] Step S302 : Based on the current solution data of each of the N images, the remaining solution processes corresponding to each of the N images are executed to obtain digital orthoimages corresponding to each of the N images.

[0066] Based on the above description, it is known that the digital surface model solution process is the process of solving an image into the digital surface model image corresponding to the image, that is, the process of extracting the sparse point cloud and pose information corresponding to the image, and then obtaining the digital surface model image corresponding to the image through methods such as inverse distance weighting or triangulation based on the obtained sparse point cloud. Since the sparse point cloud and pose information are easy to obtain, the time required to obtain the digital surface model image is relatively short. Therefore, the digital surface model solution process is relatively time-consuming. During the operation of the mobile device, each of the N images can obtain their own digital surface model image. In contrast, the process of solving the digital surface model image into a digital orthophoto image is more time-consuming. In some cases, the digital orthophoto image cannot be obtained from M images, and the solution needs to be continued, where M is a positive integer less than N.

[0067] Exemplarily, the specific implementation method of the above-mentioned step S302 can be: if there are M images among the N images, and the remaining solution processes corresponding to each of them are digital orthophoto image solution processes, then for each image in the M images, based on the pose data and digital surface model image in the current solution data of the image, an orthorectification operation is performed to obtain the digital orthophoto image corresponding to the image.

[0068] In an embodiment of the present application, based on the current solution data of each of the N images, the remaining solution processes corresponding to each of the N images are determined, so as to skip the completed solution processes of each of the N images and continue to solve the N images, thereby effectively reducing the task pressure of the subsequent solution platform.

[0069] Figure 4 The figure shows a schematic diagram of the structure of an image processing device provided by an embodiment of the present application. The image processing device is applied to a ground solution server, such as Figure 4 As shown, the image processing device 100 includes a receiving module 101 and a solving module 102 .

[0070] In this embodiment of the present application, the receiving module 101 is configured to receive current solution data for each of N images captured by the mobile device and N images to be solved. The current solution data for the image to be solved is determined based on the solution process already completed for the image to be solved in the mobile device, where N is a positive integer. The solving module 102 is configured to continue solving the N images based on the current solution data for each of the N images to obtain target mapping data corresponding to each of the N images.

[0071] In some embodiments, the solution module 102 is further configured to determine the remaining solution process corresponding to each of the N images based on the current solution data of each of the N images; and execute the remaining solution process corresponding to each of the N images based on the current solution data of each of the N images to obtain the target mapping data corresponding to each of the N images.

[0072] In some embodiments, the current solution data includes different types of image result data, and the solution module 102 is further configured to determine the current solution process corresponding to each of the N images based on the type of image result data corresponding to each of the N images; and determine the remaining solution process corresponding to each of the N images based on the current solution process corresponding to each of the N images.

[0073] In some embodiments, the solution module 102 is further configured to, if the type of the image result data corresponding to the image is a digital surface model image, determine that the current solution process of the image has completed the digital surface model solution process; if the type of the image result data corresponding to the image is a digital orthoimage, determine that the current solution process of the image has completed all solution processes.

[0074] In some embodiments, the solution module 102 is further configured to perform an orthorectification operation on each of the M images if the remaining solution processes corresponding to each of the M images among the N images are digital orthoimage solution processes, based on the pose data and the digital surface model image in the current solution data of the image, to obtain a digital orthoimage corresponding to the image, where M is a positive integer less than N.

[0075] Figure 5 FIG. 1 is a schematic diagram of the structure of an image processing device provided by another embodiment of the present application. The image processing device is applied to a mobile device. Figure 5 As shown, the image processing device 200 includes an acquisition module 201 , a processing module 202 and a storage module 203 .

[0076] In this embodiment of the present application, the acquisition module 201 is configured to acquire N images, where N is a positive integer. The processing module 202 is configured to perform real-time image processing on the images after they are acquired and determine the current image processing data based on the image processing progress completed. The storage module 203 is configured to store the N images and the current image processing data for each of the N images.

[0077] In some embodiments, the processing module 202 is further configured to, at a preset time point, use all data generated in the completed solution process of the image as the current solution data; wherein the preset time point is the time point when the latest image in the N images has completed the digital surface model solution process, or the time point when the movable device completes the task and returns to the docking site.

[0078] The specific functions and operations of the above-mentioned image processing device have been Figure 2 and Figure 3 The image processing method part shown in FIG is introduced in detail, so its repeated description will be omitted here.

[0079] Figure 6 Shown is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. Figure 6 The electronic device 300 shown (the electronic device 300 may be a computer device) includes a memory 301, a processor 302, a communication interface 303, and a bus 304. The memory 301, the processor 302, and the communication interface 303 are connected to each other via the bus 304.

[0080] Memory 301 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). Memory 301 can store programs. When the program stored in memory 301 is executed by processor 302, processor 302 and communication interface 303 are used to perform the various steps of the image processing method of the embodiment of the present disclosure.

[0081] The processor 302 can adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the functions required to be performed by the ground solution server or the unit in the mobile device of the embodiment of the present disclosure.

[0082] The processor 302 may also be an integrated circuit chip with signal processing capabilities. During implementation, the various steps of the image processing method disclosed herein may be performed by hardware integrated logic circuits or software instructions in the processor 302. The aforementioned processor 302 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure may be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or the like. The storage medium is located in the memory 301, and the processor 302 reads the information in the memory 301 and combines its hardware to complete the functions required to be performed by the units included in the ground solution server or mobile device of the embodiment of the present disclosure, or executes the image processing method of the embodiment of the method of the present disclosure.

[0083] The communication interface 303 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the electronic device 300 and other devices or a communication network. For example, sensor data can be obtained through the communication interface 303.

[0084] The bus 304 may include a path for transmitting information between various components of the electronic device 300 (eg, the memory 301 , the processor 302 , and the communication interface 303 ).

[0085] It should be noted that although Figure 6 The electronic device 300 shown only shows a memory, a processor, and a communication interface. However, in the specific implementation process, those skilled in the art should understand that the electronic device 300 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the electronic device 300 may also include hardware devices that implement other additional functions. In addition, those skilled in the art should understand that the electronic device 300 may also include only the devices necessary to implement the embodiments of the present disclosure, and does not necessarily include Figure 6 All devices shown in .

[0086] In addition to the above-mentioned methods, apparatuses and devices, embodiments of the present disclosure may also be computer program products, which include computer program instructions, which, when executed by a processor, enable the processor to perform the various steps of the image processing methods provided by various embodiments of the present disclosure.

[0087] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0088] In addition, the embodiments of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the various steps of the image processing method provided by the various embodiments of the present disclosure.

[0089] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0090] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0091] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0092] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0093] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a similar region segmentation unit, each unit may exist physically separately, or two or more units may be integrated into one unit.

[0095] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0096] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An image processing method, characterized in that: Applied to a ground solution server, the method includes: Receiving N images captured by a mobile device and current solution data of each of the N images, wherein the current solution data of the image is determined based on a solution process of the image that has been completed in the mobile device, and N is a positive integer; Determining, based on current solution data of each of the N images, a remaining solution process corresponding to each of the N images; Based on the current solution data of each of the N images, the remaining solution processes corresponding to each of the N images are executed to obtain the target mapping data corresponding to each of the N images.

2. The method according to claim 1, characterized in that The current solution data includes different types of image result data; The determining of the remaining solution processes corresponding to the N images based on the current solution data of the N images includes: Determining a current solution process corresponding to each of the N images based on a type of image result data corresponding to each of the N images; Based on the current solving processes corresponding to the N images, the remaining solving processes corresponding to the N images are determined.

3. The method according to claim 2, characterized in that The determining, based on the type of the image result data corresponding to each of the N images, the current solution process corresponding to each of the N images includes: If the type of the image result data corresponding to the image is a digital surface model image, determining that the current solution process of the image is a completed digital surface model solution process; If the type of the image result data corresponding to the image is a digital orthoimage, it is determined that the current solution process of the image has completed all solution processes.

4. The method according to any one of claims 1 to 3, characterized in that The step of executing the remaining solution processes corresponding to the N images based on the current solution data of each of the N images to obtain the target mapping data corresponding to each of the N images includes: If the remaining solution processes corresponding to M images among the N images are digital orthoimage solution processes, then for each image in the M images, an orthorectification operation is performed based on the pose data and the digital surface model image in the current solution data of the image to obtain a digital orthoimage corresponding to the image, where M is a positive integer less than N.

5. An image processing method, characterized in that: Applied to a mobile device, the mobile device includes a collection module and a processing module, and the method includes: Obtain N images collected by the acquisition module, where N is a positive integer; After the image is captured, the processing module performs real-time processing on the image and determines current processing data of the image based on the completed processing of the image; The N images and their respective current solution data are stored and transmitted to a ground solution server, which performs the following steps: Determining, based on current solution data of each of the N images, a remaining solution process corresponding to each of the N images; Based on the current solution data of each of the N images, the remaining solution processes corresponding to each of the N images are executed to obtain the target mapping data corresponding to each of the N images.

6. The method according to claim 5, characterized in that The determining of current solution data of the image based on the completed solution process of the image includes: At a preset time point, all data generated in the completed image solving process is used as the current solved data; The preset time point is the time point at which the latest image in the N images has completed the digital surface model solution process, or the time point at which the movable device completes the task and returns to the docking site.

7. An image processing device, characterized in that: Applied to a ground solution server, the device includes: a receiving module configured to receive N images captured by a mobile device and current solution data of each of the N images, wherein the current solution data of the image is determined based on a solution process of the image that has been completed in the mobile device, and N is a positive integer; The solution module is configured to determine the remaining solution process corresponding to each of the N images based on the current solution data of each of the N images; and execute the remaining solution process corresponding to each of the N images based on the current solution data of each of the N images to obtain the target mapping data corresponding to each of the N images.

8. An image processing device, characterized in that: Applied to a mobile device, the device comprises: an acquisition module configured to acquire N images, where N is a positive integer; a processing module configured to, after the image is captured, perform real-time processing on the image and determine current processing data of the image based on a completed processing process of the image; The storage module is configured to store the N images and the current solution data of each of the N images, and transmit them to a ground solution server, whereby the ground solution server performs the following steps: determining, based on the current solution data of each of the N images, a remaining solution process corresponding to each of the N images; and executing, based on the current solution data of each of the N images, a remaining solution process corresponding to each of the N images to obtain target mapping data corresponding to each of the N images.

9. An electronic device comprising: processor; as well as A memory having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 6. 10 . A computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the processor is caused to perform the method according to claim 1 .

Images