Image stitching method, electronic device, computer-readable storage medium and camera

Abstract

The present application discloses an image stitching method, an electronic device, a computer-readable storage medium and a camera. The image stitching method is applied to a camera, and the camera includes a focusing unit. The image stitching method includes: using the focusing unit to focus on a part of a target area according to a focusing parameter within a preset time interval, and collecting multiple images to be stitched corresponding to the target area at a preset time interval; based on the focusing parameter, obtaining the area to be stitched from the multiple images to be stitched; wherein the multiple areas to be stitched correspond to concentric circles matching the focusing parameter; based on the multiple areas to be stitched, obtaining a target image corresponding to the target area. The above scheme can achieve a global high-quality depth of field of the target image.

Description

Technical Field

[0001] The present application relates to the technical field of cameras, and in particular to an image stitching method, an electronic device, a computer-readable storage medium and a camera. Background Art

[0002] With the rapid development of intelligent technology, cameras, as a key visual data entry point, have increasing requirements for all-weather data collection quality. In low-light environments, if full-color effects are required, the aperture of the lens used will become larger and larger, which will lead to the industry pain point of shallow depth of field of the camera. Therefore, how to achieve high-quality depth of field of the entire target image has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide an image stitching method, an electronic device, a computer-readable storage medium and a camera, which can achieve a full-range high-quality depth of field of a target image.

[0004] To solve the above technical problems, the first aspect of the present application provides an image stitching method, which is applied to a camera, wherein the camera includes a focusing unit, and the method includes: using the focusing unit to focus on a partial area within a target area according to a focusing parameter within a preset time interval, and collecting multiple images to be stitched corresponding to the target area at the preset time interval; based on the focusing parameter, obtaining a to-be-stitched area from the multiple images to be stitched; wherein the multiple to-be-stitched areas correspond to concentric circles matching the focusing parameter; based on the multiple to-be-stitched areas, obtaining a target image corresponding to the target area.

[0005] To solve the above technical problem, the second aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] To solve the above technical problems, the fourth aspect of the present application provides a camera, comprising a lens, an image sensor and a focusing unit, and an electronic device as described in the second aspect above, and the electronic device is coupled to the image sensor and the focusing unit.

[0008] The above scheme utilizes the focusing unit of the camera to focus on a partial area within the target area multiple times according to the focusing parameters within a preset time interval, and collects multiple images to be stitched corresponding to the focused target area according to the preset time interval, obtains the area to be stitched from the multiple images to be stitched collected by the camera based on the focusing parameters, and each area to be stitched corresponds to a concentric circle matching the focusing parameters, processes the multiple areas to be stitched, and obtains the target image corresponding to the target area, and utilizes the change of the focusing parameters to form different depth of field effects in the target area, so that the final image is visually more layered and three-dimensional, thereby achieving a global high-quality depth of field of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0010] Figure 1 It is a flowchart of an implementation method of the image stitching method of the present application;

[0011] Figure 2 It is a stitching schematic diagram of an implementation method of the image stitching method of the present application;

[0012] Figure 3 It is a flowchart of another implementation method of the image stitching method of the present application;

[0013] Figure 4 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0014] Figure 5 It is a schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present application;

[0015] Figure 6 It is a structural schematic diagram of an embodiment of a camera of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of the image stitching method of the present application. The method is applied to a camera, the camera includes a focusing unit, wherein the image stitching method includes:

[0018] S101: using a focusing unit to focus on a partial area within a target area according to a focusing parameter within a preset time interval, and collecting a plurality of to-be-joined images corresponding to the target area at the preset time interval.

[0019] Specifically, a focus unit of the camera is used to focus on a partial area in the target area multiple times according to a focus parameter within a preset time interval, and a plurality of to-be-stitched images corresponding to the focused target area are acquired according to the preset time interval.

[0020] It is understandable that the number of images to be stitched is not less than two, the focus parameter is preset, and the focus parameter corresponds to the number of images to be stitched.

[0021] In one application, the focusing unit of the camera is an automatic focusing system, which uses the automatic focusing system to focus on a partial area in the target area multiple times according to focusing parameters within a preset time interval, and collects multiple images to be stitched corresponding to the target area according to the preset time interval.

[0022] In another application, the focusing unit of the camera is a phase compensation switching device, which is used to switch the phase compensator within a preset time interval to focus on a partial area within the target area multiple times according to the focusing parameters, and to capture multiple images to be stitched corresponding to the target area according to the preset time interval.

[0023] In some implementation scenarios, the preset time interval is 80 ms. In other implementation scenarios, the preset time interval can be set according to actual conditions, and this application does not impose any specific restrictions on this.

[0024] In some implementation scenarios, the focus parameters are set based on the shooting height, blind spot distance, focal length and number of images to be stitched of the camera, and the focus parameters correspond to the object distance of a part of the target area.

[0025] In some implementation scenarios, the focus parameters are set based on the camera's orientation angle, aperture parameters, focal length, and the number of images to be stitched, and the focus parameters correspond to the object distance and image distance of a portion of the target area.

[0026] S102: Based on the focus parameter, a region to be stitched is obtained from a plurality of images to be stitched; wherein the plurality of regions to be stitched correspond to concentric circles that match the focus parameter.

[0027] Specifically, based on the focusing parameter, the area to be stitched is obtained from a plurality of images to be stitched acquired by the camera, and each area to be stitched corresponds to a concentric circle matching the focusing parameter.

[0028] In one application, a plurality of matching concentric circles are determined based on the focus parameter, and corresponding to-be-stitched areas are acquired from a plurality of to-be-stitched images captured by the camera based on the plurality of concentric circles.

[0029] In another application, multiple matching concentric circles are determined based on focusing parameters, and feature points in multiple images to be stitched are obtained. Based on the multiple concentric circles and the matching feature points, corresponding areas to be stitched are obtained from multiple images to be stitched captured by the camera.

[0030] In some implementation scenarios, concentric circles are set on the image to be stitched and the area outside the image based on the focus parameter, wherein the concentric circles correspond to reference arcs on the image to be stitched, and the image to be stitched is divided based on the reference arcs to obtain the area to be stitched extracted from each image to be stitched, wherein the edges of adjacent areas to be stitched are corresponding reference arcs.

[0031] S103: Based on the multiple areas to be stitched, a target image corresponding to the target area is obtained.

[0032] Specifically, a plurality of areas to be stitched are processed to obtain a target image corresponding to the target area.

[0033] In one application, feature points are extracted from a plurality of regions to be spliced, and the plurality of regions to be spliced ​​are spliced ​​according to the extracted feature points to obtain a target image corresponding to the target region.

[0034] In one application scenario, feature points are extracted from multiple areas to be stitched, and matching feature point pairs are found using a feature matching algorithm. The transformation relationship between the areas to be stitched is obtained based on the matched feature point pairs, and the multiple areas to be stitched are stitched based on the obtained transformation relationship to obtain a target image corresponding to the target area.

[0035] In another application mode, multiple areas to be stitched are fused and stitched using image fusion technology to obtain a target image corresponding to the target area.

[0036] The above scheme utilizes the focusing unit of the camera to focus on a partial area within the target area multiple times according to the focusing parameters within a preset time interval, and collects multiple images to be stitched corresponding to the focused target area according to the preset time interval, obtains the area to be stitched from the multiple images to be stitched collected by the camera based on the focusing parameters, and each area to be stitched corresponds to a concentric circle matching the focusing parameters, processes the multiple areas to be stitched, and obtains the target image corresponding to the target area, and utilizes the change of the focusing parameters to form different depth of field effects in the target area, so that the final image is visually more layered and three-dimensional, thereby achieving a global high-quality depth of field of the target image.

[0037] In one embodiment, the camera further includes an image sensor, the radius of each concentric circle includes a first distance and a second distance, the first distance is located outside the image sensor, and the second distance is located inside the image sensor, and the focusing parameters include the first distance and the second distance corresponding to each concentric circle.

[0038] Specifically, the radius of each concentric circle includes a first distance outside the image sensor and a second distance inside the image sensor. By setting the focusing parameters of the first distance and the second distance, the camera can accurately focus on targets at different depths, thereby improving the clarity of the image. By utilizing the second distance corresponding to the concentric circles, the depth of field range can be increased during shooting, so that both the foreground and the background can remain clear.

[0039] In one implementation scenario, the focusing parameters are obtained based on the following steps: obtaining the shooting height, blind spot distance and focal length of the camera, and determining the first distance based on the shooting height, blind spot distance and focal length; determining the second distance corresponding to each concentric circle based on the number of areas to be spliced ​​and the first distance.

[0040] Specifically, the shooting height, blind spot distance and focal length used for shooting of the camera are obtained, and the first distance is determined based on the shooting height, blind spot distance and angle used for shooting.

[0041] In a specific implementation scenario, the shooting height of the camera is M, the blind spot distance is S, and the focal length used for shooting is f, then the first distance L1 needs to satisfy 0.8S×f / M≤L1≤1.2S×f / M.

[0042] Furthermore, based on the number of areas to be spliced ​​and the first distance, the second distance corresponding to each concentric circle is determined. For example, when the number of areas to be spliced ​​is 2, the number of concentric circles is also 2, then the second distance corresponding to the first concentric circle needs to be no more than the difference between the radius of the concentric circle and the first distance, and the second distance corresponding to the second concentric circle needs to be less than the difference between the radius of the concentric circle and the first distance. When the number of areas to be spliced ​​is 3, the number of concentric circles is also 3, then the second distance corresponding to the first concentric circle needs to be no more than the difference between the radius of the concentric circle and the first distance, the second distance corresponding to the second concentric circle needs to be no more than the difference between the radius of the concentric circle and the first distance, the second distance corresponding to the third concentric circle needs to be less than the difference between the radius of the concentric circle and the first distance, and so on.

[0043] In one implementation scenario, the image to be stitched and the target image have the same image size, wherein the image size includes image width and image height, and the second distances corresponding to the other concentric circles other than the concentric circle with the largest second distance are smaller than the image height.

[0044] Specifically, each image to be stitched and the final stitched target image have the same image width and image height, and the second distances corresponding to the other concentric circles other than the concentric circle with the largest second distance are smaller than the image height to ensure the clarity of the image in the acquired area to be stitched.

[0045] In a specific implementation scenario, please refer to Figure 2 , Figure 2 It is a stitching schematic diagram of an embodiment of the image stitching method of the present application, specifically, it is a schematic diagram of using three images to be stitched to form a target image, the image width of the target image is H, the image height is V, and correspondingly, there are three concentric circles in total, the radius of each concentric circle includes a first distance L1 and a second distance (not shown in the figure), the first distance L1 is outside the image sensor, and the second distance is inside the image sensor, and the first distance L1 of each concentric circle is fixed, and the second distances are different, but it is necessary to satisfy that the second distances corresponding to other concentric circles other than the concentric circle with the largest second distance are less than the image height V, and the second distance corresponding to the first concentric circle needs to not exceed the difference between the radius of the concentric circle and the first distance L1, the second distance corresponding to the second concentric circle needs to not exceed the difference between the radius of the concentric circle and the first distance L1, and the second distance corresponding to the third concentric circle needs to be less than the difference between the radius of the concentric circle and the first distance L1.

[0046] See also Figure 3 , Figure 3 1 is a flow chart of another embodiment of the image stitching method of the present application. The method is applied to a camera, the camera includes a focusing unit, wherein the image stitching method includes:

[0047] S301: using a focusing unit to collect an image to be stitched in an initial state corresponding to a target area according to a focusing parameter.

[0048] Specifically, the image to be stitched in the initial state corresponding to the target area is collected by using the focusing unit of the camera according to the focusing parameters.

[0049] In one implementation scenario, the initial state corresponding to the target area is a close-up of the target area or a distant view of the target area. When it is necessary to capture an image, the focusing unit of the camera needs to be used to adjust to the close-up of the target area and then start capturing, or to adjust to the distant view of the target area and then start capturing, thereby ensuring the continuity of image data capture and facilitating subsequent stitching.

[0050] In another implementation scenario, the initial state corresponding to the target area is the area position that was most recently adjusted using the camera's focus unit. When it is necessary to capture an image, capture starts directly from the area position, thereby enabling a quick response and saving image capture time.

[0051] S302: within a preset time interval, refocusing a part of the target area according to the focusing parameters, and acquiring the image to be stitched corresponding to the target area after the focusing is completed, until all areas in the target area are focused according to the focusing parameters, thereby obtaining a plurality of images to be stitched corresponding to the target area.

[0052] Specifically, a focusing unit is used to refocus a part of the target area according to the focusing parameters within a preset time interval, and the image to be stitched is captured after the focusing is completed, until all areas within the target area are focused according to the focusing parameters and captured by the camera, a plurality of clear images to be stitched corresponding to the target area are obtained, thereby improving the quality of the target image obtained by subsequent stitching and achieving a global depth of field effect of the target image.

[0053] Optionally, after focusing is completed, the target area may be captured multiple times, and the best quality images to be stitched may be screened out to improve the clarity of the target image subsequently stitched.

[0054] S303: Based on the focusing parameters, determine the concentric circles that match the initial state and the focusing after each preset time interval.

[0055] Specifically, based on the focusing parameters, the corresponding concentric circles in the initial state and the concentric circles matched when focusing after each preset time interval are determined.

[0056] S304: Based on the concentric circles matched during focusing, an area to be stitched is obtained from the plurality of images to be stitched.

[0057] Based on the concentric circles matched during focusing, the area to be stitched for subsequent stitching is obtained from multiple images to be stitched. By matching the concentric circles, the relative positions between the image areas to be stitched can be accurately determined, thereby improving the accuracy of subsequent stitching.

[0058] S305: Based on the multiple areas to be stitched, a target image corresponding to the target area is obtained.

[0059] Specifically, a plurality of areas to be stitched are processed to obtain a target image corresponding to the target area.

[0060] In one implementation scenario, step S305 specifically includes: in response to the fact that none of the areas to be stitched include a moving target, fusing multiple areas to be stitched together to obtain a target image; in response to the fact that at least one area to be stitched together includes a moving target, obtaining a target image based on the states of all areas to be stitched together and the moving target.

[0061] Specifically, when there is no moving target in all the areas to be stitched, multiple areas to be stitched are directly fused to obtain a target image; when at least one area to be stitched includes a moving target, the target image is obtained based on the states of all the areas to be stitched and the moving targets.

[0062] Optionally, the moving target and the background of the area to be spliced ​​may be processed separately. First, the moving target in the area to be spliced ​​is tracked using a target recognition algorithm, and the state of the moving target is obtained. Matching feature points are obtained according to the state of the moving target. The moving targets are spliced ​​and fused according to these feature points to obtain a complete moving target. The area to be spliced ​​including the moving target is subjected to moving target removal processing. All areas to be spliced ​​without the moving target are spliced ​​and fused. Then, the previously spliced ​​moving targets are added at corresponding positions to obtain a complete target image.

[0063] In this embodiment, a focusing unit is used to refocus a part of the target area according to the focusing parameters within a preset time interval, and the image to be spliced ​​is collected after the focusing is completed, until all areas in the target area are focused according to the focusing parameters and collected by the camera, so as to obtain multiple clear images to be spliced ​​corresponding to the target area, thereby improving the quality of the target image obtained by subsequent splicing and achieving the global depth of field effect of the target image. Based on the focusing parameters, the corresponding concentric circles in the initial state and the concentric circles matched when focusing after each preset time interval are determined. By matching the concentric circles, the relative positions between the image areas to be spliced ​​can be accurately determined, and multiple areas to be spliced ​​are processed to obtain the target image corresponding to the target area, thereby improving the accuracy of splicing.

[0064] See also Figure 4 , Figure 44 is a schematic diagram of the structure of an embodiment of an electronic device of the present application, wherein the electronic device 40 comprises a memory 400 and a processor 402 coupled to each other, wherein the memory 400 stores program instructions (not shown), and the processor 402 is used to execute the program instructions to implement the method in any of the above embodiments, and the description of the relevant contents can be found in the detailed description of the above method embodiments, which will not be repeated here. In addition, the processor 402 can also be referred to as a CPU (Center Processing Unit). The processor 402 may be an integrated circuit chip with signal processing capabilities. The processor 402 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 402 may be implemented by an integrated circuit chip.

[0065] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 50 stores program data 500. When the program data 500 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0066] See also Figure 6 , Figure 6 It is a structural diagram of an embodiment of a camera of the present application. The camera 60 includes a lens 600, an image sensor 601 and a focusing unit 602, as well as the electronic device 40 described above, and the electronic device 40 is coupled to the image sensor 601 and the focusing unit 602.

[0067] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0068] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0069] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., and other media that can store program codes.

[0070] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image stitching method, characterized in that: Applied to a camera, the camera includes a focusing unit and an image sensor, and the method includes: Using the focusing unit to focus on a partial area within the target area according to the focusing parameters within a preset time interval, and collecting a plurality of images to be stitched corresponding to the target area at the preset time interval; Based on the focusing parameters, an area to be stitched is obtained from a plurality of images to be stitched; wherein, a plurality of the areas to be stitched correspond to concentric circles matching the focusing parameters, the radius of each of the concentric circles includes a first distance and a second distance, the first distance is located outside the image sensor, and the second distance is located inside the image sensor, and the focusing parameters include the first distance and the second distance corresponding to each of the concentric circles; wherein the first distance of each of the concentric circles is fixed, and the second distances are different, the focusing parameters correspond to the object distance of a part of the target area, and the focusing parameters are obtained based on the following steps: obtaining the shooting height, blind spot distance and focal length of the camera, and determining the first distance based on the shooting height, the blind spot distance and the focal length; determining the second distance corresponding to each concentric circle based on the number of the areas to be stitched and the first distance; Based on the multiple areas to be stitched, a target image corresponding to the target area is obtained.

2. The image stitching method according to claim 1, characterized in that: The image to be stitched and the target image have the same image size, wherein the image size includes image width and image height, and the second distances corresponding to other concentric circles other than the concentric circle with the largest second distance are smaller than the image height; wherein the focusing parameter corresponds to the object distance of a partial area within the target area.

3. The image stitching method according to claim 1, characterized in that: The focusing unit is used to focus on a partial area within the target area according to the focusing parameters within a preset time interval, and a plurality of images to be stitched corresponding to the target area are collected at the preset time interval, including: Using the focusing unit to collect the image to be stitched in the initial state corresponding to the target area according to the focusing parameters; Within the preset time interval, a partial area within the target area is refocused according to the focusing parameters, and the image to be stitched corresponding to the target area is acquired after the focusing is completed, until all areas within the target area are focused according to the focusing parameters, thereby obtaining multiple images to be stitched corresponding to the target area.

4. The image stitching method according to claim 3, characterized in that: The step of obtaining a to-be-stitched area from the plurality of to-be-stitched images based on the focusing parameter comprises: Based on the focusing parameters, determining the initial state and the concentric circles that match when focusing after each preset time interval; Based on the concentric circles matched during focusing, the area to be stitched is acquired from the plurality of images to be stitched.

5. The method according to claim 1, characterized in that The step of obtaining a target image corresponding to the target area based on the plurality of areas to be stitched includes: In response to the fact that none of the regions to be stitched include a moving target, a plurality of the regions to be stitched are merged to obtain the target image; In response to at least one of the regions to be stitched including the moving target, the target image is obtained based on states of all the regions to be stitched and the moving target.

6. An electronic device, characterized in that: The invention comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the image stitching method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the image stitching method according to any one of claims 1 to 5.

8. A camera, characterized in that: The electronic device comprises a lens, an image sensor and a focusing unit, and the electronic device as claimed in claim 6, wherein the electronic device is coupled to the image sensor and the focusing unit.

Images