Image completion method, system and device and computer readable storage medium

Abstract

The invention discloses an image completion method, system and device and a computer readable storage medium. The method comprises the steps that an original image is acquired, and the original image is an image shot by a fisheye camera; determining an imaging area and a non-imaging area in the original image; an expanded image is obtained, and the expanded image is an image comprising an edge part picture of the imaging area and a picture corresponding to the non-imaging area; determining an imaging image corresponding to the non-imaging area according to the original image and the extended image; and complementing the non-imaging area in the original image according to the imaging image corresponding to the non-imaging area to obtain a target image. The fisheye image filling method and device solve the technical problems that when the black area in the fisheye image is filled in the related technology, the correlation between the filling content and the fisheye image is low, and the impressions are poor.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image completion method, system, device and computer-readable storage medium. Background Art

[0002] The fisheye camera is used with a fisheye lens to form an effective circular image. The image or video captured by the sensor and encoder is finally formed into a rectangle with a certain aspect ratio. Therefore, many common fisheye cameras have invalid shadow areas in their final output field of view.

[0003] The common methods for removing shadows are as follows: 1. direct cropping, which has the disadvantage of greatly losing the field of view of the picture; 2. distortion correction, which usually loses picture information after correction and changes the image.

[0004] In order to solve the above problems, the prior art proposes a method of first adjusting the brightness value of the black area to make the edge of the area transition smoothly, and then filling the black area by LSC compensation. This method fills the original black area more softly and smoothly than the original image, but contains less information. The picture outside the effective imaging circle cannot be associated with the information inside the effective imaging circle. When filling the black area in the fisheye image, the correlation between the filling content and the fisheye image itself is low, which can easily cause technical problems of poor viewing experience.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] Embodiments of the present invention provide an image completion method, system, device and computer-readable storage medium to at least solve the technical problem that when the related technology fills the black area in the fisheye image, the correlation between the filling content and the fisheye image itself is low, which easily causes poor viewing experience.

[0007] According to one aspect of an embodiment of the present invention, there is provided an image completion method, which comprises acquiring an original image, wherein the original image is an image captured by a fisheye camera; determining an imaging area and a non-imaging area in the original image; acquiring an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; determining an imaging image corresponding to the non-imaging area based on the original image and the extended image; and completing the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a target image.

[0008] Optionally, determining the imaging image corresponding to the non-imaging area based on the original image and the extended image includes: determining the original feature points of the edge portion of the imaging area in the original image; determining the extended feature points in the extended image corresponding to the original feature points, wherein a similarity index between the extended feature points and the original feature points exceeds a predetermined threshold; and determining the imaging image corresponding to the non-imaging area based on the extended feature points.

[0009] Optionally, determining the extended feature point in the extended image corresponding to the original feature point includes: determining a plurality of candidate feature points in the extended image corresponding to the original feature point, wherein similarity indexes corresponding to the plurality of candidate feature points and the original feature point respectively exceed a predetermined threshold; determining matching errors corresponding to the plurality of candidate feature points respectively; and determining, from the plurality of candidate feature points, a feature point having the smallest corresponding matching error as the extended feature point.

[0010] Optionally, acquiring the extended image includes: when the fisheye camera is a rotatable camera for shooting, determining an extended shooting position corresponding to the fisheye camera; controlling the fisheye camera to rotate to the extended shooting position for shooting, so as to obtain the extended image.

[0011] Optionally, obtaining the extended image includes: inputting the original image into an image extension model to obtain the extended image, wherein the image extension model is obtained by training with sample data, and the sample data includes a sample image and a sample extended image corresponding to the sample image.

[0012] Optionally, determining the imaging image corresponding to the non-imaging area based on the original image and the extended image includes: determining an image type of the extended image, wherein the image type includes at least a fisheye image type; determining a distortion adjustment method corresponding to the image type; based on the original image, adjusting the extended image using the distortion adjustment method to obtain an adjusted extended image; and determining the imaging image based on the original image and the adjusted extended image.

[0013] Optionally, the method of completing the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a target image includes: completing the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a completed image; adjusting image parameters between the imaging area and the non-imaging area in the completed image to obtain the target image, so as to achieve a smooth transition between the imaging area and the non-imaging area in the target image.

[0014] According to one aspect of an embodiment of the present invention, there is provided an image completion system, comprising: a fisheye camera, and a controller, wherein the fisheye camera is connected to the controller, and the controller is used to obtain an original image, wherein the original image is an image captured by the fisheye camera; determine an imaging area and a non-imaging area in the original image; obtain an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; determine an imaging image corresponding to the non-imaging area based on the original image and the extended image; and complete the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a target image.

[0015] Optionally, the system further includes: other cameras, wherein the other cameras are connected to the controller, and the other cameras are used to capture the extended image.

[0016] According to one aspect of an embodiment of the present invention, there is provided an image completion device, comprising a first acquisition module for acquiring an original image, wherein the original image is an image captured by a fisheye camera; a first determination module for determining an imaging area and a non-imaging area in the original image; a second acquisition module for acquiring an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; a second determination module for determining an imaging image corresponding to the non-imaging area based on the original image and the extended image; and a completion module for completing the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a target image.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned image completion methods.

[0018] According to another aspect of the present invention, an electronic device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein any one of the above-mentioned image completion methods is executed when the program is run.

[0019] According to yet another aspect of the present invention, a computer program product is provided, comprising a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned image completion methods are implemented.

[0020] In an embodiment of the present invention, an original image is obtained, wherein the original image is an image captured by a fisheye camera; an imaging area and a non-imaging area in the original image are determined; an extended image is obtained, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; an imaging image corresponding to the non-imaging area is determined based on the original image and the extended image; and the non-imaging area in the original image is completed based on the imaging image corresponding to the non-imaging area to obtain a target image. The embodiment of the present invention completes the black part (non-imaging area) of the original image by obtaining an extended image including an edge portion of the imaging area and an image corresponding to the non-imaging area, so that in the obtained target image, the filling content is related to the original image itself, thereby improving the user's perception, thereby solving the technical problem that when the related technology fills the black area in the fisheye image, the filling content has a low correlation with the fisheye image itself, which easily causes poor perception. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0022] Figure 1 is a flow chart of an image completion method according to an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a fisheye image provided by an optional embodiment of the present invention;

[0024] Figure 3 It is a simple model schematic diagram provided by an optional embodiment of the present invention;

[0025] Figure 4 is a flow chart of an image completion method provided by an optional embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of feature matching provided by an optional embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of target feature point selection provided by an optional implementation manner of the present invention;

[0028] Figure 7 A flowchart of an AI model provided in an optional embodiment of the present invention;

[0029] Figure 8 is a schematic diagram of a system provided by an optional embodiment of the present invention;

[0030] Fig. 9 is a structural block diagram of an image completion system according to an embodiment of the present invention;

[0031] Fig.10 is a structural block diagram of an image completion device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0035] LSC: Stands for Lens Shading Correction, which is lens shading correction or lens vignetting correction. This is a technology used to correct the uneven brightness of the image caused by the optical characteristics of the lens and image sensor. Specifically, due to the physical characteristics of the lens and image sensor, the center of the image tends to be brighter than the edge, forming the so-called "dark corner" or "vignetting" phenomenon. The goal of the LSC algorithm is to correct this brightness unevenness so that the image has the same brightness level throughout the field of view.

[0036] Fisheye camera: a camera designed with a fisheye lens. A fisheye lens is an ultra-wide-angle lens with a field of view far exceeding that of an ordinary lens, usually reaching a field of view of 180 to 270 degrees. The design of this lens is inspired by the visual effect produced by fish when observing the scenery on the water surface in the water, hence the name "fisheye lens".

[0037] Fisheye image: Fisheye image refers to the image taken through a fisheye lens.

[0038] It should be noted that the embodiments of the present invention can be used in scenes containing fisheye images to complete the fisheye images. Since fisheye cameras have the characteristics of ultra-wide-angle field of view, they can capture scenes wider than ordinary lenses and are suitable for shooting large-scale or panoramic images. They are often used in monitoring, panoramic photography, virtual reality (VR) and other fields, and are also often used in creative photography to produce unique visual effects.

[0039] The fisheye camera is used with a fisheye lens to form an effective circular image. The image or video captured by the sensor and encoder is finally formed into a rectangle with a certain aspect ratio. Therefore, many common fisheye cameras have invalid shadow areas in their final output field of view.

[0040] The commonly used methods for removing shadows are as follows: 1. Direct cropping, the disadvantage is a significant loss of the field of view of the picture; 2. Distortion correction, which usually results in a small loss of picture information after correction and changes the imaging picture.

[0041] In order to solve the above problems, a method of filling the screen is proposed in the prior art. The method of filling the screen includes first adjusting the brightness value of the black area to make the edge of the area transition smoothly, and then filling the black area by LSC compensation. This method fills the original black area more softly and smoothly than the original image, but contains less information. The picture outside the effective imaging circle cannot be associated with the information inside the effective imaging circle. When filling the black area in the fisheye image, the correlation between the filling content and the fisheye image itself is low, which can easily cause technical problems of poor viewing experience.

[0042] The embodiment of the present invention utilizes the internal characteristic information of the effective imaging circle and the real image input externally to supplement the black area, thereby maximizing the effective information of the black area to solve the above-mentioned problem. The embodiment of the present invention is introduced below.

[0043] According to an embodiment of the present invention, an embodiment of a method for image completion is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] Figure 1 is a flow chart of an image completion method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0045] Step S102, obtaining an original image, wherein the original image is an image captured by a fisheye camera;

[0046] In step S102 provided in the present application, an original image is obtained, and the original image may be image data that has not been processed or has only been minimally processed. This generally means that after the image data is acquired, except for possible basic pre-processing such as denoising and white balance adjustment, no post-processing such as cropping, scaling, color correction or filter application is performed.

[0047] The original image is taken by a fisheye camera. Since it is an original image taken by a fisheye camera, such an image not only has the ultra-wide-angle field of view and barrel distortion unique to fisheye lenses, but also retains the original information when the image was acquired. These original images can be used for distortion correction, completion and other processing to generate images that better meet specific application requirements.

[0048] Step S104, determining the imaging area and the non-imaging area in the original image;

[0049] In step S104 provided in the present application, the imaging area and the non-imaging area in the original image are determined.

[0050] Among them, the "imaging area" refers to the part of the scene in the image that is actually captured by the lens and clearly presented, while the "non-imaging area" refers to those areas that contain no valid information or only a small amount of blurred information due to lens characteristics, shooting environment or image processing process. For example, the surrounding areas of fisheye images, especially the shadow areas or black areas at the four corners, can serve as typical examples of non-imaging areas.

[0051] It should be noted that the black areas around the image are usually caused by the ultra-wide-angle characteristics of the fisheye lens. Since the viewing angle of the lens is much larger than that of a traditional camera lens, light cannot be effectively focused on the image sensor at the edge of the image, resulting in black or nearly black areas. These areas do not contain valid information.

[0052] For example:

[0053] In the scene of an office shot using a fisheye camera, the middle part of the original image shows part of the office, and the surrounding parts of the image are not imaged and may appear unshadowed or black in the image. If an image is formed, a wider area of ​​the office will be displayed.

[0054] At this time, the imaging area in the original image is the area in the middle portion showing part of the office, and the non-imaging area is the area around the original image that is not imaged.

[0055] Step S106, acquiring an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area;

[0056] In step S106 provided in the present application, an extended image is acquired, wherein the extended image is an image including an edge portion of the image area and an image corresponding to the non-imaging area.

[0057] The edge images of the imaging area refer to those images that are close to the boundary of the imaging area. They often contain transition information between the imaging area and the external non-imaging area, which is very useful for understanding the layout of the entire scene and performing subsequent image completion.

[0058] The picture corresponding to the non-imaging area generally refers to the actual display picture corresponding to the non-imaging area.

[0059] For example, in the above example of using a fisheye camera to shoot an office scene, the imaging area in the original image is the middle part that shows part of the office, and the unimaged area is the part of the original image that is not imaged on all sides. It can be displayed without shadow or black in the image. If imaged, it will show a larger area of ​​the office.

[0060] The extended image includes the edge portion of the image area, such as a small portion of the office image, and a larger area of ​​the office image that is not displayed in the original image.

[0061] Step S108, determining an imaging image corresponding to the non-imaging area according to the original image and the extended image;

[0062] In step S108 provided in the present application, it is described how to determine the imaging image corresponding to the non-imaging area based on the original image and the extended image.

[0063] Since the non-imaging area exceeds the original field of view of the lens, it does not contain information that can be directly imaged. This step "restores" or "reconstructs" the area that was not directly imaged due to lens design or shooting conditions into image content with practical significance in some way.

[0064] In this step, the imaging image corresponding to the non-imaging area is determined through the original image and the extended image to complete the original image. Since the extended image including the edge part of the imaging area and the corresponding part of the non-imaging area is obtained to complete the black part (non-imaging area) of the original image, the filling content in the obtained target image is related to the original image itself, so as to achieve the effect of improving the user's viewing experience.

[0065] Step S110 , based on the imaging image corresponding to the non-imaging area, the non-imaging area in the original image is completed to obtain a target image.

[0066] In step S110 provided in the present application, the non-imaging area in the original image is complemented based on the imaging image corresponding to the non-imaging area, thereby achieving the purpose of obtaining the target image.

[0067] In the steps provided in the present application, an original image is obtained, wherein the original image is an image taken by a fisheye camera; an imaging area and a non-imaging area in the original image are determined; an extended image is obtained, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; an imaging image corresponding to the non-imaging area is determined based on the original image and the extended image; and the non-imaging area in the original image is completed based on the imaging image corresponding to the non-imaging area to obtain a target image. The embodiment of the present invention completes the black part (non-imaging area) of the original image by obtaining an extended image including an edge portion of the imaging area and an image corresponding to the non-imaging area, so that the filling content in the obtained target image is related to the original image itself, thereby improving the user's perception, thereby solving the technical problem that when the related technology fills the black area in the fisheye image, the filling content has a low correlation with the fisheye image itself, which is easy to cause poor perception.

[0068] As an optional embodiment, the execution subject of the method of this embodiment can be a controller for completing the image, and the controller can be located on a terminal or a server. For example, when applied to a terminal for performing image processing, it can easily implement image processing in simple image scenes; for another example, when applied to a server, it can call on the server's rich computing resources, or a relatively larger and more accurate image model, thereby more accurately recognizing the image.

[0069] It should be noted that the above-mentioned terminal can be of various types, for example, it can be a mobile terminal with certain computing capabilities, or it can be a fixed computer device with recognition capabilities, etc. The above-mentioned server can also be of various types, for example, it can be a local server or a virtual cloud server. The server can be a single computer device or a computer cluster integrated with multiple computer devices according to its computing power. This is not limited here, and can be adaptively set according to actual applications and scenarios.

[0070] As an optional embodiment, determining an imaging image corresponding to a non-imaging area based on an original image and an extended image includes: determining original feature points at an edge of an imaging area in the original image; determining extended feature points in the extended image corresponding to the original feature points, wherein a similarity index between the extended feature points and the original feature points exceeds a predetermined threshold; and determining an imaging image corresponding to the non-imaging area based on the extended feature points.

[0071] In this embodiment, the steps of determining the imaging image corresponding to the non-imaging area based on the original image and the extended image are described. The original feature points of the edge of the imaging area in the original image can be determined first.

[0072] In the above steps, original feature points are involved. Original feature points refer to feature points at the edge of the imaging area in the original image. Feature points are points in the image with unique properties. For example, if they are located at the edge, they have obvious features and are easy to find corresponding points in the extended image.

[0073] There are many ways to obtain original feature points. In order to more accurately determine the original feature points at the edge of the imaging area, special feature point detection algorithms are usually used, such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features), ORB (Oriented FAST and Rotated BRIEF), etc. These algorithms can automatically identify points with significant features in the image for subsequent use.

[0074] Afterwards, the extended feature points in the extended image corresponding to the original feature points are determined. By matching the feature points at the edges of different images, seamless stitching and completion of the images can be achieved.

[0075] The extended feature point is a feature point in the extended image that corresponds to the original feature point, wherein the similarity index between the extended feature point and the original feature point exceeds a predetermined threshold. When the similarity index between the extended feature point and the original feature point exceeds the predetermined threshold, it is ensured that the extended feature point is a point in the extended image that corresponds to the original feature point and is similar to the original feature point. This avoids the phenomenon of incorrect completion caused by finding the wrong feature point. The predetermined threshold can be set adaptively according to the actual application and scenario, and is not limited here.

[0076] For example, in the above office scene, the physical representation corresponding to the original feature point is chair A, so the physical representation corresponding to the extended feature point is also chair A, so as to find the corresponding part in the image for completion.

[0077] It should be noted that in practical applications, there are some interferences in determining the original feature points at the edge of the imaging area, such as illumination changes, noise interference, occlusion, etc. At this time, a multi-scale feature detection algorithm can be used to improve the robustness to illumination changes, a non-local denoising technique can be used to reduce noise interference, and deep learning technology can be used to automatically identify and match feature points to determine the extended feature points whose similarity index corresponding to the original feature points exceeds a predetermined threshold.

[0078] Finally, the imaging image corresponding to the non-imaging area is determined based on the extended feature points, so that the imaging image corresponding to the non-imaging area is accurate and correct.

[0079] As an optional embodiment, determining an extended feature point in an extended image corresponding to an original feature point includes: determining a plurality of candidate feature points in the extended image corresponding to the original feature point, wherein similarity indexes corresponding to the plurality of candidate feature points and the original feature point respectively exceed a predetermined threshold; determining matching errors corresponding to the plurality of candidate feature points respectively; and determining, from the plurality of candidate feature points, a feature point having the smallest corresponding matching error as the extended feature point.

[0080] In this embodiment, the steps of determining the extended feature points corresponding to the original feature points in the extended image and the process of selecting the extended feature points are described. A plurality of feature points to be selected corresponding to the original feature points in the extended image may be determined first.

[0081] In this step, we already have a set of original feature points extracted from the original image. Next, we need to find feature points in the extended image that correspond to these original feature points. Since feature points may change in position, shape, or scale due to different images, we usually find multiple candidate feature points in the extended image that may be corresponding points. Therefore, we need to select the closest feature point as the extended feature point.

[0082] It should be noted that the similarity indexes of the multiple candidate feature points and the original feature points respectively exceed the predetermined threshold. When the similarity indexes of the multiple candidate feature points and the original feature points respectively exceed the predetermined threshold, it is ensured that the multiple candidate feature points are points in the extended image that correspond to the original feature points and are similar points.

[0083] After that, the matching errors corresponding to the multiple candidate feature points are determined to further accurately determine the most similar extended feature points. For each original feature point, the multiple candidate feature points found in the extended image need to be evaluated by a matching algorithm to determine their similarity or matching degree with the original feature point. This evaluation process will produce a matching error, which measures the difference between the candidate feature points and the original feature points. The matching error may be based on a variety of factors, such as the distance of the feature descriptors, the similarity of the pixels around the feature points, geometric consistency, etc.

[0084] Among them, determining multiple candidate feature points can be considered as coarse detection and rough screening, and selecting extended feature points from multiple candidate feature points can be considered as fine detection and fine screening. Through two screenings, the purpose of more accurately determining the extended feature points can be achieved.

[0085] Finally, the feature point with the smallest matching error is determined from the multiple candidate feature points as the extended feature point. That is, after comparing the matching errors of all candidate feature points, the feature point with the smallest matching error is selected as the final extended feature point. This feature point is considered to be the most likely to correspond to the original feature point because it is closest to the original feature point in multiple evaluation dimensions. This ensures that the determined extended feature point is accurate.

[0086] It should be noted that when determining the matching error, there are many algorithms that can be selected, including but not limited to matching based on feature descriptors (such as matching using feature descriptors such as SIFT, SURF, and ORB), matching algorithms based on template matching, matching algorithms based on machine learning, etc. These algorithms have their own advantages and disadvantages and can be adaptively applied to different scenarios.

[0087] As an optional embodiment, obtaining an extended image includes: when the fisheye camera is a rotatable camera, determining an extended shooting position corresponding to the fisheye camera; controlling the fisheye camera to rotate to the extended shooting position for shooting, and obtaining an extended image.

[0088] In this embodiment, an optional way of acquiring an extended image is described. In the case where the fisheye camera is a camera that can rotate to shoot, an extended shooting position corresponding to the fisheye camera is determined.

[0089] It is stated that the fisheye camera is rotatable for shooting, which means that the fisheye camera used not only has wide-angle shooting capability (the fisheye lens can capture a field of view close to 180 degrees or even wider), but also its installation or design allows rotation or adjustment of the shooting direction within a certain range. This rotatability enables the camera to capture scenes in different directions or angles, thereby increasing the flexibility and coverage of shooting and being able to capture images from other angles or ranges.

[0090] In this step, the target position (i.e., the extended shooting position) to which the fisheye camera needs to rotate is determined based on specific application requirements or scenario analysis. The selection of this position may be based on a variety of factors, such as the need to meet the connection requirements with other captured images, such as the original image. The optimal extended shooting position can then be determined through calculation or manual judgment.

[0091] After that, the fisheye camera can be controlled to rotate to the extended shooting position to shoot and obtain an extended image. Once the extended shooting position is determined, the rotation of the fisheye camera needs to be accurately controlled by a control system (such as motor drive, servo control, etc.). The camera will rotate to the specified position according to a predetermined path or angle and shoot there. Optionally, this process needs to ensure the stability and accuracy of the camera to avoid affecting the shooting quality due to jitter or deviation caused during the rotation process.

[0092] After the fisheye camera rotates to the extended shooting position and completes shooting, a new image, the extended image, will be obtained. This image will contain new scene information that is not in the original image, or recapture the information of the known scene from a new angle for subsequent image completion processing.

[0093] As an optional embodiment, obtaining an extended image includes: inputting the original image into an image extension model to obtain an extended image, wherein the image extension model is obtained by training with sample data, and the sample data includes a sample image and a sample extended image corresponding to the sample image.

[0094] In this embodiment, another method of obtaining an extended image is described, that is, obtaining the extended image by inputting the original image into an image extension model.

[0095] Among them, an image extension model is involved. The image extension model is a pre-trained machine learning or deep learning model. This model can receive the original image as input and generate one or more extended images as output. The extended image usually contains more extensive content than the original image, or contains content that is not reflected in the original image.

[0096] The image extension model is trained using a large amount of sample data. These sample data include pairs of sample images and their corresponding sample extended images. During the training process, the model learns how to extract key information from sample images and generate corresponding extended images based on this information. Through continuous iteration and optimization, the model can gradually improve the accuracy and quality of its generated extended images.

[0097] In this way, the generated extended image can be made vivid, interesting and diverse, and fit the scene involved in the original image.

[0098] It should be noted that the image expansion model can adopt a variety of different machine learning or deep learning architectures, such as convolutional neural networks (CNN), generative adversarial networks (GAN), variational autoencoders (VAE), etc. Each architecture has its own unique advantages and applicable scenarios, and the appropriate model can be selected according to the specific task requirements. When training the image expansion model, a loss function can also be defined to evaluate the difference between the generated expanded image and the real sample expanded image. By minimizing this loss function, the model can continuously optimize its parameters and weights to improve the quality of the generated image. The training process may require a lot of computing resources and time, and may require the use of various optimization algorithms and techniques to accelerate convergence and prevent overfitting.

[0099] As an optional embodiment, an imaging image corresponding to a non-imaging area is determined based on an original image and an extended image, including: determining an image type of the extended image, wherein the image type includes at least a fisheye image type; determining a distortion adjustment method corresponding to the image type; based on the original image, adjusting the extended image using the distortion adjustment method to obtain an adjusted extended image; and determining an imaging image based on the original image and the adjusted extended image.

[0100] In this embodiment, the process of adjusting distortion in the process of determining an image is described.

[0101] Since the ways of adjusting distortion for different image types are different, it is necessary to identify and analyze the specific type of the extended image. Among them, the image type includes at least the fisheye image type, that is, it will be determined whether the extended image belongs to a fisheye image. Fisheye images have unique distortion characteristics due to their ultra-wide-angle field of view, such as straight lines bending into arcs in the image. Therefore, it is necessary to consider this distortion characteristic area for distortion correction. That is, the distortion adjustment method corresponding to the image type is determined. In addition, the image type may also include other types, such as ordinary wide-angle images, panoramic images, etc. Different distortion adjustment methods corresponding to different image types can be determined for targeted processing.

[0102] Afterwards, the extended image is adjusted using a distortion adjustment method based on the original image to obtain an adjusted extended image. For example, for fisheye type images, the distortion adjustment method usually involves remapping the pixel positions in the image to eliminate or reduce the shape distortion caused by the lens characteristics. For fisheye images, commonly used distortion adjustment methods include distortion correction based on projection models (such as equidistant projection, equi-stereoscopic projection, etc.) and correction methods based on geometric transformation (such as polynomial correction, radial distortion correction, etc.). Through adjustment, it can be ensured that the adjusted image can accurately reflect the shape and layout of the actual scene.

[0103] The extended image after distortion adjustment may have some overlapping areas or complementary information with the original image. In this step, this information will be used to determine the final image. This ensures that the image retains the original image information while also including the new content in the extended image.

[0104] As an optional embodiment, the non-imaging area in the original image is complemented according to the imaging image corresponding to the non-imaging area to obtain a target image, including: complementing the non-imaging area in the original image according to the imaging image corresponding to the non-imaging area to obtain a complemented image; adjusting image parameters between the imaging area and the non-imaging area in the complemented image to obtain the target image, so as to achieve a smooth transition between the imaging area and the non-imaging area in the target image.

[0105] In this embodiment, operations such as smooth transitions performed during completion are described. Imaging information that may exist in non-imaging areas in other perspectives or images (i.e., imaging images corresponding to the non-imaging areas) is used to complete these non-imaging areas. Completion methods may include image stitching, image fusion, image interpolation, or image generation technology based on deep learning. After the above-mentioned completion process, the non-imaging areas in the original image are replaced by corresponding imaging information, thereby obtaining a completed image containing more scene information. This image is visually more complete, but there may be obvious differences or discontinuities at the junction of the imaging area and the non-imaging area.

[0106] In order to eliminate or reduce this difference, it is necessary to adjust the image parameters between the imaging area and the non-imaging area in the complement image. These parameters may include brightness, contrast, color balance, texture details, etc. The purpose of the adjustment is to make the two areas visually closer, so as to achieve a smooth transition. The adjustment method may involve local image processing technology, image filtering, edge fusion, etc. After the above adjustment, the transition between the imaging area and the non-imaging area in the complement image becomes smooth and natural, thus obtaining the final target image. This image not only contains all the information in the original image, but also supplements the content of the non-imaging area, and maintains continuity and consistency in the overall visual sense.

[0107] Based on the above embodiments and optional embodiments, an optional implementation is provided, which is described in detail below.

[0108] An optional implementation of the present invention relates to an image completion method and system, which provides a method for filling the invalid black area (non-imaging area) of a fisheye camera by using valid image feature points combined with external input. The external input has a wider field of view and covers the invalid black area of ​​the original image, and the corresponding pixels are extracted and merged into the original image. On the basis of the original fisheye camera image output, the steps of imaging area calculation, imaging area feature point extraction, external image input, image fusion filling, etc. are added.

[0109] The following is an introduction to them:

[0110] 1) Introduce the scenarios in which the optional implementation modes of the present invention are applied:

[0111] The image completion method provided in the optional embodiment of the present invention can be applied to the scene captured by the fisheye camera to complete the fisheye image. Since the fisheye camera has the characteristics of ultra-wide-angle field of view, it can capture a wider scene than ordinary lenses, and is suitable for shooting large-scale or panoramic images. It is often used in monitoring, panoramic photography, virtual reality (VR) and other fields, and is also often used in creative photography to produce unique visual effects.

[0112] Since the fisheye camera is used with a fisheye lens for imaging, the effective image formed is a circle. After the sensor and encoder, the image or video finally formed is a rectangle with a certain aspect ratio. Therefore, many common fisheye cameras have invalid shadow areas in the final output field of view. Figure 2 is a schematic diagram of a fisheye image provided by an optional embodiment of the present invention, represented by a simple model, Figure 3 is a simple model schematic diagram provided by an optional embodiment of the present invention, such as Figure 2 , 3 As shown in the figure, the circle is the effective imaging area, and the rectangle is the actual user-acquired picture, which may include or exclude two scenarios. The shaded part is the invalid shadow area. Such shadows affect the aesthetics of the final picture and the viewing experience.

[0113] 2) Problems in related technologies:

[0114] The commonly used methods for removing shadows are as follows: 1. Direct cropping, the disadvantage is a significant loss of the field of view of the picture; 2. Distortion correction, which usually results in a small loss of picture information after correction and changes the imaging picture.

[0115] In order to solve the above problems, the prior art proposes a method of first adjusting the brightness value of the black area to make the edge of the area transition smoothly, and then filling the black area by LSC compensation. This method fills the original black area more softly and smoothly than the original image, but contains less information. The picture outside the effective imaging circle cannot be associated with the information inside the effective imaging circle. When filling the black area in the fisheye image, the correlation between the filling content and the fisheye image itself is low, which can easily cause technical problems of poor viewing experience.

[0116] Based on the above, an optional implementation of the present invention utilizes the internal feature information of the effective imaging circle and the real image input externally to supplement the black area, thereby maximizing the effective information of the black area to solve the above problem.

[0117] 3) Introduce the image completion method:

[0118] Figure 4 is a flow chart of an image completion method provided by an optional embodiment of the present invention, such as Figure 4 As shown:

[0119] S1. Obtaining the original image data collected by the camera.

[0120] As described above, an original image is obtained, wherein the original image is an image captured by a fisheye camera.

[0121] S2. Calculate the center and radius of the imaging area in the output image area, and simultaneously obtain the range of the non-imaging black area.

[0122] As described above, the imaging area and the non-imaging area in the original image are determined.

[0123] S3, obtaining an external input image with a wider field of view including the edge of the image. This step can be taken by using a wider angle camera and input into the system.

[0124] As described above, an extended image is acquired, wherein the extended image is an image including the edge portion of the image area and the image corresponding to the non-imaging area.

[0125] S4. Extract feature values ​​from the blocks at the edge of the effective image in the original image.

[0126] As described above, the original feature points at the edge of the imaging area in the original image are determined.

[0127] This step involves feature point extraction technology. Currently, there are many feature point extraction technologies, such as gist features, etc. Different feature extraction methods do not affect the description of the optional implementation methods of the present invention.

[0128] S5. Perform feature matching on the blocks at the edge of the effective image in the original image and the blocks in the external image to find the most similar parts.

[0129] Figure 5 is a schematic diagram of feature matching provided by an optional embodiment of the present invention, such as Figure 5 As shown, a feature matching operation is performed on the two images.

[0130] As described above, the extended feature points corresponding to the original feature points in the extended image are determined, and the imaging image corresponding to the non-imaging area is determined based on the extended feature points.

[0131] Figure 6 This is a schematic diagram of target feature point selection provided by an optional implementation manner of the present invention. As shown in 6, this step can utilize the area to be completed to match the corresponding extended area of ​​the external image feature point. For each feature point (such as a pixel), multiple spaces can be searched to calculate the minimum matching error and select the best completion position point, that is, determine the optimal extended feature point from multiple feature points.

[0132] As described above, multiple candidate feature points corresponding to the original feature points in the extended image are determined, matching errors corresponding to the multiple candidate feature points are determined, and the feature point with the smallest corresponding matching error is determined from the multiple candidate feature points as the extended feature point for subsequent image completion.

[0133] S6. The external image completes the black shadow at the edge of the fisheye camera.

[0134] As described above, the non-imaging area in the original image is completed based on the imaging image corresponding to the non-imaging area to achieve the purpose of obtaining the target image.

[0135] Optionally, when completing an image, it is not necessary to take the value of the corresponding pixel point alone. After completion, the brightness and other information can be adjusted to achieve a smooth transition with the edge. That is, as described above, the non-imaging area in the original image is completed based on the imaging image corresponding to the non-imaging area to obtain a completed image. Then, the image parameters between the imaging area and the non-imaging area in the completed image are adjusted to obtain a target image, so as to achieve a smooth transition between the imaging area and the non-imaging area in the target image.

[0136] Optionally, when determining the above-mentioned extended image, a method can also be provided for AI expansion using only internal feature points without external input. In this method, the external input image is only used for pre-training the model built into the fisheye camera. Figure 7 The process step diagram of the AI ​​model provided by the optional embodiment of the present invention is as follows: Figure 7 As shown, the above extended image can be obtained through the AI ​​extension module.

[0137] The process steps for using the AI ​​model are as follows:

[0138] S1. Use external input images as training sets to train the AI ​​extended algorithm model.

[0139] AI expansion algorithms have been widely used in the market, and the selection of an algorithm does not affect the implementation of the present invention.

[0140] S2. Obtaining the original image data collected by the camera.

[0141] S3. Calculate the center and radius of the imaging area in the output image area, and simultaneously obtain the range of the non-imaging black area.

[0142] S4. Input the effective imaging area information and expand the image in the non-imaging area through the AI ​​algorithm.

[0143] During this process, if the camera lacks computing power, it can be completed in the cloud and returned to the camera. The processing location of this process does not affect the implementation of the optional implementation mode of the present invention.

[0144] S5. Output the expanded image. This process can be performed multiple times, and the user determines the best expansion effect.

[0145] Optionally, the confirmed satisfactory extended non-imaging area is fixed and fused with the image generated by the subsequent camera, and the above process can be repeated for updating.

[0146] In this way, a complete system for completing the fisheye camera image is realized using AI extension technology. AI extension may match the actual image, but it cannot match it completely in most cases. However, it can be generated multiple times and has rich imagination, and can be used as a generator of interesting filling methods.

[0147] The present invention can solve the technical problem that when the related art fills the black area in the fisheye image, the correlation between the filling content and the fisheye image itself is low, which easily causes poor viewing experience through the following settings.

[0148] 4) System corresponding to the execution of the image completion method:

[0149] Based on the optional implementation of the present invention, it can also be expanded into a system for automatically filling in the non-imaging area of ​​the fisheye camera. The system consists of a fisheye camera and a plurality of external cameras that work independently and can cover the non-imaging black shadows of the fisheye camera.

[0150] Figure 8 is a schematic diagram of a system provided by an optional embodiment of the present invention, such as Figure 8 As shown, the system can provide external images to the fisheye camera at fixed intervals, and the time interval can be determined by performance, so as to achieve the effect of automatic updating of non-imaging areas according to the actual picture.

[0151] Fig. 9 is a structural block diagram of an image completion system according to an embodiment of the present invention. Fig. 9 As shown, according to another aspect of the present invention, an image completion system is provided, comprising: a fisheye camera 902, and a controller 904, wherein the fisheye camera 902 is connected to the controller 904,

[0152] Controller 902 is used to obtain an original image, wherein the original image is an image captured by a fisheye camera; determine an imaging area and a non-imaging area in the original image; obtain an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; determine an imaging image corresponding to the non-imaging area based on the original image and the extended image; and complete the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a target image.

[0153] For the system provided in the present application, an original image is obtained, wherein the original image is an image taken by a fisheye camera; an imaging area and a non-imaging area in the original image are determined; an extended image is obtained, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; an imaging image corresponding to the non-imaging area is determined based on the original image and the extended image; and the non-imaging area in the original image is completed based on the imaging image corresponding to the non-imaging area to obtain a target image. The embodiment of the present invention completes the black part (non-imaging area) of the original image by obtaining an extended image including an edge portion of the imaging area and an image corresponding to the non-imaging area, so that in the obtained target image, the filling content is related to the original image itself, thereby improving the user's perception, thereby solving the technical problem that when the related technology fills the black area in the fisheye image, the filling content has a low correlation with the fisheye image itself, which is easy to cause poor perception.

[0154] As an optional embodiment, the system further includes other cameras, wherein the other cameras are connected to the controller and are used to capture extended images.

[0155] In this embodiment, it is explained that the system also includes other cameras, which are connected to the controller and are used to shoot extended images. By setting other cameras, an optional way of obtaining extended images is set. By controlling other cameras to shoot, a new image, i.e., an extended image, will be obtained. This image will contain new scene information that the original image does not have, or recapture the information of the known scene from a new angle. This is to facilitate subsequent image completion processing.

[0156] The examples and application scenarios implemented by the corresponding steps of the controller are the same, but are not limited to the contents disclosed in the above embodiments.

[0157] Fig.10 is a structural block diagram of an image completion device according to an embodiment of the present invention. Fig.10 According to another aspect of the present invention, an image completion device is provided, comprising: a first acquisition module 1002, a first determination module 1004, a second acquisition module 1006, a second determination module 1008 and a completion module 1010, wherein:

[0158] The first acquisition module 1002 is used to acquire an original image, wherein the original image is an image captured by a fisheye camera; the first determination module 1004 is connected to the above-mentioned first acquisition module 1002, and is used to determine the imaging area and the non-imaging area in the original image; the second acquisition module 1006 is connected to the above-mentioned first determination module 1004, and is used to acquire an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; the second determination module 1008 is connected to the above-mentioned second acquisition module 1006, and is used to determine the imaging image corresponding to the non-imaging area based on the original image and the extended image; the completion module 1010 is connected to the above-mentioned second determination module 1008, and is used to complete the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a target image.

[0159] The above-mentioned first acquisition module 1002, first determination module 1004, second acquisition module 1006, second determination module 1008 and completion module 1010 correspond one-to-one to steps S101 to S105, and the instances and application scenarios implemented by multiple modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0160] An embodiment of the present invention may provide an electronic device, which includes a memory and a processor. The processor may call the information and application stored in the memory through a transmission device to perform the following steps: obtaining an original image, wherein the original image is an image captured by a fisheye camera; determining an imaging area and a non-imaging area in the original image; obtaining an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; determining an imaging image corresponding to the non-imaging area based on the original image and the extended image; and completing the non-imaging area in the original image based on the imaging image corresponding to the non-imaging area to obtain a target image.

[0161] Optionally, based on the original image and the extended image, determining the imaging image corresponding to the non-imaging area includes: determining the original feature points of the edge portion of the imaging area in the original image; determining the extended feature points in the extended image corresponding to the original feature points; and determining the imaging image corresponding to the non-imaging area based on the extended feature points.

[0162] Optionally, determining an extended feature point in the extended image corresponding to the original feature point includes: determining multiple candidate feature points in the extended image corresponding to the original feature point; determining matching errors corresponding to the multiple candidate feature points respectively; and determining a feature point with the smallest corresponding matching error from the multiple candidate feature points as the extended feature point.

[0163] Optionally, obtaining an extended image includes: when the fisheye camera is a rotatable camera for shooting, determining an extended shooting position corresponding to the fisheye camera; controlling the fisheye camera to rotate to the extended shooting position for shooting, and obtaining the extended image.

[0164] Optionally, obtaining an extended image includes: inputting the original image into an image extension model to obtain an extended image, wherein the image extension model is obtained by training with sample data, and the sample data includes a sample image and a sample extended image corresponding to the sample image.

[0165] Optionally, based on the original image and the extended image, an imaging image corresponding to the non-imaging area is determined, including: determining an image type of the extended image, wherein the image type includes at least a fisheye image type; determining a distortion adjustment method corresponding to the image type; based on the original image, using the distortion adjustment method to adjust the extended image to obtain an adjusted extended image; and determining an imaging image based on the original image and the adjusted extended image.

[0166] Optionally, based on the imaging image corresponding to the non-imaging area, the non-imaging area in the original image is complemented to obtain a target image, including: based on the imaging image corresponding to the non-imaging area, the non-imaging area in the original image is complemented to obtain a complemented image; and image parameters between the imaging area and the non-imaging area in the complemented image are adjusted to obtain the target image, so as to achieve a smooth transition between the imaging area and the non-imaging area in the target image.

[0167] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a non-volatile storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0168] The embodiment of the present invention further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the image completion method provided by the above embodiment.

[0169] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the image completion method in each embodiment of the present application when the computer program is executed by a processor.

[0170] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0171] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0174] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0175] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0176] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An image completion method, characterized in that: Acquire an original image, wherein the original image is an image captured by a fisheye camera; Determining an imaging area and a non-imaging area in the original image; Acquire an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; Determining an imaging image corresponding to the non-imaging area according to the original image and the extended image; The non-imaging area in the original image is completed according to the imaging image corresponding to the non-imaging area to obtain a target image.

2. The method according to claim 1, characterized in that: The step of determining the imaging image corresponding to the non-imaging area according to the original image and the extended image includes: Determining original feature points at the edge of the imaging area in the original image; Determining an extended feature point in the extended image that corresponds to the original feature point, wherein a similarity index between the extended feature point and the original feature point exceeds a predetermined threshold; An imaging image corresponding to the non-imaging area is determined according to the extended feature points.

3. The method according to claim 2, characterized in that The determining, in the extended image, an extended feature point corresponding to the original feature point comprises: Determine a plurality of candidate feature points corresponding to the original feature points in the extended image, wherein similarity indexes of the plurality of candidate feature points respectively corresponding to the original feature points exceed the predetermined threshold; Determine the matching errors respectively corresponding to the plurality of feature points to be selected; A feature point with the smallest corresponding matching error is determined from the multiple candidate feature points as the extended feature point.

4. The method according to claim 1, characterized in that: The step of acquiring the extended image comprises: In the case where the fisheye camera is a rotatable camera for shooting, determining an extended shooting position corresponding to the fisheye camera; The fisheye camera is controlled to rotate to the extended shooting position to shoot, so as to obtain the extended image.

5. The method according to claim 1, characterized in that The step of acquiring the extended image comprises: The original image is input into an image extension model to obtain the extended image, wherein the image extension model is obtained by training with sample data, and the sample data includes a sample image and a sample extended image corresponding to the sample image.

6. The method according to claim 1, characterized in that The step of determining the imaging image corresponding to the non-imaging area according to the original image and the extended image includes: Determining an image type of the extended image, wherein the image type at least includes a fisheye image type; Determining a distortion adjustment method corresponding to the image type; According to the original image, the extended image is adjusted using the distortion adjustment method to obtain an adjusted extended image; The imaged image is determined according to the original image and the adjusted extended image.

7. The method according to any one of claims 1 to 6, characterized in that: The method of completing the non-imaging area in the original image according to the imaging image corresponding to the non-imaging area to obtain the target image includes: Completing the non-imaging area in the original image according to the imaging image corresponding to the non-imaging area to obtain a completed image; The image parameters between the imaging area and the non-imaging area in the complement image are adjusted to obtain the target image, so as to achieve a smooth transition between the imaging area and the non-imaging area in the target image.

8. An image completion system, characterized in that: include: A fisheye camera and a controller, wherein the fisheye camera is connected to the controller. The controller is used to obtain an original image, wherein the original image is an image captured by the fisheye camera; Determining an imaging area and a non-imaging area in the original image; Acquire an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; Determining an imaging image corresponding to the non-imaging area according to the original image and the extended image; The non-imaging area in the original image is completed according to the imaging image corresponding to the non-imaging area to obtain a target image.

9. The system according to claim 8, characterized in that Also includes: Other cameras, wherein the other cameras are connected to the controller, and the other cameras are used to shoot the extended image.

10. An image completion device, characterized in that: A first acquisition module is used to acquire an original image, wherein the original image is an image captured by a fisheye camera; A first determination module, used to determine an imaging area and a non-imaging area in the original image; A second acquisition module is used to acquire an extended image, wherein the extended image is an image including an edge portion of the imaging area and an image corresponding to the non-imaging area; A second determination module, configured to determine an imaging image corresponding to the non-imaging area according to the original image and the extended image; The completion module is used to complete the non-imaging area in the original image according to the imaging image corresponding to the non-imaging area to obtain a target image.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the image completion method according to any one of claims 1 to 7.

12. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program executes the image completion method described in any one of claims 1 to 7 when running.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the image completion method according to any one of claims 1 to 7 are implemented.

Images