High-altitude parabolic image synthesis method, synthesis device and computer readable storage medium

Abstract

The application discloses a high-altitude parabolic image synthesis method, which comprises the following steps: acquiring a foreground target for synthesizing a high-altitude parabolic image; acquiring a background image for synthesizing the high-altitude parabolic image, and marking a starting point and an ending point of a parabolic process of the foreground target in the background image; simulating high-altitude parabolic positions of the foreground target in the background image in the parabolic process from the starting point to the ending point and a scaling ratio of the foreground target corresponding to each high-altitude parabolic position; and generating a high-altitude parabolic video image by fusing: scaling the foreground target according to the scaling ratio corresponding to each high-altitude parabolic position, and fusing the scaled foreground target to the high-altitude parabolic position. The high-altitude parabolic image synthesis method can fuse the foreground target selected according to requirements into the background image to generate a high-altitude parabolic video image, thereby meeting the requirements of a data set for subsequent learning and training. The application further provides a high-altitude parabolic image synthesis device and a computer readable storage medium.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, more particularly to a high-altitude projectile image synthesis method, a synthesis device and a computer readable storage medium. BACKGROUND

[0002] In recent years, with the rapid development of urbanization, high-rise buildings are also increasing, and the high-altitude projectile behavior is also common, which leads to more and more injuries and receives widespread attention from the society. With the rapid development of artificial intelligence technology, the existing high-altitude projectile detection model trained by deep learning is used to detect high-altitude projectile behavior. However, for deep learning, the acquisition of data set is a crucial link. A sufficient data set is beneficial to guarantee the generalization ability of the trained high-altitude projectile detection model. At present, the data acquisition method by online crawling needs to spend a lot of time to select manually, or for example, directly using PS software to synthesize the image of high-rise and foreground. This method synthesizes the most accurate and most real image, but it takes a lot of manual time to synthesize each image. Therefore, there is an urgent need for a high-altitude projectile image synthesis method that can fuse the foreground target of the projectile into the background image to meet the data requirements for training the high-altitude projectile detection model. SUMMARY

[0003] For the prior art, the technical problem solved by the present application is to provide a high-altitude projectile image synthesis method, a synthesis device and a computer readable storage medium capable of generating high-altitude projectile image data.

[0004] To solve the above technical problems, in a first aspect, the present application provides a high-altitude projectile image synthesis method, comprising:

[0005] obtaining a foreground target used for synthesizing a high-altitude projectile image;

[0006] obtaining a background image used for synthesizing a high-altitude projectile image, and marking the starting point and the ending point of the projectile process of the foreground target in the background image;

[0007] simulating the high-altitude projectile position of the foreground target in the background image in the projectile process from the starting point to the ending point, and the scaling ratio of the foreground target corresponding to each high-altitude projectile position; and

[0008] generating a high-altitude projectile video image by fusing: scaling the foreground target according to the scaling ratio corresponding to each high-altitude projectile position, and fusing the scaled foreground target to the high-altitude projectile position.

[0009] In a possible implementation, the step of simulating the high-altitude projectile position of the foreground target in the background image in the projectile process from the starting point to the ending point, and the scaling ratio of the foreground target corresponding to each high-altitude projectile position comprises:

[0010] obtaining a height h and a width w of the foreground object;

[0011] specifying a horizontal initial velocity v0 and an acceleration a in the vertical direction when the parabola is specified, and giving a height H0 and Hs of the foreground object at a starting point O0(x0, y0) and an ending point O s (x s ,y s ) in the background image respectively, and H0 < Hs;

[0012] sequentially simulating and calculating a high-altitude parabolic position O1(x1, y1), O2(x2, y2)…, O i (x i ,y i ) of the foreground object in the background image when the foreground object is parabolic from the 1st frame to the i-th frame, and a height H1, H2, H 3… ,H i of the foreground object corresponding to each high-altitude parabolic position, obtaining a scaling ratio b i = H i / h of the foreground object corresponding to the high-altitude parabolic position when each frame is parabolic;

[0013] looping the simulation until the high-altitude parabolic position meets a parabolic constraint condition, and the parabolic simulation ends;

[0014] wherein, i ≥ 1, the parabolic constraint condition is: x i+1 is greater than the width of the background image, or y i+1 is greater than the height of the background image, or y i+1 is greater than y s , or x i+1 is greater than x s .

[0015] In a possible implementation, the high-altitude parabolic position O i (x i ,y i ) of the foreground object in the background image when the foreground object is parabolic in the i-th frame and the height H i of the foreground object are simulated and calculated as follows:

[0016] x i = x0 + v0*(i-1);

[0017] y i = H0 + a(i-1) 2 ;

[0018]

[0019] In a possible implementation, the step of obtaining the foreground object for synthesizing the high-altitude parabolic image comprises: obtaining a foreground image and extracting the foreground object in the foreground image for synthesizing the high-altitude parabolic image, calculating a circumscribed rectangle of the foreground object, and taking the height h and the width w of the foreground object as the height and the width of the circumscribed rectangle.

[0020] In a possible implementation, the horizontal initial velocity v0 is 3, and the vertical acceleration a is 0.2.

[0021] In a possible implementation, the foreground object is scaled according to the scaling ratio corresponding to each high-altitude parabolic position, that is, the height and the width of the foreground object are scaled by the same scaling ratio according to the scaling ratio.

[0022] In a possible implementation, the step of fusing the scaled foreground object into the high-altitude parabolic position comprises:

[0023] obtaining the center point of the scaled foreground object;

[0024] pasting the scaled foreground object onto the background image and making the center of the foreground object coincide with the high-altitude parabolic position;

[0025] performing harmonization processing on the pasted foreground object and the background image.

[0026] In a possible implementation, the step of performing harmonization processing on the pasted foreground object and the background image comprises:

[0027] adjusting the brightness of the region of the pasted foreground object: obtaining the brightness mean value L1 of the pasted foreground region and obtaining the brightness mean value L2 of the region of the background image occupied by the foreground object, and scaling the size of the pixel value of each pixel point of the pasted foreground region by the ratio of L2 / L1;

[0028] performing smoothing processing on the edge of the pasted foreground object: extracting the edge of the pasted foreground object and performing Gaussian blur processing on the transformation.

[0029] In a second aspect, the present application provides a high-altitude parabolic synthesis device, comprising:

[0030] an image obtaining unit, configured to obtain a foreground object and a background image;

[0031] a high-altitude parabolic simulation unit, connected to the image obtaining unit, and configured to mark the starting point and the ending point of the parabolic process of the foreground object in the background image, and simulate the high-altitude parabolic position of the foreground object in the background image in the parabolic process from the starting point to the ending point and the scaling ratio of the foreground object corresponding to each high-altitude parabolic position; and

[0032] The high-altitude parabolic video image synthesis unit is connected to the high-altitude parabolic simulation unit and is configured to scale the foreground target according to a corresponding scaling ratio of each high-altitude parabolic position, and to fuse the scaled foreground target into the high-altitude parabolic position.

[0033] In a third aspect, the present application provides a computer readable storage medium for storing a computer program, which, when executed by a processor, is configured to implement the high-altitude parabolic image synthesis method.

[0034] In the high-altitude parabolic image synthesis method, the synthesis device and the computer readable storage medium, the high-altitude parabolic positions of the foreground target in the background image during the high-altitude parabolic process from the starting point to the ending point and the corresponding scaling ratios of the foreground target at each high-altitude parabolic position are simulated, and then the foreground target is fused into the corresponding high-altitude parabolic position according to the corresponding scaling ratio, thereby generating a high-altitude parabolic video image. Therefore, the high-altitude parabolic image synthesis method can fuse the foreground target selected according to the requirement into the background image to generate a high-altitude parabolic video image, thereby meeting the data set required for subsequent learning and training. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0036] Figure 1 The high-altitude parabolic image synthesis method of the embodiments of the present application is shown in the overall flowchart.

[0037] Figure 2 The foreground target image obtained by the embodiments of the present application is shown in the foreground target image.

[0038] Figure 3 The step flowchart of simulating the high-altitude parabolic positions of the foreground target in the background image during the parabolic process from the starting point to the ending point and the scaling ratios of the foreground target at each high-altitude parabolic position is shown in the foreground target image.

[0039] Figure 4 The step flowchart of fusing the scaled foreground target into the high-altitude parabolic position is shown in the foreground target image.

[0040] Figures 5(a)-5(e) The multiple frames of high-altitude parabolic images generated by the high-altitude parabolic image synthesis method of the embodiments of the present application are shown in the foreground target image. DETAILED DESCRIPTION

[0041] In order to make the technical problems, technical solutions and beneficial effects of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, and not to limit the present application.

[0042] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0043] It should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0044] In addition, the terms "first", "second", "third", etc. are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0045] The high-altitude projectile image synthesis method, synthesis device and computer readable storage medium of the present application will now be described in detail with reference to the accompanying drawings.

[0046] Please refer to Figure 1 The high-altitude projectile image synthesis method provided by the embodiments of the present application includes the following steps:

[0047] Step S100: Obtain a foreground target for synthesizing a high-altitude projectile image; the foreground target can be selected as required, and the obtained foreground target is as shown in Figure 2 .

[0048] Step S200: Obtain a background image for synthesizing a high-altitude projectile image, and mark the starting point and the ending point of the projectile process of the foreground target in the background image; the background image can also be selected as required.

[0049] Step S300: Simulate the high-altitude projectile position of the foreground target in the background image during the projectile process from the starting point to the ending point, and the scaling ratio of the foreground target corresponding to each high-altitude projectile position.

[0050] Step S400: generating the high-altitude parabolic video image by fusing the foreground target scaled according to the scaling ratio corresponding to each high-altitude parabolic position to the high-altitude parabolic position.

[0051] In the high-altitude parabolic image synthesis method, the high-altitude parabolic positions of the foreground target in the background image between the start point and the end point in the high-altitude parabolic process and the corresponding scaling ratios of the foreground target at each high-altitude parabolic position are simulated, and then the foreground target is fused to the corresponding high-altitude parabolic position according to each scaling ratio, thereby generating the high-altitude parabolic video image. Therefore, the high-altitude parabolic image synthesis method can fuse the foreground target selected as required to the background image to generate the high-altitude parabolic video image, thereby meeting the data set required for subsequent learning and training.

[0052] In an application embodiment, in step S100, a foreground image is obtained, which can be understood to contain the foreground target used for synthesizing the high-altitude parabolic image; the foreground target in the foreground image used for synthesizing the high-altitude parabolic image is extracted, and the circumscribed rectangular frame of the foreground target is calculated, and the height h and the width w of the foreground target are recorded as the height and the width of the circumscribed rectangular frame. In this embodiment, the foreground target in the foreground image can be obtained by using the labelme annotation software.

[0053] In an application embodiment, in step S200, the obtained background image can be an image from a top view angle, such as a high-rise building background image from a top view angle. In step S200, the start point and the end point are marked in the background image by using the labelme annotation software, and the start point coordinates are recorded as O0(x0, y0) and the end point coordinates are recorded as O s (x s ,y s )。

[0054] It should be noted that in the present application embodiment, the height refers to the height in the Y-axis direction of the image coordinate system, and the width refers to the width in the X-axis direction of the image coordinate system, and the origin of the image coordinate system is at the upper left corner of the image.

[0055] Please refer to Figure 3 In step S300, simulating the high-altitude parabolic positions of the foreground target in the background image in the parabolic process from the start point to the end point and the scaling ratios of the foreground target corresponding to each high-altitude parabolic position include the following steps:

[0056] Step S310: obtaining the height h and the width w of the foreground target;

[0057] Step S320: specifying the horizontal initial velocity v0 and the vertical acceleration a in the parabolic process, and giving the start point O0(x0, y0) and Os (x s ,y s The heights at the endpoint are H0 and Hs;

[0058] Step S330: Let i be the number of the simulated high-altitude parabolic frame, i = 1;

[0059] Step S340: Simulate and calculate the position O of the high-altitude parabola in the background image when the foreground target is parabolic in the i-th frame. i (x i ,y i ) and the location of the object thrown from a height O i (x i ,y i The corresponding height H of the foreground target i ;

[0060] Step S350: Obtain the scaling factor b of the foreground target corresponding to the position of the high-altitude projectile during the i-th frame. i =H i / h;

[0061] Step S360: Determine whether the position of the high-altitude parabola in the i-th frame of the simulation satisfies the parabola constraint condition; if yes, the parabola simulation ends; otherwise, the number of image frames i = i + 1 in the high-altitude parabola simulation is set and the process returns to continue executing step S340.

[0062] Understandably, in steps S310 to S360, the high-altitude parabolic positions O1(x1,y1), O2(x2,y2), ..., O in the background image of the foreground target during the parabolic events in frames 1 to 1i are sequentially simulated and calculated. i (x i ,y i ) and the heights H1, H2, H of the foreground targets corresponding to each high-altitude projectile position. 3… H i Obtain the scaling factor b of the foreground target corresponding to the high-altitude parabolic position in each frame. i =H i / h; The simulation continues until the position of the high-altitude projectile meets the parabolic constraint conditions, at which point the simulation ends.

[0063] In one embodiment of the application, the initial horizontal velocity v0 is 3, and the vertical acceleration a is 0.2.

[0064] In step S360 above, the parabolic constraint condition is: x i+1 Larger than the width of the background image, or y i+1 Greater than the height of the background image, or y i+1 Greater than y s , or x i+1 Greater than x s .

[0065] In step S340, the simulation calculates the high-altitude parabolic position O of the foreground object in the background image at the i-th frame parabolic time i (x i ,y i ) and the height H i of the foreground object as follows: x i = x0+ v0*(i-1); y i = H0+ a(i-1) 2 .

[0066] In steps S100 and S400, the simulation of the parabolic process in step S300 simulates a plurality of continuous high-altitude parabolic images, and calculates the high-altitude parabolic position in each frame parabolic image and the scaling ratio of the foreground object at the high-altitude parabolic position. It can be understood that the foreground object has an original size, but when a person stands on the ground to observe the parabolic object in an upward manner, the size of the parabolic object is getting larger and larger. After the height of the foreground object at the parabolic starting point in the background image is set, in the simulation of the parabolic process, in order to comply with the principle that the size of the parabolic object is getting larger and larger, the size of the foreground object at the high-altitude parabolic position in each frame image of the generated high-altitude parabolic video image is scaled relative to the original size of the foreground object by a certain scaling ratio, and the scaling ratio is the aforementioned b i .

[0067] In an application embodiment, in step S400, the foreground object is scaled according to the scaling ratio corresponding to each high-altitude parabolic position, which includes: the height and the width of the foreground object are scaled by the same scaling ratio according to the scaling ratio.

[0068] In an application embodiment, in step S400, the step of fusing the scaled foreground object to the high-altitude parabolic position includes: step S410: obtaining the center point of the scaled foreground object; step S420: pasting the scaled foreground object to the background image and making the center of the foreground object coincide with the high-altitude parabolic position; and step S430: performing harmonization processing on the pasted foreground object and the background image.

[0069] The step of performing harmonization processing on the pasted foreground object and the background image includes:

[0070] Step S431: adjusting the brightness of the region of the pasted foreground object: obtaining the brightness mean value L1 of the pasted foreground region and obtaining the brightness mean value L2 of the region of the background image occupied by the foreground object, and scaling the size of the pixel value of each pixel point of the pasted foreground region by L2 / L1 ratio;

[0071] Step S432: smoothing the edge of the pasted foreground object: extracting the edge of the pasted foreground object and performing Gaussian blur processing on the transformation.

[0072] It can be understood that in step S431, the value of any pixel point of the region of the foreground object before the brightness adjustment is A, and the value of the pixel point after the adjustment is (L2 / L1) x A.

[0073] wherein, Figures 5(a)-5(e) is used to generate multiple frames of images in the aerial throwing video image using the aerial throwing image synthesis method, and Figures 5(a)-5(e) is arranged in the order of before and after the parabolic process. In FIG. Figures 5(a)-5(e) , the foreground object of the aerial throwing is framed with a rectangular frame for viewing.

[0074] In the aerial throwing image synthesis method, the aerial throwing image synthesis method can automatically fuse to generate an aerial throwing video image; at the same time, different scaling ratios are introduced for different aerial throwing positions, so that the size of the foreground object changes dynamically in the parabolic process; and when the foreground object is fused into the background image, the brightness of the foreground object region is adjusted according to the background image, and the edge of the foreground object fused into the background image is blurred to make the edge of the foreground object transition more smoothly, so that the brightness and edge processing of the foreground object fused into the background image makes the foreground object and the background image more harmonious, that is, makes the fused image not conspicuous to conform to the real scene.

[0075] The embodiment of the present application also provides an aerial throwing synthesis device comprising an image acquisition unit, an aerial throwing simulation unit and an aerial throwing video image synthesis unit. Specifically, the image acquisition unit is used to acquire a foreground object and a background image. The aerial throwing simulation unit is connected to the image acquisition unit and is used to mark the starting point and the ending point of the parabolic process of the foreground object in the background image, and simulate the aerial throwing position of the foreground object in the background image in the parabolic process from the starting point to the ending point and the scaling ratio of the foreground object corresponding to each aerial throwing position. The aerial throwing video image synthesis unit is connected to the aerial throwing simulation unit and is used to scale the foreground object according to the scaling ratio corresponding to each aerial throwing position, and fuse the scaled foreground object into the aerial throwing position.

[0076] The embodiment of the present application also provides a computer readable storage medium for storing a computer program, when the computer program is executed by a processor, the computer program is used to implement the above-mentioned aerial throwing image synthesis method.

[0077] In the embodiment, the processor can be an integrated circuit chip with processing capability of signals. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The processor can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The computer readable storage medium stores program codes, including a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0078] The above merely provides the preferred embodiments of the application, and is not intended to limit the application. Any modification, equivalent replacement and improvement made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A high-altitude parabolic image synthesis method, characterized by, The method comprises: obtaining a foreground object for synthesizing a high-altitude parabolic image; obtaining a background image for synthesizing the high-altitude parabolic image, and marking a starting point and an ending point of a parabolic process of the foreground object in the background image; simulating a high-altitude parabolic position of the foreground object in the background image in the parabolic process from the starting point to the ending point and a scaling ratio of the foreground object corresponding to each high-altitude parabolic position; and generating a high-altitude parabolic video image by scaling the foreground object according to the scaling ratio corresponding to each high-altitude parabolic position and fusing the scaled foreground object into the high-altitude parabolic position. The step of simulating the high-altitude parabolic position of the foreground object in the background image in the parabolic process from the starting point to the ending point and the scaling ratio of the foreground object corresponding to each high-altitude parabolic position comprises: obtaining a height h and a width w of the foreground object; cycling until the parabolic simulation ends when the high-altitude parabolic position meets a parabolic constraint condition; A horizontal initial velocity v0 and an acceleration a in the vertical direction are specified when the parabola is designated, and the starting point O0(x0, y0) and the ending point O s (x s ,y s ) of the foreground object in the background image are given, respectively, and the heights thereof are H0 and Hs, and H0 < Hs; Simulate the high-altitude parabolic position O1(x1, y1), O2(x2, y2)…, O i (x i ,y i ) of the foreground target in the background image from the first frame parabola to the i-th frame parabola in sequence and the height H1, H2, H3…, H i of the foreground target corresponding to each high-altitude parabolic position, obtain the scaling ratio b i =H i / h of the foreground target corresponding to the high-altitude parabolic position of each frame parabola. The step of obtaining the foreground object for synthesizing the high-altitude parabolic image comprises: obtaining a foreground image and extracting a foreground object in the foreground image for synthesizing the high-altitude parabolic image, calculating an outer rectangular frame of the foreground object, and recording a height h and a width w of the foreground object as a height and a width of the outer rectangular frame. Wherein, i≥1, the parabolic constraint condition is: x i+1 greater than the background image width, or y i+1 greater than the background image height, or y i+1 greater than y s , or x i+1 greater than x s ; Simulate the high-altitude parabolic position O of the foreground target in the background image at the i-th frame parabolic time i (x i ,y i ) and the height H of the foreground target i is: x i = x0+ v0*(i-1); y i = H0+ a(i - 1) 2 ; 2. The high-altitude throwback image compositing method of claim 1, wherein, The horizontal initial velocity v0 is 3, and the vertical acceleration a is 0.

2.

3. The high-altitude throwback image compositing method of claim 1, wherein, The step of scaling the foreground object according to the scaling ratio corresponding to each high-altitude parabolic position comprises: scaling the height and the width of the foreground object by the same ratio according to the scaling ratio.

4. The high-altitude throwback image compositing method of claim 1, wherein, The step of fusing the scaled foreground object into the high-altitude parabolic position comprises:

5. The high-altitude throwback image compositing method of claim 1, wherein, obtaining a center point of the scaled foreground object; pasting the scaled foreground object onto the background image and making the center of the foreground object coincide with the high-altitude parabolic position; and performing harmonization processing on the pasted foreground object and the background image. The step of performing harmonization processing on the pasted foreground object and the background image comprises:

6. The high-altitude throwback image compositing method of claim 5, wherein, adjusting the brightness of the region of the pasted foreground object: obtaining a brightness mean value L1 of the pasted foreground region and a brightness mean value L2 of the region of the background image occupied by the foreground object, and scaling the size of the pixel value of each pixel point of the pasted foreground region by a L2 / L1 ratio; smoothing the edge of the pasted foreground object: extracting the edge of the pasted foreground object and performing Gaussian blur processing on the transformation. The method for implementing the high-altitude parabolic image synthesis method according to any one of claims 1 to 6 comprises:

7. A high altitude ballistic synthesis apparatus, characterized in that, an image acquisition unit for obtaining the foreground object and the background image; a high-altitude parabolic simulation unit connected to the image acquisition unit and configured to mark a starting point and an ending point of a parabolic process of the foreground object in the background image, and simulate a high-altitude parabolic position of the foreground object in the background image in the parabolic process from the starting point to the ending point and a scaling ratio of the foreground object corresponding to each high-altitude parabolic position; and a high-altitude parabolic video image synthesis unit connected to the high-altitude parabolic simulation unit and configured to scale the foreground object according to the scaling ratio corresponding to each high-altitude parabolic position and fuse the scaled foreground object into the high-altitude parabolic position. ​ 8. A computer-readable storage medium, characterized in that, The computer readable storage medium is configured to store a computer program, which, when executed by a processor, is configured to implement the high-altitude parabolic image synthesis method according to any one of claims 1 to 6.

Images