Image generation method based on vanishing point, electronic equipment and storage medium
Through the image generation method based on vanishing points, the vanishing point is calculated and perspective blurred processing is performed, and the problem of difficult to take dynamic images with clear objects but blurred backgrounds in the prior art is solved, and user-friendly dynamic image shooting and perspective effects are realized.
Patent Information
- Application Number
- CN202410927601.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-08
AI Technical Summary
It is difficult to capture dynamic images with clear objects but blurred backgrounds in the prior art. Especially for users who do not have professional photography technology, it is difficult to achieve good shooting effects and affect user experience.
By using the image generation method based on vanishing points, the vanishing point is calculated by obtaining the motion direction of the target object, the ground reference point and the minimum external ellipse, and the image is perspective blurred to generate the second image of the target object, so that the pixels in the object area are The value is the same as the first image, and the pixel value outside the object area is the same as the perspective blur image.
You can take a dynamic image of the target object with clear but blurred background without professional photography, improve the user experience, and present the object's speeding effect through perspective blur effect.
Smart Images

Figure CN118918027A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to a vanishing point-based image generation method, electronic device, and storage medium. Background Art
[0002] With the widespread application of camera technology, people have become accustomed to using cameras (such as cameras on mobile phones) to capture images and record their lives. However, for sports photography, it is often necessary to have relatively professional photography techniques to achieve it. For example, if you want to capture a dynamic image of a moving target object with a clear object but a blurred background, you need to use relatively professional photography techniques such as follow-up shooting. This is quite difficult for people who do not have professional photography skills, and they cannot achieve a better shooting effect, which affects the user's experience of using the camera to capture images.
[0003] Accordingly, the art needs a new technical solution to solve the above problems. Summary of the invention
[0004] In order to overcome the above-mentioned defects, the present application is proposed to provide an image generation method, so that even if the user does not have professional photography skills, he can still take dynamic images with clear objects but blurred backgrounds, thereby improving the user experience.
[0005] In a first aspect, a vanishing point-based image generation method is provided, comprising: acquiring a first image of a target object, and acquiring an object region of the target object in the first image; acquiring a minimum circumscribed ellipse of the object region, and acquiring a movement direction of the target object according to the major axis direction of the minimum circumscribed ellipse; taking a ground reference point of the target object in the first image as a starting point, extending a first distance along the movement direction to obtain a vanishing point; processing the first image according to the vanishing point to obtain a perspective blurred image; generating a second image of the target object according to the first image, the object region and the perspective blurred image; wherein, in the second image, the pixel values of the pixels in the object region are the same as the pixel values of the pixels at the same position in the first image, and the pixel values of the pixels outside the object region are the same as the pixel values of the pixels at the same position in the perspective blurred image.
[0006] In a technical solution of the above-mentioned vanishing point-based image generation method, the ground reference point is a contact point where the target object in the first image contacts the ground.
[0007] In a technical solution of the above-mentioned vanishing point-based image generation method, the processing of the first image according to the vanishing point to obtain the perspective blurred image includes: performing convolution blur processing on the first image according to the vanishing point to obtain the perspective blurred image; wherein the pixel points in the perspective blurred image can form a plurality of straight lines intersecting at the vanishing point, and the pixel values of the pixel points of the straight line in the perspective blurred image are obtained by performing convolution blur processing on the pixel values of the pixel points of the straight line in the first image.
[0008] In a technical solution of the above-mentioned vanishing point-based image generation method, the first distance is obtained by: obtaining the length of the major axis of the minimum circumscribed ellipse; obtaining the perspective viewing angle of the first image, the perspective viewing angle being the angle between the major axis of the minimum circumscribed ellipse of the object area in the first image and the X-axis of the image coordinate system of the first image; obtaining a first ratio n1 corresponding to the perspective viewing angle, the first ratio n1 being in a negatively correlated corresponding relationship with the perspective viewing angle; and obtaining the first distance according to the product of the major axis length and the first ratio n1.
[0009] In a technical solution of the above-mentioned vanishing point-based image generation method, the method also includes obtaining the touchdown point in the following manner: obtaining the center of gravity of the object area; taking the center of gravity as the starting point, extending a second distance along the negative direction of the Y-axis of the image coordinate system of the first image to obtain the touchdown point.
[0010] In a technical solution of the above-mentioned vanishing point-based image generation method, the obtaining of the center of gravity of the object area includes: obtaining coordinates and grayscale values of pixels in the object area according to the first image, the coordinates including X-axis coordinates and Y-axis coordinates of the pixels in the image coordinate system; obtaining the sum of the grayscale values of all pixels in the object area; obtaining a first product of the X-axis coordinate of the pixel and the grayscale value, and obtaining the sum of the first products of all pixels; obtaining a second product of the Y-axis coordinate of the pixel and the grayscale value, and obtaining the sum of the second products of all pixels; obtaining the X-axis coordinate of the center of gravity in the image coordinate system according to the ratio between the sum of the first products and the sum of the grayscale values; and obtaining the Y-axis coordinate of the center of gravity in the image coordinate system according to the ratio between the sum of the second products and the sum of the grayscale values.
[0011] In a technical solution of the above-mentioned vanishing point-based image generation method, the second distance is obtained by: obtaining the short axis length of the minimum circumscribed ellipse; obtaining the perspective viewing angle of the first image, the perspective viewing angle being the angle between the long axis of the minimum circumscribed ellipse of the object area in the first image and the X-axis of the image coordinate system; obtaining a second ratio n2 corresponding to the perspective viewing angle, the second ratio n2 being positively correlated with the perspective viewing angle; and obtaining the second distance according to the product of the short axis length and the second ratio n2.
[0012] In a technical solution of the above-mentioned method for generating an image based on a vanishing point, after obtaining the vanishing point, the method further includes taking the Y-axis coordinate of the vanishing point in the image coordinate system of the first image as an initial Y-axis coordinate, and adjusting the initial Y-axis coordinate in the following manner: obtaining the axis length ratio between the minor axis length and the major axis length of the minimum circumscribed ellipse; respectively obtaining a third ratio n3 and a fourth ratio n4 corresponding to the axis length ratio, wherein the third ratio n3 is in a positively correlated corresponding relationship with the axis length ratio, and the fourth ratio n4 is in a negatively correlated corresponding relationship with the axis length ratio; obtaining a maximum value and a minimum value of the minor axis of the minimum circumscribed ellipse in the Y-axis of the image coordinate system, obtaining a third product between the maximum value and the third ratio n3, and obtaining a fourth product between the minimum value and the fourth ratio n4; selecting a larger one from the initial Y-axis coordinate and the third product as a first candidate Y-axis coordinate of the vanishing point; selecting a smaller one from the first candidate Y-axis coordinate and the fourth product as a second candidate Y-axis coordinate of the vanishing point; and obtaining a final Y-axis coordinate of the vanishing point according to the second candidate Y-axis coordinate.
[0013] In a technical solution of the above-mentioned vanishing point-based image generation method, the value range of the third ratio n3 is [0.95-2.15]; the value range of the fourth ratio n4 is [0.55-1.15].
[0014] In a technical solution of the above-mentioned method for generating an image based on a vanishing point, the step of obtaining the final Y-axis coordinate of the vanishing point according to the second candidate Y-axis coordinate includes: obtaining the aspect ratio of the target object according to the axis ratio; and obtaining the final Y-axis coordinate of the vanishing point according to the fifth product of the second candidate Y-axis coordinate and the aspect ratio. In a technical solution of the above-mentioned method for generating an image based on a vanishing point, the step of obtaining the aspect ratio of the target object according to the axis ratio includes: obtaining a first fitting function, the first fitting function being used to represent the corresponding relationship between the axis ratio and the aspect ratio of the target object; and using the axis ratio as an input of the first fitting function to obtain the aspect ratio of the target object corresponding to the axis ratio through the first fitting function.
[0015] In a technical solution of the above-mentioned vanishing point-based image generation method, the first fitting function is obtained in the following manner: using a camera to capture multiple target images of the target object at multiple different perspective angles; obtaining the axis length ratio between the short axis length and the long axis length of the minimum circumscribed ellipse of the object area in the target image, and obtaining the aspect ratio of the target object in the target image; performing nonlinear function fitting on the axis length ratio and the aspect ratio of the target object in the multiple target images to obtain a first fitting function; wherein the perspective angle is the angle between the long axis of the minimum circumscribed ellipse of the object area in the target image and the X-axis of the image coordinate system of the target image.
[0016] In a technical solution of the above-mentioned method for generating an image based on a vanishing point, after obtaining the vanishing point, the method further includes taking the X-axis coordinate of the vanishing point in the image coordinate system of the first image as the initial X-axis coordinate, and adjusting the initial X-axis coordinate in the following manner: obtaining the perspective viewing angle of the first image, where the perspective viewing angle is the angle between the major axis of the minimum circumscribed ellipse of the object area in the first image and the X-axis of the image coordinate system; obtaining a fifth ratio n5 corresponding to the perspective viewing angle, where the fifth ratio n5 is in a negatively correlated corresponding relationship with the perspective viewing angle; obtaining a fifth ratio n5 between the initial X-axis coordinate and the fifth ratio n5 six products; obtaining the major axis length, the lower limit value of the ratio, and the upper limit value of the ratio of the minimum circumscribed ellipse; obtaining the seventh product between the major axis length and the lower limit value of the ratio, and obtaining the eighth product between the major axis length and the upper limit value of the ratio; selecting the larger one from the sixth product and the seventh product as the candidate X-axis coordinate of the vanishing point; selecting the smaller one from the candidate X-axis coordinate and the eighth product as the final X-axis coordinate of the vanishing point; wherein the lower limit value of the ratio is the lower limit value of the magnification of the major axis length, and the upper limit value of the ratio is the upper limit value of the magnification of the major axis length.
[0017] In a technical solution of the above-mentioned vanishing point-based image generation method, the obtaining of the fifth ratio n5 corresponding to the perspective viewing angle includes: obtaining a second fitting function, where the second fitting function is used to represent the correspondence between the perspective viewing angle and the fifth ratio n5; using the perspective viewing angle of the first image as an input of the second fitting function, and obtaining the fifth ratio n5 corresponding to the perspective viewing angle through the second fitting function.
[0018] In a technical solution of the above-mentioned vanishing point-based image generation method, the second fitting function is obtained in the following manner: using a camera to collect multiple target images of the target object at multiple different perspective angles; taking the contact point where the target object in the target image contacts the ground as the starting point, extending a third distance along the movement direction of the target object to obtain a vanishing point; obtaining a first X-axis coordinate of the vanishing point in the image coordinate system of the target image; obtaining a second X-axis coordinate of the real vanishing point in the target image in the image coordinate system; obtaining a fifth ratio n5 according to the ratio of the second X-axis coordinate to the first X-axis coordinate; performing nonlinear function fitting on the perspective angles of the multiple target images and the fifth ratio n5 obtained according to the multiple target images to obtain a second fitting function; wherein the perspective angle is the angle between the major axis of the minimum circumscribed ellipse of the object area in the target image and the X-axis of the image coordinate system of the target image.
[0019] In a technical solution of the above-mentioned vanishing point-based image generation method, the convolution blur processing is performed on the first image according to the vanishing point to obtain a perspective blurred image, including: obtaining a convolution kernel of the convolution; obtaining a convolution area of a first pixel point in the first image according to the radius of the convolution kernel, the convolution area being an area centered on the first pixel point and having a radius the same as the radius of the convolution kernel, and the first pixel point being any pixel point in the first image; obtaining a second pixel point in the convolution area that is located on the same straight line as the first pixel point and the vanishing point; obtaining an average pixel value of the first pixel point and all second pixel points; and using the average pixel value as the pixel value of the first pixel point in the perspective blurred image.
[0020] In a technical solution of the above-mentioned vanishing point-based image generation method, the obtaining of a second pixel point in the convolution area that is located on the same straight line as the first pixel point and the vanishing point includes: obtaining a first vector between the first pixel point and the vanishing point, obtaining a first angle between the first vector and an X-axis of an image coordinate system of the first image; and obtaining the second pixel point according to the first angle; wherein a second angle corresponding to the second pixel point is the same as the first angle, the second angle is the angle between a second vector corresponding to the second pixel point and the X-axis of the image coordinate system, and the second vector is a vector between the second pixel point and the vanishing point.
[0021] In a technical solution of the above-mentioned vanishing point-based image generation method, the obtaining of the second pixel point according to the first angle includes: respectively obtaining the cosine value and the sine value of the first angle; obtaining the step size dr of the i-th second pixel point in the convolution area, dr=i×k, k<1, i=[-r,r], and r is the radius of the convolution kernel; obtaining the X-axis offset according to the product between the cosine value and the step size dr, and obtaining the Y-axis offset according to the product between the sine value and the step size dr; obtaining the X-axis coordinate of the i-th second pixel point in the image coordinate system according to the sum of the X-axis coordinate of the first pixel point in the image coordinate system and the X-axis offset; obtaining the Y-axis coordinate of the i-th second pixel point in the image coordinate system according to the sum of the Y-axis coordinate of the first pixel point in the image coordinate system and the Y-axis offset.
[0022] In a second aspect, an electronic device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned vanishing point-based image generation method is implemented.
[0023] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned vanishing point-based image generation method.
[0024] Solution 1. A method for generating an image based on a vanishing point, characterized in that the method comprises: obtaining a first image of a target object, and obtaining an object area of the target object in the first image; obtaining a minimum circumscribed ellipse of the object area, and obtaining a movement direction of the target object according to the major axis direction of the minimum circumscribed ellipse; taking a ground reference point of the target object in the first image as a starting point, extending a first distance along the movement direction to obtain a vanishing point; processing the first image according to the vanishing point to obtain a perspective blurred image; generating a second image of the target object according to the first image, the object area and the perspective blurred image; wherein, in the second image, the pixel values of the pixels in the object area are the same as the pixel values of the pixels at the same position in the first image, and the pixel values of the pixels outside the object area are the same as the pixel values of the pixels at the same position in the perspective blurred image.
[0025] Solution 2. The method according to Solution 1 is characterized in that the ground reference point is the touchdown point where the target object in the first image contacts the ground.
[0026] Solution 3. The method according to Solution 1 is characterized in that the processing of the first image according to the vanishing point to obtain the perspective blurred image includes: performing convolution blur processing on the first image according to the vanishing point to obtain the perspective blurred image; the pixel points in the perspective blurred image can form a plurality of straight lines intersecting at the vanishing point, and the pixel values of the pixel points of the straight line in the perspective blurred image are obtained by performing convolution blur processing on the pixel values of the pixel points of the straight line in the first image.
[0027] Solution 4. The method according to Solution 1 is characterized in that the first distance is obtained by: obtaining the length of the major axis of the minimum circumscribed ellipse; obtaining the perspective viewing angle of the first image, the perspective viewing angle being the angle between the major axis of the minimum circumscribed ellipse of the object area in the first image and the X-axis of the image coordinate system of the first image; obtaining a first ratio n1 corresponding to the perspective viewing angle, the first ratio n1 being negatively correlated with the perspective viewing angle; and obtaining the first distance based on the product of the major axis length and the first ratio n1.
[0028] Solution 5. The method according to Solution 1 is characterized in that the method also includes obtaining the touchdown point in the following manner: obtaining the center of gravity of the object area; taking the center of gravity as the starting point, extending a second distance along the negative direction of the Y-axis of the image coordinate system of the first image to obtain the touchdown point.
[0029] Solution 6. The method according to Solution 5 is characterized in that obtaining the center of gravity of the object area includes: obtaining the coordinates and grayscale values of the pixel points in the object area according to the first image, the coordinates including the X-axis coordinates and Y-axis coordinates of the pixel points in the image coordinate system; obtaining the sum of the grayscale values of all the pixels in the object area; obtaining the first product of the X-axis coordinate of the pixel point and the grayscale value, and obtaining the sum of the first products of all the pixels; obtaining the second product of the Y-axis coordinate of the pixel point and the grayscale value, and obtaining the sum of the second products of all the pixels; obtaining the X-axis coordinate of the center of gravity in the image coordinate system according to the ratio between the sum of the first products and the sum of the grayscale values; and obtaining the Y-axis coordinate of the center of gravity in the image coordinate system according to the ratio between the sum of the second products and the sum of the grayscale values.
[0030] Solution 7. The method according to Solution 5 is characterized in that the second distance is obtained by: obtaining the short axis length of the minimum circumscribed ellipse; obtaining the perspective viewing angle of the first image, the perspective viewing angle being the angle between the long axis of the minimum circumscribed ellipse of the object area in the first image and the X-axis of the image coordinate system; obtaining a second ratio n2 corresponding to the perspective viewing angle, the second ratio n2 being positively correlated with the perspective viewing angle; and obtaining the second distance based on the product of the short axis length and the second ratio n2.
[0031] Solution 8. The method according to Solution 1 is characterized in that, after the vanishing point is obtained, the method further includes taking the Y-axis coordinate of the vanishing point in the image coordinate system of the first image as the initial Y-axis coordinate, and adjusting the initial Y-axis coordinate in the following manner: obtaining the axis length ratio between the minor axis length and the major axis length of the minimum circumscribed ellipse; respectively obtaining a third ratio n3 and a fourth ratio n4 corresponding to the axis length ratio, wherein the third ratio n3 is in a positively correlated corresponding relationship with the axis length ratio, and the fourth ratio n4 is in a negatively correlated corresponding relationship with the axis length ratio; obtaining the maximum value and the minimum value of the minor axis of the minimum circumscribed ellipse in the Y-axis of the image coordinate system, obtaining a third product between the maximum value and the third ratio n3, and obtaining a fourth product between the minimum value and the fourth ratio n4; selecting the larger one from the initial Y-axis coordinate and the third product as the first candidate Y-axis coordinate of the vanishing point; selecting the smaller one from the first candidate Y-axis coordinate and the fourth product as the second candidate Y-axis coordinate of the vanishing point; and obtaining the final Y-axis coordinate of the vanishing point according to the second candidate Y-axis coordinate.
[0032] Solution 9. The method according to Solution 8 is characterized in that the numerical range of the third ratio n3 is [0.95-2.15]; the numerical range of the fourth ratio n4 is [0.55-1.15].
[0033] Solution 10. The method according to Solution 8 is characterized in that obtaining the final Y-axis coordinate of the vanishing point based on the second candidate Y-axis coordinate includes: obtaining the aspect ratio of the target object based on the axis length ratio; and obtaining the final Y-axis coordinate of the vanishing point based on the fifth product between the second candidate Y-axis coordinate and the aspect ratio.
[0034] Scheme 11. The method according to Scheme 10 is characterized in that obtaining the aspect ratio of the target object based on the axis length ratio includes: obtaining a first fitting function, the first fitting function being used to represent the correspondence between the axis length ratio and the aspect ratio of the target object; using the axis length ratio as an input quantity of the first fitting function, and obtaining the aspect ratio of the target object corresponding to the axis length ratio through the first fitting function.
[0035] Solution 12. The method according to Solution 11 is characterized in that the first fitting function is obtained in the following manner: using a camera to capture multiple target images of the target object at multiple different perspective angles; obtaining the axis length ratio between the short axis length and the long axis length of the minimum circumscribed ellipse of the object area in the target image, and obtaining the aspect ratio of the target object in the target image; performing nonlinear function fitting on the axis length ratio and the aspect ratio of the target object in the multiple target images to obtain a first fitting function; wherein the perspective angle is the angle between the long axis of the minimum circumscribed ellipse of the object area in the target image and the X-axis of the image coordinate system of the target image.
[0036] Solution 13. The method according to Solution 1 is characterized in that, after obtaining the vanishing point, the method further comprises taking the X-axis coordinate of the vanishing point in the image coordinate system of the first image as the initial X-axis coordinate, and adjusting the initial X-axis coordinate by: obtaining the perspective viewing angle of the first image, where the perspective viewing angle is the angle between the major axis of the minimum circumscribed ellipse of the object area in the first image and the X-axis of the image coordinate system; obtaining a fifth ratio n5 corresponding to the perspective viewing angle, where the fifth ratio n5 is in a negatively correlated corresponding relationship with the perspective viewing angle; obtaining a sixth ratio between the initial X-axis coordinate and the fifth ratio n5 product; obtaining the major axis length, the lower limit value of the ratio, and the upper limit value of the ratio of the minimum circumscribed ellipse; obtaining the seventh product between the major axis length and the lower limit value of the ratio, and obtaining the eighth product between the major axis length and the upper limit value of the ratio; selecting the larger one from the sixth product and the seventh product as the candidate X-axis coordinate of the vanishing point; selecting the smaller one from the candidate X-axis coordinate and the eighth product as the final X-axis coordinate of the vanishing point; wherein the lower limit value of the ratio is the lower limit value of the magnification of the major axis length, and the upper limit value of the ratio is the upper limit value of the magnification of the major axis length.
[0037] Scheme 14. The method according to Scheme 13 is characterized in that the method of obtaining the fifth ratio n5 corresponding to the perspective viewing angle includes: obtaining a second fitting function, wherein the second fitting function is used to represent the correspondence between the perspective viewing angle and the fifth ratio n5; using the perspective viewing angle of the first image as an input of the second fitting function, and obtaining the fifth ratio n5 corresponding to the perspective viewing angle through the second fitting function.
[0038] Solution 15. The method according to Solution 14 is characterized in that the second fitting function is obtained in the following manner: using a camera to collect multiple target images of the target object at multiple different perspective angles; taking the contact point where the target object in the target image contacts the ground as the starting point, extending a third distance along the movement direction of the target object to obtain a vanishing point; obtaining the first X-axis coordinate of the vanishing point in the image coordinate system of the target image; obtaining the second X-axis coordinate of the real vanishing point in the target image in the image coordinate system; obtaining a fifth ratio n5 according to the ratio of the second X-axis coordinate to the first X-axis coordinate; performing nonlinear function fitting on the perspective angles of the multiple target images and the fifth ratio n5 obtained according to the multiple target images to obtain a second fitting function; wherein the perspective angle is the angle between the major axis of the minimum circumscribed ellipse of the object area in the target image and the X-axis of the image coordinate system of the target image.
[0039] Scheme 16. The method according to Scheme 1 is characterized in that the first image is subjected to convolution blur processing according to the vanishing point to obtain a perspective blurred image, including: obtaining a convolution kernel of the convolution; obtaining a convolution area of a first pixel point in the first image according to the radius of the convolution kernel, the convolution area being an area centered on the first pixel point and having a radius the same as the radius of the convolution kernel, the first pixel point being any pixel point in the first image; obtaining a second pixel point in the convolution area that is on the same straight line as the first pixel point and the vanishing point; obtaining an average pixel value of the first pixel point and all second pixel points; and using the average pixel value as the pixel value of the first pixel point in the perspective blurred image.
[0040] Solution 17. The method according to Solution 16 is characterized in that obtaining a second pixel point in the convolution area that is located on the same straight line as the first pixel point and the vanishing point includes: obtaining a first vector between the first pixel point and the vanishing point, obtaining a first angle between the first vector and the X-axis of the image coordinate system of the first image; obtaining the second pixel point according to the first angle; wherein the second angle corresponding to the second pixel point is the same as the first angle, the second angle is the angle between the second vector corresponding to the second pixel point and the X-axis of the image coordinate system, and the second vector is the vector between the second pixel point and the vanishing point.
[0041] Scheme 18. The method according to Scheme 17 is characterized in that obtaining the second pixel point according to the first angle includes: respectively obtaining the cosine value and the sine value of the first angle; obtaining the step size dr of the i-th second pixel point in the convolution area, dr=i×k, k<1, i=[-r,r], r is the radius of the convolution kernel; obtaining the X-axis offset according to the product of the cosine value and the step size dr, and obtaining the Y-axis offset according to the product of the sine value and the step size dr; obtaining the X-axis coordinate of the i-th second pixel point in the image coordinate system according to the sum of the X-axis coordinate of the first pixel point in the image coordinate system and the X-axis offset; obtaining the Y-axis coordinate of the i-th second pixel point in the image coordinate system according to the sum of the Y-axis coordinate of the first pixel point in the image coordinate system and the Y-axis offset.
[0042] Solution 19. An electronic device, characterized in that it includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the vanishing point-based image generation method described in any one of Solutions 1 to 18 is implemented.
[0043] Solution 20. A computer-readable storage medium storing a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the vanishing point-based image generation method described in any one of Solutions 1 to 18.
[0044] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:
[0045] In a technical solution for implementing the vanishing point-based image generation method provided in the present application, a first image of a target object can be obtained, and an object area of the target object in the first image can be obtained; a minimum circumscribed ellipse of the object area is obtained, and a movement direction of the target object is obtained according to the major axis direction of the minimum circumscribed ellipse; a ground reference point of the target object in the first image is used as a starting point, and a first distance is extended along the movement direction to obtain a vanishing point; the first image is processed according to the vanishing point to obtain a perspective blurred image; a second image of the target object is generated according to the first image, the object area, and the perspective blurred image; wherein, in the second image, the pixel values of the pixels in the object area are the same as the pixel values of the pixels at the same position in the first image, and the pixel values of the pixels outside the object area are the same as the pixel values of the pixels at the same position in the perspective blurred image.
[0046] The second image can be understood as a dynamic image in which the target object is clear but the background is blurred. For users, there is no need to have professional photography skills. As long as the camera captures an image containing the target object, the dynamic image of the target object can be obtained conveniently and accurately through the above method, which greatly improves the user experience. In addition, the blur information in the second image is obtained by using the pixel values of the pixels in the perspective blurred image. Therefore, the second image can present the perspective effect of the target object flying. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:
[0048] Figure 1 is a schematic flow chart of main steps of a method for generating an image based on a vanishing point according to an embodiment of the present application;
[0049] Figure 2 is a schematic diagram of a first image of a vehicle captured by a camera according to an embodiment of the present application;
[0050] Figure 3 is from Figure 2 A schematic diagram of a vehicle body region obtained in ;
[0051] Figure 4 yes Figure 3 Schematic diagram of the minimum circumscribed ellipse of the vehicle body area;
[0052] Figure 5 yes Figure 2 A schematic diagram showing a contact point of the vehicle in the first image and a vanishing point obtained using the contact point;
[0053] Figure 6 is to use Figure 2 A schematic diagram of a second image obtained from the first image shown;
[0054] Figure 7 is a schematic flow chart of main steps for obtaining a first distance according to an embodiment of the present application;
[0055] Figure 8 is a schematic flow chart of the main steps of obtaining the touchdown point according to an embodiment of the present application;
[0056] Fig. 9 yes Figure 2 A schematic diagram showing the center of gravity of the vehicle in the first image and a touchdown point obtained using the center of gravity;
[0057] Fig.10 is a schematic flow chart of main steps of adjusting the Y-axis coordinate of a vanishing point in an image coordinate system of a first image according to an embodiment of the present application;
[0058] Fig.11 is a schematic flow chart of main steps of adjusting the X-axis coordinate of a vanishing point in an image coordinate system of a first image according to an embodiment of the present application;
[0059] Fig.12 is a schematic flow chart of main steps of performing convolution blur processing on a first image according to a vanishing point according to an embodiment of the present application;
[0060] Fig.13 is a schematic flow chart of main steps of a method for generating an image based on a vanishing point according to another embodiment of the present application;
[0061] Fig.14 It is a schematic diagram of the main structure of an electronic device according to an embodiment of the present application.
[0062] Reference numerals: 11: image sensor; 12: processor; 13: memory. DETAILED DESCRIPTION
[0063] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.
[0064] In the description of this application, "processor" may include hardware, software or a combination of the two. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc.
[0065] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.
[0066] The user personal information processed by this application will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. This application will treat the user's personal information and its processing with a high degree of diligence.
[0067] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.
[0068] The following is an explanation of an embodiment of the image generation method based on vanishing points provided by the present application. Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a method for generating an image based on a vanishing point according to an embodiment of the present application. Figure 1 As shown, the vanishing point-based image generation method in the embodiment of the present application mainly includes the following steps S101 to S105.
[0069] Step S101: acquiring a first image of a target object, and acquiring an object region of the target object in the first image.
[0070] The first image can be an image captured in real time by a camera on an electronic device, or an image stored in an image library of the electronic device. The images stored in the image library can be historical images captured by a camera, or images sent to the electronic device by other devices, or images downloaded from a web page or APP by the electronic device, etc. The target object can be understood as the object that the user wants to achieve a flying perspective effect. In the following embodiments, objects refer to target objects, and object areas refer to object areas of target objects.
[0071] In this embodiment, a conventional semantic segmentation model can be used to perform semantic segmentation on the first image to obtain the area where the target object is located in the first image. In some embodiments, a semantic segmentation model deepv3 plus can be used. The target object may include a moving vehicle, a motorcycle, a bicycle, a running pedestrian or an animal, and other movable objects, which are not limited in this application.
[0072] See attached Figure 2 and attached Figure 3 , Figure 2 The first image of the vehicle captured by the camera is shown as an example. Figure 3 An example is shown from Figure 2 The body area of the vehicle obtained in .
[0073] Step S102: obtaining the minimum circumscribed ellipse of the object area, and obtaining the moving direction of the target object according to the major axis direction of the minimum circumscribed ellipse.
[0074] The minimum circumscribed ellipse of the object region refers to the ellipse with the smallest area among the ellipse set circumscribed to the object region. In this embodiment, a conventional method for obtaining the minimum circumscribed ellipse can be used to obtain the minimum circumscribed ellipse of the object region according to the position information of the object region. Figure 4 , Figure 4 The minimum circumscribed ellipse of the vehicle body area is exemplarily shown, wherein AB represents two end points of the major axis of the minimum circumscribed ellipse, and CD represents two end points of the minor axis of the minimum circumscribed ellipse.
[0075] The direction of movement of the object is parallel to the major axis of the minimum circumscribed ellipse. When obtaining the direction of movement of the object, the two endpoints of the major axis of the minimum circumscribed ellipse can be obtained, including the tail endpoint and the head endpoint, and the direction from the tail endpoint to the head endpoint is taken as the direction of movement. The tail endpoint is the endpoint located at the tail of the object, and the head endpoint is the endpoint located at the head of the object. When semantic segmentation is performed on the first image to obtain the area where the object is located, the head and tail of the object can also be identified. Therefore, based on the head and tail of the object, it can be determined which of the two endpoints of the major axis of the minimum circumscribed ellipse is the tail endpoint and which is the head endpoint. Figure 4 As shown, the direction of movement of the vehicle is the direction from end point B to end point A.
[0076] Step S103: Taking the ground reference point of the target object in the first image as the starting point, extending the first distance along the moving direction to obtain a vanishing point.
[0077] The target object is located on the ground. According to the relationship between the target object and the ground, the ground is divided into two parts, one part is the first ground directly below the target object, and the other part is the second ground outside the target object. The ground reference point is a point that can represent the position of the first ground in the first image. The ground reference point can be any ground point in the first ground, that is, the position of the first ground is represented by the position of any ground point; the ground reference point can also be an object point on the target object that is adjacent to the first ground position, that is, the position of the object point is represented by the position of the first ground. In some embodiments, the contact point where the target object contacts the ground can be obtained as the ground reference point, and this contact point is a point located on the first ground.
[0078] The vanishing point can be understood as the point where the line of sight of the object, viewed from the direction of the object's movement, intersects with the parallel line of sight. Figure 5 , Figure 5 The contact point of the vehicle and the vanishing point obtained using the contact point are exemplarily shown.
[0079] Since the vanishing point is the visual intersection point with the parallel lines, the vanishing point is usually located on the ground in front of the object, rather than on the ground directly below the object. The ground reference point represents the first ground directly below the object, so the ground reference point can be extended along the direction of movement by a first distance to obtain the vanishing point, so that the vanishing point is located on the ground in front of the object.
[0080] The length of the major axis of the minimum circumscribed ellipse of the object area is greater than or equal to the length of the ground directly below the object (i.e., the length of the orthographic projection of the major axis of the minimum circumscribed ellipse on the X-axis in the image coordinate system of the first image). Therefore, when determining the value of the first distance, it can be determined based on the length of the major axis of the minimum circumscribed ellipse. Specifically, the value of the first distance can be set to a value greater than the above-mentioned preset value of the major axis length, that is, the first distance is the sum of the above-mentioned major axis length and the preset value. When setting the preset value, a large number of different images can be obtained, and the difference between the major axis length of the minimum circumscribed ellipse of the object area in these images and the length of the ground directly below the object is counted, and the preset value is set according to the maximum difference, so that the preset value is greater than this maximum difference.
[0081] Step S104: Processing the first image according to the vanishing point to obtain a perspective blurred image.
[0082] In some embodiments, the first image may be subjected to convolution blur processing according to the vanishing point to obtain a perspective blurred image. Specifically, the pixels in the perspective blurred image can form a plurality of straight lines intersecting at the vanishing point. For each straight line, the pixel value of the pixel of the straight line in the perspective blurred image is obtained by performing convolution blur processing on the pixel value of the pixel of the straight line in the first image.
[0083] In this embodiment, a conventional convolution blur processing method can be used to blur the pixel values of the pixel points of the straight line in the first image to obtain blurred pixel values, that is, the pixel values of these pixels in the perspective blurred image.
[0084] Step S105: generating a second image of the target object according to the first image, the object region and the perspective blurred image.
[0085] In the second image, the pixel values of the pixels in the object area are the same as the pixel values of the pixels at the same position in the first image, and the pixel values of the pixels outside the object area are the same as the pixel values of the pixels at the same position in the perspective blurred image. Specifically, a first pixel in the object area can be obtained from the first image, and a second pixel outside the object area can be obtained from the perspective blurred image, and the second image can be generated based on the first and second pixel points.
[0086] In some implementations, the first image, the object region, and the perspective blurred image may be nonlinearly mixed using the following formula (1) to obtain a second image.
[0087] Mout[i, j] = Msrc[i, j] * Mm[i, j] + Mp[i, j] * (255- Mm[i, j]) (1)
[0088] The meanings of the parameters in formula (1) are as follows:
[0089] Mout represents the second image, Msrc represents the first image, Mm represents the object area, Mp represents the perspective blurred image, [i, j] represents the pixel point in the i-th row and j-th column, i = 0 ~ H, j = 0 ~ W, H represents the image height, and W represents the image width. The second image, the first image, and the perspective blurred image have the same height and width.
[0090] See attached Figure 6 , Figure 6 The second image of the vehicle is shown as an example. Figure 6 As shown, the pixels within the vehicle body area are not blurred, while the pixels outside the vehicle body area are blurred, and a perspective effect of a speeding vehicle is presented.
[0091] Based on the method described in steps S101 to S106, the user does not need to have professional photography skills. He only needs to capture an image containing an object through a camera, and can obtain a dynamic image of the object conveniently and accurately through the method, which greatly improves the user experience. In addition, the blur information in the second image is obtained by using the pixel values of the pixels in the perspective blur image. Therefore, the second image can present a perspective effect of the object flying.
[0092] The above steps S103 to S104 are described below.
[0093] 1. Explain step S103.
[0094] In some implementations of the above step S103, Figure 7 The following steps S1031 to S1034 shown are used to obtain the first distance.
[0095] Step S1031: Obtain the major axis length of the minimum circumscribed ellipse of the object area. Figure 4 As shown, the major axis length of the vehicle's minimum circumscribed ellipse can be expressed as |AB|.
[0096] Step S1032: Acquire the perspective viewing angle of the first image.
[0097] The perspective angle is the angle between the major axis of the smallest circumscribed ellipse of the object area in the first image and the X axis of the image coordinate system of the first image. Figure 4 As shown, θ represents the perspective viewing angle.
[0098] Step S1033: Obtain a first ratio n1 corresponding to the perspective viewing angle.
[0099] Step S1034: Obtain a first distance according to the product of the major axis length and the first ratio n1. Specifically, the product can be used as the first distance.
[0100] The first ratio n1 is negatively correlated with the perspective angle, that is, the larger the perspective angle, the smaller the first ratio n1, and vice versa.
[0101] The following is combined with Figure 4 The corresponding relationship of the above negative correlation is explained.
[0102] like Figure 4 As shown, if the perspective angle θ=0°, then the first length of the major axis of the minimum circumscribed ellipse and the second length of the orthographic projection of this major axis onto the X-axis are equal; if the perspective angle θ>0°, then the first length is greater than the second length, and the larger the perspective angle θ, the greater the deviation between the first length and the second length.
[0103] Since the first ratio n1 is used to multiply the length of the major axis, the product is used as the first distance, that is, the first ratio n1 is a multiple of the length of the major axis. If the deviation between the first length and the second length is relatively large, then a smaller first ratio n1 can be set; if the deviation between the first length and the second length is relatively small, then a larger first ratio n1 can be set. Furthermore, since the larger the perspective angle θ, the larger the deviation between the first length and the second length, it can be determined that the larger the perspective angle θ, the smaller the first ratio n1, and the smaller the perspective angle θ, the larger the first ratio n1, that is, the first ratio n1 is negatively correlated with the perspective angle.
[0104] In some embodiments, a large number of different images can be obtained, and the distance between the vanishing point and the object's touchdown point in these images, as well as the length of the major axis of the minimum circumscribed ellipse of the object area in these images and the length of the major axis's orthographic projection on the image coordinate system's X-axis are analyzed to determine a numerical range of the above-mentioned first ratio n1. Within this numerical range, the larger the numerical value, the smaller the corresponding perspective angle, and the smaller the numerical value, the larger the corresponding perspective angle. When it is necessary to obtain the first ratio n1, the perspective angle of the first image can be obtained first, and then a first ratio n1 can be selected from this numerical range according to the size of the perspective angle. Taking the object as a vehicle as an example, in some embodiments, the numerical range of the first ratio n1 can be [1.2-6.0].
[0105] Based on the method described in steps S1031 to S1034 above, the first distance can be acquired conveniently and accurately by utilizing the perspective viewing angle of the first image.
[0106] In some implementations of the above step S103, Figure 8 The following steps S1035 to S1036 shown in the figure are used to obtain the contact point between the object and the ground in the first image.
[0107] Step S1035: Obtain the centroid of the object area.
[0108] Step S1036: Taking the center of gravity as the starting point, extend the second distance along the negative direction of the Y axis of the image coordinate system of the first image to obtain the touchdown point. Fig. 9 , Fig. 9 The center of gravity of the vehicle and the contact point obtained using the center of gravity are exemplarily shown.
[0109] When determining the value of the second distance, a large number of different images can be obtained, and the vertical distance between the center of gravity of the object area in the image and the touchdown point (the distance along the negative direction of the Y axis of the image coordinate system) is counted, and a value is determined as the second distance based on the statistics. For example, the average value of the above distances is used as the second distance.
[0110] Based on the methods described in steps S1035 to S1036 above, the touchdown point of the object can be accurately obtained using the center of gravity of the object area.
[0111] The above-mentioned step S1035 and step S1036 are explained below.
[0112] (i) Explain step S1035.
[0113] In some implementations of the above step S1035, the center of gravity of the object area can be obtained through the following steps 11 to 16.
[0114] Step 11: According to the first image, the coordinates and grayscale values of the pixels in the object area are obtained. The coordinates include the X-axis coordinates and Y-axis coordinates of the pixels in the image coordinate system.
[0115] Step 12: Get the sum of the grayscale values of all pixels in the object area.
[0116] Step 13: Obtain the first product of the X-axis coordinate of the pixel and the grayscale value, and obtain the sum of the first products of all pixels.
[0117] Step 14: Obtain the second product of the Y-axis coordinate of the pixel and the grayscale value, and obtain the sum of the second products of all pixels.
[0118] Step 15: According to the ratio between the sum of the first products and the sum of the gray values, obtain the X-axis coordinate of the center of gravity in the image coordinate system.
[0119] The X-axis coordinate of the center of gravity is shown in the following formula (2).
[0120]
[0121] The meanings of the parameters in formula (2) are as follows:
[0122] Q x Indicates the X-axis coordinate of the center of gravity, M m represents the object area, (x, y) represents the coordinates of the pixel in the image coordinate system, x and y represent the X-axis coordinate and Y-axis coordinate respectively, and f(x, y) represents the grayscale value of the pixel at the (x, y) position.
[0123] Step 16: According to the ratio between the sum of the second products and the sum of the gray values, obtain the Y-axis coordinate of the center of gravity in the image coordinate system.
[0124] The Y-axis coordinate of the center of gravity is shown in the following formula (3).
[0125]
[0126] Q in formula (3) xRepresents the Y-axis coordinate of the center of gravity. Other meanings are the same as those of the parameters in formula (2).
[0127] Based on the methods described in steps 11 to 16 above, the center of gravity of the object area can be obtained conveniently and accurately using the grayscale value of the pixel point.
[0128] (ii) Explain step S1036.
[0129] In some implementations of the above step S1036, the second distance may be acquired through the following steps 21 to 24.
[0130] Step 21: Get the minor axis length of the minimum circumscribed ellipse of the object area.
[0131] Step 22: Obtain the perspective viewing angle of the first image. The perspective viewing angle is the angle between the major axis of the minimum circumscribed ellipse of the object region in the first image and the X-axis of the image coordinate system.
[0132] Step 23: Obtain a second ratio n2 corresponding to the perspective viewing angle.
[0133] Step 24: Obtain a second distance according to the product of the minor axis length and the second ratio n2. Specifically, the product is used as the second distance.
[0134] The second ratio n2 is in a positively correlated corresponding relationship with the perspective angle, that is, the larger the perspective angle is, the larger the second ratio n2 is, and vice versa.
[0135] The following is combined with Figure 4 The corresponding relationship of the above positive correlation is explained.
[0136] like Figure 4 As shown, if the perspective angle θ is larger, the center of gravity is higher, that is, the vertical distance between the center of gravity and the ground (the distance along the Y axis of the image coordinate system) is larger, otherwise the center of gravity is lower and the vertical distance between the center of gravity and the ground is smaller.
[0137] Since the second ratio n2 is used to multiply the length of the minor axis, the product is used as the second distance, that is, the second ratio n2 is a multiple of the length of the minor axis. If the center of gravity is relatively high, a larger second ratio n2 can be set; if the center of gravity is relatively low, a smaller second ratio n2 can be set. Furthermore, since the larger the perspective angle θ, the higher the center of gravity, and the smaller the perspective angle θ, the lower the center of gravity, it can be determined that the larger the perspective angle θ, the larger the second ratio n2, and the smaller the perspective angle θ, the smaller the second ratio n2, that is, the second ratio n2 is positively correlated with the perspective angle.
[0138] In some embodiments, a large number of different images can be obtained, and the vertical distance between the center of gravity and the touchdown point in these images, as well as the minor axis length of the minimum circumscribed ellipse of the object in these images are analyzed to determine a numerical range of the second ratio n2. Within this numerical range, the larger the numerical value, the larger the corresponding perspective angle, and the smaller the numerical value, the smaller the corresponding perspective angle. When it is necessary to obtain the second ratio n2, the perspective angle of the first image can be obtained first, and then a second ratio n2 can be selected from this numerical range according to the size of the perspective angle. Taking the object as a vehicle as an example, in some embodiments, the numerical range of the second ratio n2 can be [0.4-1.2].
[0139] Based on the methods described in steps 21 to 24 above, the second distance can be acquired conveniently and accurately by utilizing the perspective viewing angle of the first image.
[0140] In some implementations of the above step S103, after the vanishing point is obtained, the Y-axis coordinate of the vanishing point in the image coordinate system of the first image can be used as the initial Y-axis coordinate, and Fig.10 The following steps S201 to S207 shown adjust the initial Y-axis coordinate.
[0141] Step S201: Obtain the axis length ratio between the minor axis length and the major axis length of the minimum circumscribed ellipse.
[0142] Step S202: respectively obtaining a third ratio n3 and a fourth ratio n4 corresponding to the axis length ratio.
[0143] The third ratio n3 is positively correlated with the axial length ratio, that is, the larger the axial length ratio is, the larger the third ratio n3 is, and vice versa.
[0144] The fourth ratio n4 is negatively correlated with the axial length ratio, that is, the larger the axial length ratio is, the smaller the fourth ratio n4 is, and vice versa.
[0145] Step S203: Obtain the maximum and minimum values of the minor axis of the minimum circumscribed ellipse on the Y axis of the image coordinate system.
[0146] Step S204: obtaining a third product between the maximum value and the third ratio n3, and obtaining a fourth product between the minimum value and the fourth ratio n4.
[0147] Step S205: selecting a larger one between the initial Y-axis coordinate and the third product as the first candidate Y-axis coordinate of the vanishing point.
[0148] Since the third ratio n3 is positively correlated with the axis length ratio, the third product is a relatively large value. By selecting the larger one between the initial Y-axis coordinate and the third product, the maximum value constraint of the Y-axis coordinate of the vanishing point can be achieved, so that the Y-axis coordinate of the vanishing point will not be too small. If the Y-axis coordinate is too small, it is easy to produce a distortion feeling of the object warping up when presenting the perspective effect of the object flying on the second image.
[0149] The first candidate Y-axis coordinate can be expressed as follows (4).
[0150] Vy1=max(Vy0,n0×Cy) (4)
[0151] The meanings of the parameters in formula (4) are as follows:
[0152] Vy1 represents the first candidate Y-axis coordinate, Vy0 represents the initial Y-axis coordinate, n0 represents the third ratio n3, and Cy represents the maximum value of the minor axis of the minimum circumscribed ellipse. Figure 4 As shown, Cy can also be understood as the coordinate of the endpoint C on the Y axis of the image coordinate system.
[0153] In some embodiments, a large number of different images can be obtained, and the vanishing points, the above-mentioned axis length ratios of the minimum circumscribed ellipse of the object, etc. in these images are analyzed to determine a numerical range of the above-mentioned third ratio n3. In this numerical range, the larger the numerical value, the larger the axis length ratio of the vanishing point, and the smaller the numerical value, the smaller the axis length ratio. When it is necessary to obtain the third ratio n3, the above-mentioned axis length ratio of the minimum circumscribed ellipse of the object can be obtained first, and then a third ratio n3 can be selected from this numerical range according to the size of the axis length ratio. Taking the object as an example, in some embodiments, the numerical range of the third ratio n3 can be [0.95-2.15]. If the Y-axis coordinate of the vanishing point is relatively small, it may cause the highest point of the vehicle (i.e., the maximum value of the short axis of the minimum circumscribed ellipse on the Y-axis of the image coordinate system) to be greater than the Y-axis coordinate of the vanishing point and the deviation between the two is relatively large, resulting in a sense of the vehicle being far above the vanishing point. In this regard, after repeated research and testing, the inventors can select a third ratio n3 corresponding to the vehicle's axle length ratio based on the numerical range of the third ratio n3 [0.95-2.15], and use the third product between the selected third ratio n3 and the highest point of the vehicle as a possible Y-axis coordinate of a vanishing point. If the Y-axis coordinate of the vanishing point is relatively small, the third product can be used as the Y-axis coordinate of the vanishing point to avoid the distortion of the vehicle tilting upward due to the relatively small Y-axis coordinate of the vanishing point.
[0154] Step S206: Selecting a smaller one between the first candidate Y-axis coordinate and the fourth product as the second candidate Y-axis coordinate of the vanishing point.
[0155] Since the fourth ratio n4 is negatively correlated with the above-mentioned axis length ratio, the above-mentioned fourth product is a smaller value. By selecting a smaller one from the first candidate Y-axis coordinate and the fourth product, the minimum value constraint of the Y-axis coordinate of the vanishing point can be achieved, so that the Y-axis coordinate of the vanishing point will not be too large. If the Y-axis coordinate is too large, it is easy to produce a distortion feeling of the object falling when presenting the perspective effect of the object flying on the second image.
[0156] The second candidate Y-axis coordinate can be expressed as follows (5).
[0157] Vy2=min(Vy1,n1×Dy) (5)
[0158] The meanings of the parameters in formula (5) are as follows:
[0159] Vy2 represents the second candidate Y-axis coordinate, Vy1 represents the first candidate Y-axis coordinate, n1 represents the second preset ratio, and Dy represents the minimum value of the minor axis of the minimum circumscribed ellipse. Figure 4 As shown, Dy can also be understood as the coordinate of the endpoint D on the Y axis of the image coordinate system.
[0160] In some embodiments, a large number of different images can be obtained, and the vanishing point in the image and the above-mentioned axis length ratio of the minimum circumscribed ellipse of the object in the image are analyzed to determine a numerical range of the above-mentioned fourth ratio n4. In this numerical range, the larger the value, the smaller the axis length ratio, and the smaller the value, the larger the axis length ratio. When it is necessary to obtain the fourth ratio n4, the above-mentioned axis length ratio of the minimum circumscribed ellipse of the object can be obtained first, and then a fourth ratio n4 can be selected from this numerical range according to the size of the axis length ratio. Taking the object as a vehicle as an example, in some embodiments, the numerical range of the fourth ratio n4 can be [0.55-1.15]. If the Y-axis coordinate of the vanishing point is relatively large, it may cause the lowest point of the vehicle (i.e., the minimum value of the short axis of the minimum circumscribed ellipse on the Y-axis of the image coordinate system) to be smaller than the Y-axis coordinate of the vanishing point and the deviation between the two is relatively large, resulting in a sense of the vehicle falling far below the vanishing point. In this regard, after repeated research and testing, the inventors can select a fourth ratio n4 corresponding to the vehicle's axle length ratio based on the numerical range of the fourth ratio n4 [0.55-1.15], and use the fourth product between the selected fourth ratio n4 and the lowest point of the vehicle as a possible Y-axis coordinate of a vanishing point. If the Y-axis coordinate of the vanishing point is relatively large, the fourth product can be used as the Y-axis coordinate of the vanishing point to avoid the distortion of the vehicle falling due to the large Y-axis coordinate of the vanishing point.
[0161] Step S207: Obtain the final Y-axis coordinate of the vanishing point according to the second candidate Y-axis coordinate. Specifically, the second candidate Y-axis coordinate can be used as the final Y-axis coordinate.
[0162] Based on the method described in steps S201 to S207, the Y-axis coordinate of the vanishing point can be constrainedly adjusted, so as to ensure that after the second image is obtained according to the vanishing point, the second image presents a better perspective effect of the flying object.
[0163] The above step S207 is described below.
[0164] In some implementations of the above step S207, the final Y-axis coordinate of the vanishing point may be obtained through the following steps S2071 to S2073.
[0165] Step S2071: Obtain the axis length ratio between the minor axis length and the major axis length of the minimum circumscribed ellipse. Figure 4 As shown, the axial length ratio can be expressed as r1=|CD| / |AB|, where |CD| represents the minor axis length and |AB| represents the major axis length.
[0166] Step S2072: Obtain the aspect ratio of the object according to the axis length ratio.
[0167] Specifically, the axial length ratio can be regarded as the aspect ratio.
[0168] Step S2073: Obtain the final Y-axis coordinate of the vanishing point according to the fifth product of the second candidate Y-axis coordinate and the aspect ratio.
[0169] Specifically, the product can be used as the final Y-axis coordinate.
[0170] Based on the method described in steps S2071 to S2073 above, the final Y-axis coordinate of the vanishing point can be changed following the change in the height of the object, thereby further improving the perspective effect of the flying object presented in the second image.
[0171] In some implementations of the above step S2073, the aspect ratio of the object can be obtained through the following steps 31 to 32.
[0172] Step 31: Obtain a first fitting function, which is used to represent the corresponding relationship between the axis length ratio and the height-to-width ratio of the object. The axis length ratio is the ratio between the length of the minor axis and the length of the major axis of the minimum circumscribed ellipse of the object region in the image. The height-to-width ratio is the ratio between the height and width of the object in the image.
[0173] Step 32: Using the axis length ratio as an input of the first fitting function, and obtaining the aspect ratio of the object corresponding to the axis length ratio through the first fitting function.
[0174] In the first fitting function, the independent variable is the axis length ratio, and the dependent variable is the aspect ratio, which is the input of the first fitting function. Therefore, the axis length ratio is used as the input of the first fitting function, that is, the axis length ratio is substituted into the independent variable in the first fitting function to obtain the value of the dependent variable, that is, the aspect ratio.
[0175] Based on the method described in steps 31 to 32 above, the first fitting function can be used to conveniently and accurately obtain the aspect ratio corresponding to the axial length ratio.
[0176] In some embodiments, the first fitting function can be obtained by following steps 41 to 43. It should be noted that when using the first fitting function to obtain the aspect ratio, the first fitting function obtained by following steps 41 to 43 can be called, and it is not necessary to perform fitting by following steps every time the first fitting function is used.
[0177] Step 31: Use a camera to capture multiple target images of the object at multiple perspective angles. The perspective angle can be understood as the angle between the major axis of the minimum circumscribed ellipse of the object area in the target image and the X-axis of the image coordinate system of the target image.
[0178] Step 32: Obtain the axis length ratio between the minor axis length and the major axis length of the minimum circumscribed ellipse of the object in the target image, and obtain the aspect ratio of the object in the target image.
[0179] Step 33: Perform nonlinear function fitting on the axial length ratios and aspect ratios of the objects in the multiple target images to obtain a first fitting function.
[0180] For an object in the same target image, the axial length ratio and the height-width ratio of the object correspond to each other. Therefore, multiple sets of corresponding axial length ratios and height-width ratios can be obtained by using multiple target images, and nonlinear function fitting is performed on these corresponding axial length ratios and height-width ratios to obtain a first fitting function. In addition, conventional nonlinear function fitting methods can be used for fitting.
[0181] In some embodiments, the object is a vehicle, and a first fitting function shown in the following equation (6) can be obtained.
[0182] r2= 1- pow((a1 - a2×r1), 2) (6)
[0183] The meanings of the parameters in formula (6) are as follows:
[0184] r2 represents the dependent variable of the first fitting function, that is, the aspect ratio; r1 represents the axial length ratio; pow represents the power function, (a1-a2×r1) represents the base in the power function, 2 represents the power of the power function, and a1 and a2 are both constants. The values of a1 and a2 are the values fitted when performing nonlinear function fitting. Taking a1 as an example, when fitting, if the number of target images used is different, the fitted value of a1 may be different. In addition, the more target images there are, the more accurate the fitted value of a1. In some embodiments, using at least 1,000 target images for nonlinear fitting, a1=1.4 and a2=1.8 can be obtained.
[0185] Based on the methods described in steps 41 to 43 above, the accuracy of obtaining the aspect ratio using the first fitting function can be improved.
[0186] In some embodiments of the above step S103, after the vanishing point is obtained, the X-axis coordinate of the vanishing point in the image coordinate system of the first image may be used as the initial X-axis coordinate. Fig.11 Steps S301 to S307 are shown to adjust the initial X-axis coordinates.
[0187] Step S301: Obtain the perspective angle of the first image. The perspective angle is the angle between the major axis of the minimum circumscribed ellipse of the object in the first image and the X-axis of the image coordinate system.
[0188] Step S302: Obtain a fifth ratio n5 corresponding to the perspective viewing angle.
[0189] The fifth ratio n5 is in a negatively correlated relationship with the perspective angle, that is, the larger the perspective angle is, the smaller the fifth ratio n5 is, and vice versa.
[0190] The corresponding relationship of the above negative correlation is explained below.
[0191] If the perspective angle θ is larger, the real vanishing point is closer to the object; if the perspective angle θ is smaller, the real vanishing point is farther from the object.
[0192] The fifth ratio n5 is used to multiply the initial X-axis coordinate and determine the final X-axis coordinate of the vanishing point based on the product. Therefore, if the perspective angle θ is relatively small, a larger fifth ratio n5 can be set so that the product of the two multiplying together is closer to the X-axis coordinate of the real vanishing point; if the perspective angle θ is relatively large, a smaller fifth ratio n5 can be set so that the product of the two multiplying together is also closer to the X-axis coordinate of the real vanishing point. Further, since the perspective angle θ is negatively correlated with the X-axis coordinate of the real vanishing point, it can be determined that the fifth ratio n5 is also negatively correlated with the perspective angle.
[0193] In some implementations, a large number of different images may be acquired, and the real vanishing points, perspective angles, etc. in the images may be analyzed to determine a numerical range of the fifth ratio n5. Within this numerical range, a larger numerical value corresponds to a smaller perspective angle, and a smaller numerical value corresponds to a larger perspective angle. When the fifth ratio n5 needs to be acquired, the perspective angle of the first image may be acquired first, and then a fifth ratio n5 may be selected from this numerical range according to the size of the perspective angle.
[0194] Step S303: Obtain the sixth product between the initial X-axis coordinate and the fifth ratio n5.
[0195] Step S304: Obtain the major axis length, the lower limit value of the ratio, and the upper limit value of the ratio of the minimum circumscribed ellipse.
[0196] The lower limit value of the ratio can be understood as the lower limit value of the magnification factor of the long axis length, and the upper limit value of the ratio can be understood as the upper limit value of the magnification factor of the long axis length. Both the lower limit value of the ratio and the upper limit value of the ratio are greater than 1.
[0197] When determining the values of the lower limit value and the upper limit value of the ratio, a large number of different images can be obtained, and the major axis length of the minimum circumscribed ellipse of the object in the image and the X-axis coordinate of the real vanishing point are analyzed to determine the minimum and maximum values of the ratio between the X-axis coordinate of the real vanishing point and the major axis length, and the lower limit value of the ratio is set according to the minimum value, and the upper limit value of the ratio is set according to the maximum value. In some embodiments, the object is a vehicle, and the lower limit value and the upper limit value of the ratio are 1.25 and 1.75, respectively.
[0198] Step S305: Obtain the seventh product between the major axis length and the lower limit value of the ratio, and obtain the eighth product between the major axis length and the upper limit value of the ratio.
[0199] Step S306: Selecting a larger one from the sixth product and the seventh product as the candidate X-axis coordinate of the vanishing point.
[0200] By selecting a larger one as the candidate X-axis coordinate, it is possible to prevent the vanishing point from being too close to the object, thereby affecting the perspective effect presented by the second image.
[0201] Step S307: Selecting a smaller one from the candidate X-axis coordinates and the eighth product as the final X-axis coordinate of the vanishing point.
[0202] By selecting the smaller one as the final X-axis coordinate, it is possible to prevent the vanishing point from being too far away from the object, thereby affecting the perspective effect presented by the second image.
[0203] The final X-axis coordinate can be expressed as follows:
[0204] Vx2 = min(max(b1×|BA|, ratio×Vx1) , b2×|BA|) (7)
[0205] The meanings of the parameters in formula (7) are as follows:
[0206] Vx2 represents the final X-axis coordinate, Vx1 represents the initial X-axis coordinate, ratio represents the fifth ratio n5, b1 represents the ratio lower limit, b2 represents the ratio upper limit, |BA| represents the major axis length. In some embodiments, b1=1.25, b2=1.75.
[0207] Based on the method described in steps S301 to S307, the X-axis coordinate of the vanishing point can be constrainedly adjusted, so as to ensure that after the second image is obtained according to the vanishing point, the second image presents a better perspective effect of the flying object.
[0208] The above step S302 is described below.
[0209] In some implementations of the above step S302, the fifth ratio n5 corresponding to the perspective viewing angle may be acquired through the following steps S3021 to S3022.
[0210] Step S3021: obtaining a second fitting function, where the second fitting function is used to represent the corresponding relationship between the perspective viewing angle and the fifth ratio n5.
[0211] Step S3022: taking the perspective viewing angle of the first image as an input of the second fitting function, and obtaining a fifth ratio n5 corresponding to the perspective viewing angle through the second fitting function.
[0212] In the second fitting function, the independent variable is the perspective angle, and the dependent variable is the fifth ratio n5, which is the input of the second fitting function. Therefore, the perspective angle is used as the input of the second fitting function, that is, the perspective angle is substituted into the independent variable in the second fitting function to obtain the value of the dependent variable, that is, the fifth ratio n5.
[0213] Based on the above steps S3021 to S3022, the fifth ratio n5 corresponding to the perspective viewing angle can be obtained conveniently and accurately using the second fitting function.
[0214] In some embodiments, the second fitting function can be obtained by following steps 51 to 55. It should be noted that when the second fitting function is used to obtain the aspect ratio, the second fitting function obtained by following steps 51 to 56 can be called, and it is not necessary to perform fitting by following steps every time the second fitting function is used.
[0215] Step 51: Use a camera to capture multiple target images of the object at multiple perspective angles. The perspective angle can be understood as the angle between the major axis of the minimum circumscribed ellipse of the object area in the target image and the X axis of the image coordinate system.
[0216] Step 52: Taking the contact point where the object in the target image contacts the ground as the starting point, extend a third distance along the moving direction of the object to obtain a vanishing point.
[0217] The third distance is the same as the first distance in the aforementioned method embodiment.
[0218] Step 53: Obtain the first X-axis coordinate of the vanishing point in the image coordinate system of the target image.
[0219] Step 54: Obtain the second X-axis coordinate of the real vanishing point in the target image in the image coordinate system.
[0220] Step 55: Obtain a fifth ratio n5 according to the ratio of the second X-axis coordinate to the first X-axis coordinate.
[0221] Step 56: Perform nonlinear function fitting on the perspective viewing angles and X-axis magnification ratios of the multiple target images to obtain a second fitting function.
[0222] For the same target image, the perspective angle of view and the fifth ratio n5 in the target image are mutually corresponding. Therefore, multiple sets of mutually corresponding perspective angles of view and fifth ratios n5 can be obtained by using multiple target images, and nonlinear fitting is performed on these mutually corresponding perspective angles of view and fifth ratios n5 to obtain a second fitting function. In addition, conventional nonlinear function fitting methods can be used for fitting.
[0223] In some embodiments, the object is a vehicle, and a second fitting function shown in the following equation (8) can be obtained.
[0224] ratio = 1.0 +c1×(c2- (θ-c3) / c4) (8)
[0225] The meanings of the parameters in formula (8) are as follows:
[0226] ratio represents the fifth ratio n5, θ represents the perspective angle, and c1~c4 are all constants. The values of c1~c4 are all the values fitted when performing nonlinear function fitting. Taking c1 as an example, when fitting, if the number of target images used is different, the fitted value of c1 may be different. In addition, the more target images there are, the more accurate the fitted value of c1. In some embodiments, at least 1000 target images are used for nonlinear fitting, and c1=0.5, c2=1.15, c3=10, and c4=35 can be obtained.
[0227] Based on the methods described in steps 51 to 56 above, the accuracy of obtaining the X-axis magnification ratio using the second fitting function can be improved.
[0228] 2. Explain step S104.
[0229] In some implementations of the above step S104, when the first image is subjected to convolution blur processing according to the vanishing point, the convolution blur processing may be performed by: Fig.12 The following steps S1041 to S1045 shown perform convolution blur processing.
[0230] Step S1041: Obtain the convolution kernel of the convolution.
[0231] Convolution is a process used to blur pixels in an image.
[0232] Step S1042: Obtain a convolution area of the first pixel in the first image according to the radius of the convolution kernel.
[0233] The convolution area is an area centered on the first pixel and having a radius that is the same as the radius of the convolution kernel, and the first pixel is any pixel in the first image.
[0234] Step S1043: Acquire a second pixel point in the convolution region that is located on the same straight line as the first pixel point and the vanishing point.
[0235] Step S1044: Obtain an average pixel value of the first pixel point and all second pixel points.
[0236] Step S1045: taking the average pixel value as the pixel value of the first pixel in the perspective blurred image.
[0237] Based on the above steps S1041 to S1045, convolution can be used to quickly and accurately obtain the pixel value of each pixel in the first image in the perspective blurred image, thereby completing the blurring process of the first image.
[0238] In some implementations of the above step S1043, the second pixel point can be obtained through the following steps 61 to 62.
[0239] Step 61: Obtain a first vector between a first pixel point and a vanishing point, and obtain a first angle between the first vector and an X-axis of an image coordinate system of a first image.
[0240] Step 62: Acquire a second pixel point according to the first angle.
[0241] The second angle corresponding to the second pixel point is the same as the first angle. The second angle is the angle between the second vector corresponding to the second pixel point and the X-axis of the image coordinate system. The second vector is the vector between the second pixel point and the vanishing point.
[0242] Based on the above steps 61 to 62, the second pixel point that is located on the same straight line as the vanishing point and the first pixel point can be conveniently and accurately obtained by using the vector angle.
[0243] In some implementations of the above step 62, the second pixel point can be acquired according to the first angle through the following steps 621 to 625.
[0244] Step 621: Get the cosine value and sine value of the first angle respectively. The cosine value is the value obtained by substituting the first angle into the cosine function, and the sine value is the value obtained by substituting the first angle into the sine function. Step 622: Get the step size dr of the i-th second pixel point in the convolution area, dr=i×k, k<1, i=[-r,r], r is the radius of the convolution kernel. Step 623: Get the X-axis offset according to the product between the cosine value and the step size dr, and get the Y-axis offset according to the product between the sine value and the step size dr. Specifically, the product between the cosine value and the step size dr is used as the X-axis offset, and the product between the sine value and the step size dr is used as the Y-axis offset. Step 624: Get the X-axis coordinate of the i-th second pixel point in the image coordinate system according to the sum of the X-axis coordinate of the first pixel point in the image coordinate system and the X-axis offset. Specifically, the sum of the X-axis coordinate and the X-axis offset is used as the X-axis coordinate of the i-th second pixel point. Step 625: According to the sum of the Y-axis coordinate and the Y-axis offset of the first pixel point in the image coordinate system, the Y-axis coordinate of the i-th second pixel point in the image coordinate system is obtained. Specifically, the sum of the Y-axis coordinate and the Y-axis offset is used as the Y-axis coordinate of the i-th second pixel point. According to the X-axis coordinate and the Y-axis coordinate of the i-th second pixel point, the i-th second pixel point can be accurately located.
[0245] Based on the methods described in steps 621 to 625 above, a second pixel point that is located on the same straight line as the vanishing point and the first pixel point can be further accurately acquired.
[0246] The methods described in steps 621 to 625 above can be implemented by computer program code. The computer program code can be, for example:
[0247] dxy=(x,y)–(Vx,Vy);
[0248] float rad=atan(dxy.y,dxy.x);
[0249] float deg = rad*180.0 / 3.14;
[0250] float sampleNum = 0.0;
[0251] float sum_background = 0.;
[0252] float dx = cos(rad);
[0253] float dy = sin(rad);
[0254] for(float i=-r;i <r;i+=1.){
[0255] float dr = i * 0.01;
[0256] float px=x+dr*dx;
[0257] float py=y+dr*dy;
[0258] sampleValue = Msrc[px,py];
[0259] sum_background+=sampleValue;
[0260] sampleNum+=1.0;
[0261] }
[0262] float blurOut=sum_background / sampleNum;
[0263] The meanings of the parameters in the above computer program code are as follows:
[0264] dxy represents the first vector between the first pixel point (x, y) and the vanishing point (Vx, Vy). rad represents the first angle between the first vector dxy and the X axis of the image coordinate system of the first image, and this first angle is in radians. deg represents the conversion of the first angle in radians into the first angle in degrees. sampleNum represents the number of samples. sum_background represents the cumulative sum. dx=cos(rad) represents the cosine value of the first angle. dy=sin(rad) represents the sine value of the first angle. dr represents the step size. r represents the radius of the convolution kernel. px represents the X axis coordinate of the second pixel point in the image coordinate system. py represents the Y axis coordinate of the second pixel point in the image coordinate system. Msrc[px,py] represents the pixel value of the second pixel point in the first image. Msrc represents the first image, that is, the original image Msource. blurOut represents the pixel value of the first pixel point in the perspective blurred image.
[0265] The following is combined with Fig.13, the vanishing point-based image generation method provided in this application is described. Fig.13 The process of the image generation method based on vanishing points in this embodiment is exemplarily shown. Fig.13 As shown, an image can be generated through the following steps S401 to S413.
[0266] Step S401: Start the camera and enter the car camera mode. In this embodiment, the electronic device can execute the image generation method based on the vanishing point. The electronic device is equipped with a camera and configured with the car camera mode. After the user starts the camera and chooses to enter the car camera mode, the electronic device will execute the image generation method based on the vanishing point on the image collected by the camera to generate an image. Step S402: Click the shooting key to trigger a photo. After the user starts the camera and chooses to enter the car camera mode, the user can click the shooting key, and the camera will take a photo in response to the shooting key being clicked. Step S403: Get the first image of the vehicle. In this embodiment, the object is a vehicle. The user determines the vehicle to be photographed and clicks the shooting key. After the camera takes the photo, the first image of the vehicle is obtained. Step S404: Get the body area of the vehicle in the first image, which is also the object area. Step S405: Get the minimum circumscribed ellipse of the body area. Step S406: Get the angle θ and the center of gravity. The angle θ is the angle between the major axis of the minimum circumscribed ellipse of the object in the first image and the X axis of the image coordinate system, and the angle θ can also be understood as the perspective viewing angle of the first image. The center of gravity is the center of gravity of the vehicle body area. Step S407: Obtain the contact point of the vehicle with the ground in the first image. Step S408: Obtain the vanishing point in the first image. Step S409: Adjust the Y axis coordinate of the vanishing point. Specifically, the Y axis coordinate can be adjusted by the method described in steps S201 to S205 in the aforementioned method embodiment. Step S410: Adjust the X axis coordinate of the vanishing point. Specifically, the X axis coordinate can be adjusted by the method described in steps S301 to S307 in the aforementioned method embodiment. Step S411: Perform convolution blur processing on the first image to obtain a perspective blurred image. Specifically, the convolution blur processing can be performed on the first image by the method described in step S104 in the aforementioned method embodiment. Step S412: Generate a second image based on the first image, the vehicle body area and the perspective blurred image. Specifically, the second image may be generated by the method described in step S105 of the aforementioned method embodiment. Step S413: Send the second image to other business processes. For example, the other business processes may be storing the second image, transmitting the second image to other electronic devices, etc. Based on the methods described in steps S401 to S413 above, a second image that can present a speeding vehicle effect may be easily obtained.
[0267] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.
[0268] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.
[0269] Another aspect of the present application also provides a computer-readable storage medium.
[0270] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the vanishing point-based image generation method of the above method embodiment, and the program may be loaded and run by a processor to implement the above vanishing point-based image generation method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.
[0271] Another aspect of the present application provides an electronic device.
[0272] In an embodiment of an electronic device according to the present application, the electronic device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. Fig.14 , Fig.14 FIG. 4 exemplarily shows that the memory 11 and the processor 12 are communicatively connected via a bus.
[0273] The electronic device described in the present application may be, but is not limited to, a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, etc., and the embodiments of the present application are not limited to this.
[0274] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.
Claims
1. A method for generating an image based on a vanishing point, characterized in that: The method comprises: Acquire a first image of a target object, and acquire an object region of the target object in the first image; Obtaining a minimum circumscribed ellipse of the object area, and obtaining a moving direction of the target object according to the major axis direction of the minimum circumscribed ellipse; Taking the ground reference point of the target object in the first image as a starting point, extending a first distance along the movement direction to obtain a vanishing point; Processing the first image according to the vanishing point to obtain a perspective blurred image; generating a second image of the target object according to the first image, the object region and the perspective blurred image; in, In the second image, the pixel values of the pixels within the object area are the same as the pixel values of the pixels at the same position in the first image, and the pixel values of the pixels outside the object area are the same as the pixel values of the pixels at the same position in the perspective blurred image.
2. The method according to claim 1, characterized in that The ground reference point is a touchdown point where the target object in the first image contacts the ground.
3. The method according to claim 1, characterized in that The step of processing the first image according to the vanishing point to obtain a perspective blurred image includes: Performing convolution blur processing on the first image according to the vanishing point to obtain the perspective blurred image; The pixels in the perspective blurred image can form a plurality of straight lines intersecting at the vanishing point, and the pixel values of the pixels of the straight line in the perspective blurred image are obtained by performing convolution blur processing on the pixel values of the pixels of the straight line in the first image.
4. The method according to claim 1, characterized in that: The first distance is obtained by: Obtaining the length of the major axis of the minimum circumscribed ellipse; Acquire a perspective viewing angle of the first image, where the perspective viewing angle is an angle between a major axis of a minimum circumscribed ellipse of an object region in the first image and an X-axis of an image coordinate system of the first image; Acquire a first ratio n1 corresponding to the perspective viewing angle, where the first ratio n1 is in a negatively correlated corresponding relationship with the perspective viewing angle; The first distance is obtained according to the product of the major axis length and the first ratio n1.
5. The method according to claim 1, characterized in that The method further includes obtaining the touchdown point by: Obtaining the center of gravity of the object area; The touchdown point is obtained by taking the center of gravity as a starting point and extending a second distance along the negative direction of the Y axis of the image coordinate system of the first image.
6. The method according to claim 5, characterized in that The obtaining the center of gravity of the object area includes: Acquire coordinates and grayscale values of pixels in the object area according to the first image, wherein the coordinates include X-axis coordinates and Y-axis coordinates of the pixels in the image coordinate system; Obtain the sum of the grayscale values of all pixels in the object area; Obtain a first product of the X-axis coordinate of the pixel and the grayscale value, and obtain the sum of the first products of all pixels; Obtain the second product of the Y-axis coordinate of the pixel and the grayscale value, and obtain the sum of the second products of all pixels; Obtaining the X-axis coordinate of the center of gravity in the image coordinate system according to the ratio between the sum of the first products and the sum of the grayscale values; According to the ratio between the sum of the second products and the sum of the grayscale values, the Y-axis coordinate of the center of gravity in the image coordinate system is obtained.
7. The method according to claim 5, characterized in that The second distance is obtained by: Obtaining the minor axis length of the minimum circumscribed ellipse; Acquire a perspective viewing angle of the first image, where the perspective viewing angle is an angle between a major axis of a minimum circumscribed ellipse of an object region in the first image and an X-axis of the image coordinate system; Acquire a second ratio n2 corresponding to the perspective viewing angle, where the second ratio n2 is in a positively correlated corresponding relationship with the perspective viewing angle; The second distance is obtained according to the product of the short axis length and the second ratio n2.
8. The method according to claim 1, characterized in that After acquiring the vanishing point, the method further includes taking the Y-axis coordinate of the vanishing point in the image coordinate system of the first image as an initial Y-axis coordinate, and adjusting the initial Y-axis coordinate in the following manner: Obtaining the axis length ratio between the minor axis length and the major axis length of the minimum circumscribed ellipse; Respectively acquiring a third ratio n3 and a fourth ratio n4 corresponding to the axial length ratio, wherein the third ratio n3 is in a positively correlated corresponding relationship with the axial length ratio, and the fourth ratio n4 is in a negatively correlated corresponding relationship with the axial length ratio; Obtaining the maximum value and the minimum value of the minor axis of the minimum circumscribed ellipse on the Y axis of the image coordinate system, obtaining a third product between the maximum value and the third ratio n3, and obtaining a fourth product between the minimum value and the fourth ratio n4; Selecting a larger one of the initial Y-axis coordinate and the third product as a first candidate Y-axis coordinate of the vanishing point; Selecting a smaller one of the first candidate Y-axis coordinate and the fourth product as the second candidate Y-axis coordinate of the vanishing point; The final Y-axis coordinate of the vanishing point is acquired according to the second candidate Y-axis coordinate.
9. The method according to claim 8, characterized in that The value range of the third ratio n3 is [0.95-2.15]; The numerical range of the fourth ratio n4 is [0.55-1.15].
10. The method according to claim 8, characterized in that The acquiring the final Y-axis coordinate of the vanishing point according to the second candidate Y-axis coordinate includes: According to the axial length ratio, obtaining the aspect ratio of the target object; A final Y-axis coordinate of the vanishing point is acquired according to a fifth product of the second candidate Y-axis coordinate and the aspect ratio.
Citation Information
Patent Citations
Moving object shooting method based on mobile terminal, and mobile terminal
CN105847662A
Multi-target tracking method and device based on TSK fuzzy model and storage medium
CN110349188A
Image generation method and device, chip, terminal and storage medium
CN113542600A
Image processing method and device, storage medium and terminal
CN114708166A
Obstacle detection method, and vehicle, device and computer storage medium
WO2022199472A1
Cited By
Commodity placement uniformity calculation method and device, electronic equipment and storage medium
CN121564098A
