Image processing method, device, electronic equipment and medium based on vehicle body matting

Abstract

The present invention relates to the field of image processing technology, and specifically provides an image processing method, device, electronic device and medium based on vehicle body matting, aiming to solve the technical problem that existing electronic devices such as mobile phones cannot achieve creative photography related to vehicles, resulting in a low user experience. To this end, the image processing method based on vehicle body matting of the present invention includes: obtaining the original image taken by the camera; obtaining the connected area of ​​the vehicle body based on the original image; performing directional convolution on the original image based on the connected area to obtain a directional convolution result; performing blurring on the connected area to obtain a first image; and determining a second image based on the first image, the original image and the directional convolution result. In this way, the effect of creative photography of vehicle images is achieved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically provides an image processing method, device, electronic equipment and medium based on vehicle body matting. Background Art

[0002] With the rapid development of social economy, the acceleration of urban development, the continuous increase of urban population, and the continuous improvement of people's living standards, cars are becoming more and more important in people's lives. However, existing mobile phones on the market cannot realize creative photography related to cars, resulting in a poor user experience.

[0003] Accordingly, the art needs a new image processing solution based on vehicle body matting to solve the above problems. Summary of the Invention

[0004] In order to overcome the above-mentioned drawbacks, the present invention is proposed to provide a solution or at least partially solve the above-mentioned technical problems. The present invention provides an image processing method, device, electronic device and medium based on vehicle body matting.

[0005] In a first aspect, the present invention provides an image processing method based on vehicle body matting, the method comprising: acquiring an original image taken by a camera; acquiring a connected area of ​​a vehicle body based on the original image; performing directional convolution on the original image based on the connected area to obtain a directional convolution result; blurring the connected area to obtain a first image; and determining a second image based on the first image, the original image, and the directional convolution result.

[0006] In one embodiment, obtaining the connected area of ​​the vehicle body based on the original image includes: extracting the vehicle body area from the original image; extracting all connected areas from the vehicle body area; determining whether the number of all connected areas is greater than zero; and if the number of all connected areas is greater than zero, taking the connected area with the largest area as the connected area of ​​the vehicle body.

[0007] In one embodiment, extracting the vehicle body region from the original image includes: extracting the vehicle body region from the original image using a MODNet model.

[0008] In one embodiment, directional convolution is performed on the original image based on the connected area to obtain a directional convolution result, including: fitting a circumscribed rectangle on the connected area to obtain a circumscribed rectangular image; calculating the angle between the long side of the circumscribed rectangular image and the horizontal direction; judging whether the angle is within a preset angle range; and when the angle is within the preset angle range, performing directional convolution on the original image based on the angle and the radius of the convolution kernel to obtain the directional convolution result.

[0009] In one embodiment, the preset angle range is [-60°, 60°].

[0010] In one embodiment, blurring the connected region to obtain the first image includes: blurring the connected region using a BoxBlur technique to obtain the first image.

[0011] In one embodiment, determining the second image based on the first image, the original image and the directional convolution result includes: based on the first image, performing nonlinear mixing on the original image and the directional convolution result to obtain the second image.

[0012] In a second aspect, the present invention provides an image processing device based on vehicle body matting, the device comprising:

[0013] A first acquisition module is configured to acquire an original image captured by a camera;

[0014] a second acquisition module configured to acquire a connected area of ​​a vehicle body based on the original image;

[0015] a directional convolution module, configured to perform directional convolution on the original image based on the connected region to obtain a directional convolution result;

[0016] a fuzzy processing module, configured to perform fuzzy processing on the connected area to obtain a first image;

[0017] A determination module is configured to determine a second image based on the first image, the original image and the directional convolution result.

[0018] In a third aspect, an electronic device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute any of the aforementioned image processing methods based on vehicle body matting.

[0019] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, wherein the program codes are suitable for being loaded and run by a processor to execute any of the aforementioned image processing methods based on vehicle body matting.

[0020] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects:

[0021] The image processing method based on vehicle body matting in this invention first obtains the original image captured by the camera. Then, based on the original image, it obtains the connected regions of the vehicle body. Next, it performs directional convolution on the original image based on the connected regions to obtain the directional convolution result. The connected regions are then blurred to obtain the first image. Finally, the second image is determined based on the first image, the original image, and the directional convolution result. This creates a creative image of a speeding vehicle, enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The disclosure of the present invention will be more easily understood with reference to the accompanying drawings. Those skilled in the art will readily appreciate that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Furthermore, similar numbers in the drawings represent similar components, wherein:

[0023] Figure 1 This is a flow chart of main steps of an image processing method based on vehicle body matting according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of an original image in one embodiment;

[0025] Figure 3 is a schematic diagram of a vehicle body area in one embodiment;

[0026] Figure 4 is a schematic diagram of the largest connected area in one embodiment;

[0027] Figure 5 is a schematic diagram of a circumscribed rectangular image in one embodiment;

[0028] Figure 6 is a schematic diagram of an image after blurring in an embodiment;

[0029] Figure 7 is a schematic diagram of a second image in one embodiment;

[0030] Figure 8 is a complete flowchart of an image processing method in one embodiment;

[0031] Figure 9 It is a schematic diagram of the main structural block diagram of an image processing device based on vehicle body matting according to an embodiment of the present invention;

[0032] Figure 10 It is a schematic structural diagram of an electronic device in an embodiment.

[0033] Reference Signs List :

[0034] 11: First acquisition module; 12: Second acquisition module; 13: Directional convolution module; 14: Blur processing module; 15: Determination module. DETAILED DESCRIPTION

[0035] Some embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0036] In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module may include hardware circuitry, various suitable sensors, communication ports, and memory. It may also include software components, such as program code, or a combination of software and hardware. A processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. A processor has data and / or signal processing capabilities. A processor may be implemented in software, hardware, or a combination of both. Non-transitory computer-readable storage media include any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "A and / or B" refers to all possible combinations of A and B, such as only A, only B, or both A and B. The terms "at least one of A or B" or "at least one of A and B" have similar meanings to "A and / or B" and may include only A, only B, or both A and B. The singular forms "one" and "the" may also include the plural forms.

[0037] With the rapid development of social economy, the acceleration of urban development, the continuous increase of urban population, and the continuous improvement of people's living standards, cars are becoming more and more important in people's lives. However, existing mobile phones on the market cannot realize creative photography related to cars, resulting in a poor user experience.

[0038] To this end, this application provides an image processing method, device, electronic device, and medium based on vehicle body matting. First, the original image captured by a camera is acquired. Next, the connected regions of the vehicle body are obtained based on the original image. Next, a directional convolution is performed on the original image based on the connected regions to obtain a directional convolution result. The connected regions are then blurred to obtain a first image. Finally, a second image is determined based on the first image, the original image, and the directional convolution result. This creates a creative image of a speeding vehicle, bringing the user closer to the vehicle and enhancing the user experience.

[0039] See attached Figure 1 , Figure 1 1 is a flow chart of main steps of an image processing method based on vehicle body matting according to an embodiment of the present invention.

[0040] like Figure 1 As shown, the image processing method based on vehicle body matting in the embodiment of the present invention mainly includes the following steps S101 to S105.

[0041] Step S101: Acquire an original image captured by a camera.

[0042] For example, Figure 2 It can be used as an example of an original image, but is not limited to this.

[0043] Step S102: Acquire the connected area of ​​the vehicle body based on the original image.

[0044] In a specific embodiment, obtaining the connected area of ​​the vehicle body based on the original image includes: extracting the vehicle body area from the original image; extracting all connected areas from the vehicle body area; determining whether the number of all connected areas is greater than zero; and if the number of all connected areas is greater than zero, taking the connected area with the largest area as the connected area of ​​the vehicle body.

[0045] Specifically, first, the vehicle body area is extracted from the original image captured by the camera. For example, Figure 3 Can be used as a Figure 2 The example of the car body region extracted from the original image shown is not limited to this. Then all connected regions are extracted from the car body region, and it is determined whether the number of all connected regions is greater than zero. If so, it is further determined whether the number of all connected regions is equal to 1. If so, it means that there is only one connected region, which is the connected region of the car body. If the number of all connected regions is not equal to 1, the connected region with the largest area is taken as the connected region of the car body, as shown in the following example. Figure 4 shown.

[0046] In a specific embodiment, extracting the vehicle body region from the original image includes: extracting the vehicle body region from the original image using a MODNet model.

[0047] Specifically, in one embodiment, the MODNet model is used to extract the vehicle body region from the original image, but the present invention is not limited thereto and other deep learning models may also be used to extract the vehicle body region. The MODNet model is a real-time image extraction model that can accurately extract objects from images.

[0048] Step S103: performing directional convolution on the original image based on the connected region to obtain a directional convolution result.

[0049] In a specific embodiment, directional convolution is performed on the original image based on the connected area to obtain a directional convolution result, including: fitting a circumscribed rectangle on the connected area to obtain a circumscribed rectangular image; calculating the angle between the long side of the circumscribed rectangular image and the horizontal direction; judging whether the angle is within a preset angle range; when the angle is within the preset angle range, performing directional convolution on the original image based on the angle and the radius of the convolution kernel to obtain the directional convolution result.

[0050] Specifically, in one embodiment, after the connected area of ​​the vehicle body is obtained based on the aforementioned step S102, the connected area of ​​the vehicle body is fitted with a circumscribed rectangle to obtain a circumscribed rectangle image. For example, Figure 5 This may be used as an example of performing circumscribed rectangle fitting on the connected areas of the vehicle body to obtain the minimum circumscribed rectangle, but is not limited thereto.

[0051] Next, the angle θ between the long side (e.g., CD) of the circumscribed rectangular image and the horizontal direction is calculated. A determination is made as to whether the angle θ is within a preset range. If so, a directional convolution is performed on the original image based on the angle θ and the radius r of the convolution kernel to obtain the directional convolution result. The size of the convolution kernel can be determined by the user based on actual conditions and is not limited to this. Once the convolution kernel is selected, the kernel radius is obtained. For example, for a 5*5 convolution kernel, the kernel radius is 2.

[0052] The preset angle range may be a predetermined angle range. In one embodiment, the preset angle range is [-60°, 60°].

[0053] In this way, based on the circumscribed rectangle of the vehicle body fitting and the angle θ of the directional convolution, the direction of the convolution is made more accurate and more consistent with the direction of the vehicle's travel, most accurately expressing the vehicle's running forward, thereby improving the user experience.

[0054] Step S104: performing blur processing on the connected area to obtain a first image.

[0055] In a specific embodiment, blurring the connected region to obtain the first image includes: blurring the connected region using a Box Blur technique to obtain the first image.

[0056] Box Blur technology is a fast mean blur technology. Compared with traditional convolution blur, Box Blur can more efficiently complete image blur processing, thus providing basic support for the subsequent acquisition of the second image.

[0057] In one embodiment, Figure 6This can be used as an example of the first image obtained after blurring, but is not limited thereto.

[0058] Step S105: determining a second image based on the first image, the original image and the directional convolution result.

[0059] In a specific embodiment, determining the second image based on the first image, the original image, and the directional convolution result includes: performing nonlinear mixing on the original image and the directional convolution result based on the first image to obtain the second image. For example, Figure 7 This can be used as an example of the second image, but is not limited to this, which embodies the creative effect of a speeding vehicle.

[0060] Based on steps S101-S105, the original image captured by the camera is first acquired. The connected regions of the vehicle body are then determined based on the original image. A directional convolution is then performed on the original image based on the connected regions to obtain a directional convolution result. The connected regions are then blurred to obtain a first image. Finally, a second image is determined based on the first image, the original image, and the directional convolution result. This creates a creative image of a vehicle running forward, bringing the user closer to the vehicle and enhancing their experience.

[0061] In one embodiment, specifically Figure 8 As shown in the figure, the image processing method based on vehicle body matting is described in detail.

[0062] S201: Start the camera, enter the car camera mode, and click the shooting button to trigger a photo.

[0063] S202: Acquire the currently captured original image Msrc.

[0064] S203: Perform matting on the original image Msrc to extract the vehicle body area in the image to obtain an initial matting result Mm.

[0065] S204: extracting connected regions from the initial matting result Mm, storing the extracted connected regions into a Vector, recorded as Vc; and counting the number of connected regions in Vc, recorded as Nvc.

[0066] Vector is part of the C++ Standard Template Library (STL). It is considered a container because it can store various types of objects like a container. Simply put: vector is a dynamic array that can store any type of data and can add and compress data.

[0067] S205: Determine whether Nvc is greater than 0. If Nvc is not greater than 0, it indicates that there is no car in the image, and the process proceeds to step S212. If Nvc is greater than 0, continue to determine whether Nvc is equal to 1. If so, it indicates that there is only one connected car area. Extract the connected area of ​​the car body in Vc, record it as Mu, and proceed to step S206. If Nvc is not equal to 1, it indicates that there are multiple connected car areas. Sort the connected areas in Vc by area, extract the connected area with the largest area as the connected area Mu of the car body, and proceed to step S206.

[0068] S206: Perform circumscribed rectangle fitting on the connected area Mu or the largest connected area Mu of the vehicle body. The obtained circumscribed rectangle image is as follows: Figure 5 As shown, it is Rectu.

[0069] S207: Calculate the angle between the long side of Rectu and the horizontal direction, which is denoted as θ. The specific calculation formula is: θ = arctan[(Y D -Y C ) / (X D -X C )].

[0070] S208: Determine whether θ is within the preset angle range. If so, proceed to S209; if not, proceed to S212.

[0071] S209: With the convolution direction as θ and the radius of the convolution kernel as r, perform directional convolution on the original image Msrc to obtain the directional convolution result Mdir. Specifically, directional convolution can be achieved through the following steps:

[0072]

[0073] Among them, H is the original image height, W is the original image width, Mdir[i,j] is the result after convolution, dx is the step size in the x direction, dx = r*cos(θ), and dy is the step size in the y direction, dy = r*sin(θ).

[0074] S210: Perform Box blur processing on the maximum connected area Mu to obtain a first image Mbox.

[0075] S211: Based on the first image Mbox, the original image Msrc and the directional convolution result Mdir are nonlinearly mixed to obtain a second image, that is, the Fei Chi creative image. Specifically, the nonlinear mixing can be achieved by the following steps:

[0076]

[0077] S212: Update the image information and end the shooting.

[0078] By matting the car area in the original image and then fitting a circumscribed rectangle around the matted car body, we obtain the directional convolution angle θ. We then perform directional convolution on the original image based on the angle θ and the radius r of the convolution kernel, creating a high-definition image with a high-definition quality. This solves the problem of existing mobile phones and other electronic devices lacking vehicle-related creative shooting options, resulting in a poor user experience.

[0079] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0080] Furthermore, the present invention also provides an image processing device based on vehicle body matting.

[0081] See attached Figure 9 , Figure 9 It is a main structural block diagram of an image processing device based on vehicle body matting according to an embodiment of the present invention.

[0082] like Figure 9 As shown, the image processing device based on vehicle body matting in the embodiment of the present invention mainly includes a first acquisition module 11, a second acquisition module 12, a directional convolution module 13, a blur processing module 14, and a determination module 15. In some embodiments, one or more of the first acquisition module 11, the second acquisition module 12, the directional convolution module 13, the blur processing module 14, and the determination module 15 can be combined into one module.

[0083] In some embodiments, the first acquisition module 11 may be configured to acquire an original image captured by a camera.

[0084] The second acquisition module 12 may be configured to acquire a connected area of ​​the vehicle body based on the original image;

[0085] The directional convolution module 13 may be configured to perform directional convolution on the original image based on the connected region to obtain a directional convolution result;

[0086] The blur processing module 14 may be configured to perform blur processing on the connected area to obtain a first image;

[0087] The determination module 15 may be configured to determine a second image based on the first image, the original image, and the directional convolution result.

[0088] In one implementation, the description of specific implementation functions can refer to steps S101 to S105.

[0089] The above-mentioned image processing device based on vehicle body matting is used to perform Figure 1 The embodiment of the image processing method based on vehicle body matting shown in the figure has similar technical principles, technical problems solved and technical effects produced. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process and related instructions of the image processing device based on vehicle body matting can refer to the contents described in the embodiment of the image processing method based on vehicle body matting, and will not be repeated here.

[0090] Those skilled in the art will appreciate that all or part of the processes in the method for implementing the above-mentioned embodiment of the present invention may also be accomplished by instructing the relevant hardware through a computer program. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, it may implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal, and software distribution medium capable of carrying the computer program code. It should be noted that the content contained in the computer-readable storage medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0091] Furthermore, the present invention also provides an electronic device. In an embodiment of the electronic device according to the present invention, Figure 10 As shown, the electronic device includes a processor 41 and a storage device 42. The storage device can be configured to store a program for executing the image processing method based on vehicle body matting according to the above-described method embodiment. The processor can be configured to execute the program in the storage device, including but not limited to a program for executing the image processing method based on vehicle body matting according to the above-described method embodiment. For ease of explanation, only the portions relevant to the embodiments of the present invention are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of the present invention.

[0092] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the image processing method based on vehicle body matting of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned image processing method based on vehicle body matting. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0093] Furthermore, it should be understood that since the configuration of each module is merely for the purpose of illustrating the functional units of the apparatus of the present invention, the physical devices corresponding to these modules may be the processor itself, or a portion of the software in the processor, a portion of the hardware, or a combination of software and hardware. Therefore, the number of modules in the figure is merely illustrative.

[0094] Those skilled in the art will appreciate that the various modules in the device can be adaptively split or merged. Such splitting or merging of specific modules does not cause the technical solution to deviate from the principles of the present invention. Therefore, the technical solutions after splitting or merging will fall within the scope of protection of the present invention.

[0095] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. An image processing method based on vehicle body matting, characterized in that: The method comprises: Get the original image taken by the camera; Acquire a connected area of ​​the vehicle body based on the original image; Performing directional convolution on the original image based on the connected area to obtain a directional convolution result; Performing fuzzy processing on the connected area to obtain a first image; Determine a second image based on the first image, the original image, and the directional convolution result; The performing directional convolution on the original image based on the connected region to obtain a directional convolution result includes: Performing circumscribed rectangle fitting on the connected area to obtain a circumscribed rectangle image; Calculating the angle between the long side of the circumscribed rectangular image and the horizontal direction; Determining whether the angle is within a preset angle range; When the included angle is within a preset angle range, directional convolution is performed on the original image based on the included angle and the radius of the convolution kernel to obtain the directional convolution result.

2. The image processing method based on vehicle body matting according to claim 1, characterized in that: Acquiring a connected area of ​​a vehicle body based on the original image includes: Extracting a vehicle body region from the original image; Extracting all connected regions from the vehicle body region; Determine whether the number of all connected areas is greater than zero; When the number of all connected regions is greater than zero, the connected region with the largest area is used as the connected region of the vehicle body.

3. The image processing method based on vehicle body matting according to claim 2, characterized in that: Extracting a vehicle body region from the original image includes: extracting the vehicle body region from the original image using a MODNet model.

4. The image processing method based on vehicle body matting according to claim 1, characterized in that: The preset angle range is [-60°, 60°].

5. The image processing method based on vehicle body matting according to claim 1, characterized in that: The method of performing blur processing on the connected region to obtain a first image includes: performing blur processing on the connected region using a Box Blur technique to obtain the first image.

6. The image processing method based on vehicle body matting according to claim 1, characterized in that: Determining a second image based on the first image, the original image and the directional convolution result includes: based on the first image, performing nonlinear mixing on the original image and the directional convolution result to obtain the second image.

7. An image processing device based on vehicle body matting, characterized in that: The device comprises: A first acquisition module is configured to acquire an original image captured by a camera; a second acquisition module configured to acquire a connected area of ​​a vehicle body based on the original image; a directional convolution module, configured to perform directional convolution on the original image based on the connected region to obtain a directional convolution result; a fuzzy processing module, configured to perform fuzzy processing on the connected area to obtain a first image; a determination module configured to determine a second image based on the first image, the original image, and the directional convolution result; The performing directional convolution on the original image based on the connected region to obtain a directional convolution result includes: Performing circumscribed rectangle fitting on the connected area to obtain a circumscribed rectangle image; Calculating the angle between the long side of the circumscribed rectangular image and the horizontal direction; Determining whether the angle is within a preset angle range; When the included angle is within a preset angle range, directional convolution is performed on the original image based on the included angle and the radius of the convolution kernel to obtain the directional convolution result.

8. The image processing device based on vehicle body matting according to claim 7, characterized in that: The preset angle range is [-60°, 60°].

9. An electronic device comprising a processor and a storage device, wherein the storage device is adapted to store a plurality of program codes, wherein: The program code is suitable for being loaded and run by the processor to execute the image processing method based on vehicle body matting according to any one of claims 1 to 6.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the image processing method based on vehicle body matting according to any one of claims 1 to 6.

Images