Vehicle cutout method, system, device and storage medium based on neural network

Abstract

The present invention provides a vehicle cutout method, system, device and storage medium based on a neural network, the method comprising: inputting an image to be identified into a trained first convolutional neural network to extract multiple feature information of each pixel in the image information, the multiple feature information at least including color information and object identification information of each pixel; when there is a pixel whose object identification information is a vehicle, a morphological operation is performed on a region of a pixel set whose object identification information is a vehicle, and the region after the edge of the region is corroded and expanded is obtained; the multiple feature information of the pixel in the region to be identified is input into a trained second convolutional neural network to output at least a trajectory to be cutout of a vehicle; and a closed graphic region surrounded by the trajectory to be cutout in the image to be identified is cutout. The present invention can greatly improve the processing efficiency in large-scale image processing and ensure the cutout quality, and has a high application value.

Description

Background Art

[0002] Image automatic cutout technology is a branch of digital image processing and an important image editing technology. Image cutout is used to extract the outline of the target of interest in the image, avoiding the interference of the cluttered background, ensuring the effective display of the information of the displayed object, and improving the aesthetics of the picture display. Therefore, image cutout has a wide range of applications in Internet image display.

[0003] Automatic cutout of vehicle model images is an application field of automatic image cutout technology. Compared with other objects such as product images, the cutout of vehicle model images is more difficult because, in addition to displaying the main vehicle model content, vehicle images usually contain non-pure color background information that is irrelevant to the main body of the vehicle model, such as roads, sky, background buildings, etc. In addition, some images may contain other information that interferes with the vehicle model, such as non-main vehicle models, irrelevant significant figures, etc. In addition, the contour curve of the vehicle model is irregular, which requires high precision for contour extraction. Due to the high complexity of the cutout processing of vehicle model images, manual PS processing of vehicle model images is usually time-consuming and labor-intensive.

[0004] In view of this, the present invention proposes a vehicle cutout method, system, device and storage medium based on a neural network. Summary of the invention

[0005] In response to the problems in the prior art, the purpose of the present invention is to provide a vehicle cutout method, system, device and storage medium based on a neural network, which overcomes the difficulties of the prior art, can greatly improve the processing efficiency in large-scale image processing, and ensure the cutout quality, and has high application value.

[0006] An embodiment of the present invention provides a vehicle cutout method based on a neural network, comprising the following steps:

[0007] S110, inputting the image to be identified into a first trained convolutional neural network to extract multiple feature information of each pixel in the image information, wherein the multiple feature information includes at least color information of each pixel and object identification information;

[0008] S120, determining whether there is a pixel point whose object identification information is a vehicle, if so, executing step S130, if not, ending;

[0009] S130, performing morphological operations on the region of the pixel set whose object identification information is a vehicle, and obtaining the region to be identified by corroding and dilating the edge of the region;

[0010] S140, inputting multiple feature information of pixel points in the area to be identified into a trained second convolutional neural network to output at least a trajectory of the vehicle to be cut out;

[0011] S150, performing cropping on a closed graphic area surrounded by the to-be-cropped trajectory in the to-be-recognized image.

[0012] Preferably, the step S110 includes the following steps:

[0013] S111, inputting the image to be recognized into a trained first convolutional neural network to perform image segmentation based on the image;

[0014] S112, obtaining object identification information D corresponding to each segmented image region in the image through a first convolutional neural network; and

[0015] S113. The multiple feature information of each pixel includes the RGB information of the pixel and the object identification information D in the image area where the pixel is located.

[0016] Preferably, before step S110, the following steps are also included:

[0017] Acquire a first predetermined number of learning images at least including vehicles and recognition results of each graphic area in the learning images;

[0018] The learning image and the recognition results of each graphic area are input into the convolutional network, the extended network and the Softmax layer of the original convolutional neural network model, the original neural network model is trained, and the first convolutional neural network is obtained.

[0019] Preferably, before step S110, the method further includes: using a large number of vehicle images with multiple feature information to train the second convolutional neural network, the second convolutional neural network outputs the category of the vehicle and the trajectory of the vehicle to be cut out, and the trajectory to be cut out is a closed pattern.

[0020] Preferably, in step S130, the background area and the foreground area whose regions remain unchanged after the morphological operation are used as the area to be identified and the background area, respectively.

[0021] Preferably, in step S140, only the area to be identified with the largest area in the image to be identified is input into the trained second convolutional neural network to obtain the contour trajectory of the vehicle in the area to be identified.

[0022] Preferably, in step S140, only each of the areas to be identified is input into the trained second convolutional neural network to obtain the contour trajectory of the vehicle in each of the areas to be identified, and the contour trajectory with the largest total pixel area enclosed by the contour in the image is output as the trajectory to be cut out.

[0023] An embodiment of the present invention further provides a vehicle cutout system based on a neural network, which is used to implement the above-mentioned vehicle cutout method based on a neural network. The vehicle cutout system based on a neural network includes:

[0024] The region extraction module inputs the image to be identified into a first trained convolutional neural network to extract multiple feature information of each pixel in the image information, wherein the multiple feature information includes at least color information of each pixel and object identification information;

[0025] A region determination module determines whether there is a pixel point whose object identification information is a vehicle, and if so, executes a morphological operation module;

[0026] A morphological operation module performs a morphological operation on the region of the pixel set whose object identification information is a vehicle, and obtains the region to be identified after corroding and dilating the edge of the region;

[0027] A cutout trajectory module, which inputs multiple feature information of pixel points in the area to be identified into a trained second convolutional neural network and outputs at least a cutout trajectory of the vehicle;

[0028] The area cutout module cuts out the closed graphic area surrounded by the to-be-cutout trajectory from the to-be-recognized image.

[0029] An embodiment of the present invention further provides a vehicle cutout device based on a neural network, comprising:

[0030] processor;

[0031] a memory storing executable instructions of the processor;

[0032] Wherein, the processor is configured to perform the steps of the above-mentioned neural network-based vehicle cutout method by executing the executable instructions.

[0033] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of the above-mentioned vehicle cutout method based on a neural network.

[0034] The purpose of the present invention is to provide a vehicle cutout method, system, device and storage medium based on a neural network, which can greatly improve the processing efficiency in large-scale image processing and ensure the cutout quality, and has high application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objectives and advantages of the present invention will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.

[0036] Figure 1It is a flow chart of the vehicle cutout method based on neural network of the present invention.

[0037] Figures 2 to 5 It is a process diagram of the process of implementing the vehicle cutout method based on neural network of the present invention.

[0038] Figure 6 It is a structural schematic diagram of the vehicle cutout system based on neural network of the present invention.

[0039] Figure 7 It is a structural schematic diagram of a vehicle cutout device based on a neural network of the present invention.

[0040] Figure 8 It is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted.

[0042] Figure 1 FIG. 1 is a flow chart of the vehicle cutout method based on a neural network of the present invention. Figure 1 As shown, an embodiment of the present invention provides a vehicle cutout method based on a neural network, comprising the following steps:

[0043] S110, inputting the image to be identified into a trained first convolutional neural network to extract multiple feature information of each pixel in the image information, where the multiple feature information at least includes color information of each pixel and object identification information.

[0044] S120, determine whether there is a pixel point with object identification information of a vehicle, if so, execute step S130, if not, execute step S160.

[0045] S130, performing morphological operations on the region of the pixel set whose object identification information is a vehicle, and obtaining the region to be identified by corroding and dilating the edge of the region.

[0046] S140, inputting multiple feature information of pixel points in the area to be identified into a trained second convolutional neural network to output at least a trajectory of a vehicle to be cut out.

[0047] S150, cutting out a closed graphic area surrounded by the cutting out trajectory in the image to be recognized.

[0048] S160, end.

[0049] In a preferred embodiment, step S110 includes the following steps:

[0050] S111, inputting the image to be recognized into a trained first convolutional neural network for image segmentation based on the image.

[0051] S112, obtaining object identification information D corresponding to each segmented image region in the image through a first convolutional neural network. And

[0052] S113. The multiple feature information of each pixel includes the RGB information of the pixel and the object identification information D in the image area where the pixel is located.

[0053] In a preferred embodiment, before step S110, the following steps are also included:

[0054] A first predetermined number of learning images at least including the vehicle and recognition results of each graphic area in the learning images are obtained.

[0055] The learning image and the recognition results of each graphic area are input into the convolutional network, the extended network and the Softmax layer of the original convolutional neural network model, the original neural network model is trained, and the first convolutional neural network is obtained.

[0056] In a preferred embodiment, a large number of photos with sub-regions and object identifications are used to learn and train the first convolutional neural network, so that the first convolutional neural network can automatically differentiate the input photos into regions, perform image recognition on each region separately, and output the object identification of each region.

[0057] In a preferred embodiment, a large amount of point cloud information with multiple feature information is used to train the second convolutional neural network, so that the second convolutional neural network can accurately output the category of the 3D target. In the present invention, the difference between training the first convolutional neural network and the second convolutional neural network is that the first convolutional neural network has the functions of regional differentiation and primary image recognition, while the second convolutional neural network, compared with the first convolutional neural network, does not have the function of regional differentiation, but only has the function of accurate recognition of the region.

[0058] In a preferred embodiment, before step S110, the method further includes: using a large number of vehicle images with multiple feature information to train a second convolutional neural network, the second convolutional neural network outputs the category of the vehicle and the trajectory of the vehicle to be cut out, and the trajectory to be cut out is a closed pattern.

[0059] In a preferred embodiment, in step S130, the background area and the foreground area whose regions remain unchanged after the morphological operation are used as the area to be identified and the background area, respectively.

[0060] In a preferred embodiment, in step S140, only the region to be identified with the largest area in the image to be identified is input into the trained second convolutional neural network to obtain the contour trajectory of the vehicle in the region to be identified.

[0061] In a preferred embodiment, in step S140, only each area to be identified is input into the trained second convolutional neural network to obtain the contour trajectory of the vehicle in each area to be identified, and the contour trajectory with the largest total pixel area enclosed by the contour in the image is output as the trajectory to be cut out.

[0062] Figures 2 to 5 1 is a schematic diagram of the process of implementing the vehicle cutout method based on a neural network of the present invention. Figures 2 to 5 A specific implementation process of the present invention is introduced as shown below:

[0063] First, a first predetermined number of learning images containing at least a vehicle and recognition results of each graphic area in the learning images are obtained. The learning images and the recognition results of each graphic area are input into the convolutional network, the extended network and the Softmax layer of the original convolutional neural network model, and the original neural network model is trained to obtain a first convolutional neural network.

[0064] A large number of vehicle images with multiple feature information are used to train the second convolutional neural network, and the second convolutional neural network outputs the category of the vehicle and the trajectory of the vehicle to be cut out, and the trajectory to be cut out is a closed pattern. For example: a large number of point cloud information with multiple feature information is used to train the second convolutional neural network, so that the second convolutional neural network can accurately output the category of the 3D target and the range in the image, and the multiple feature information at least includes the color information RGB of each pixel and the object identification information D.

[0065] refer to Figure 2, input the image 10 to be identified, which includes the vehicles 1 and 2 parked on the ground, into the trained first convolutional neural network to extract the multiple feature information of each pixel in the image information, and the multiple feature information includes at least the color information and object identification information of each pixel. The first convolutional neural network is used to identify and add object identification information to each pixel. For example, the multiple feature information of each pixel in the pattern where the vehicle 2 is located is (R, G, B, D), and the object identification information D of this part of the pixels is the vehicle. Similarly, the multiple feature information of each pixel in the pattern where the vehicle 1 is located is (R, G, B, D), and the object identification information D of this part of the pixels is the vehicle. The multiple feature information of the remaining pixels is (R, G, B, D), and the object identification information D of this part of the pixels is the ground. In this embodiment, the first convolutional neural network plays a role in quickly locating the position of the vehicle in the picture, which is equivalent to a rough cutout operation.

[0066] refer to Figure 3 Since there are pixels with vehicle identification information, the region with vehicle identification information is subjected to morphological operation, and the region to be identified is obtained after the edge of the region is corroded and expanded. In order to prevent the rough cutout operation of the first convolutional neural network from destroying the edge contour of the vehicle, a certain area expansion is performed to obtain a pattern range that includes the complete vehicle.

[0067] refer to Figure 4 , through area comparison, the area of ​​the image area occupied by vehicle 2 is larger than the area of ​​the image area occupied by vehicle 1, then only the multiple feature information (R, G, B, D) of the pixel points of the largest area to be identified in the image 10 to be identified (the image area occupied by vehicle 2) is input into the trained second convolutional neural network, and the second convolutional neural network outputs a trajectory to be cut out of a vehicle (based on the image area occupied by vehicle 2). The second convolutional neural network is used to finely cut out the pattern range containing the complete vehicle, so as to obtain a more accurate vehicle contour.

[0068] refer to Figure 5 , the closed graphic area surrounded by the cutout trajectory in the image 10 to be recognized is cutout.

[0069] The present invention takes into account both the speed and accuracy of image cutting by using a combination of a first convolutional neural network and a second convolutional neural network that realize two functions, and can accurately cut out the most obvious vehicle in the image.

[0070] The problem to be solved by the present invention is that after removing the background interference information from the car image, it is convenient for display and marketing on the Internet platform, which can improve the user's browsing experience. Due to the complex structure of the car image, the current manual method of cutting out the car image has low processing efficiency and relatively expensive labor costs. The present invention proposes a method for automatically cutting out the car model image based on image instance segmentation and image cutting technology, which realizes the automatic cutting out of the car model image, can greatly save labor costs, and improve the efficiency of intelligent image processing.

[0071] The present invention discloses a method for automatically cutting out a vehicle model image based on instance segmentation and an automatic image cutting out method, comprising:

[0072] The following steps:

[0073] Step 1: Build an instance segmentation model

[0074] The proposed image salient instance vehicle body segmentation model includes (1) generating a set of prototype masks, and (2) predicting mask coefficients for each instance. Then, instance masks are generated by linearly combining the prototypes with the template coefficients. Since this process does not rely on repooling, this method can produce very high-quality masks. In addition, the emergent behavior of the prototypes is analyzed and shown to be fully convolutional. Finally, the prediction head is optimized with better anchor scales and aspect ratios by incorporating deformable convolutions into the backbone network, and a novel fast mask re-scoring branch is added.

[0075] Step 2: Generate a three-element graph

[0076] Based on the instance segmentation results, the car Figure 3 Generation of element graph. Specifically, first, replace the background with color or other methods, and then perform bitwise AND or OR operations on the background and foreground to merge the two into a new image. Erode and dilate operations can also be performed on the edge of the background to shrink or expand the outline. The changed area of ​​dilation and erosion is used as the uncertain area of ​​the three-element graph. The background and foreground parts of the unchanged area before and after the morphological operation are used as the target area and background area of ​​the three-element graph respectively.

[0077] Step 3: Train the cutout model

[0078] The purpose of the matting model is to make the contour of the uncertain region become detailed and realistic. Specifically, the loss function is a weighted combination of two losses: alpha loss and compositional loss. Alpha loss is the error between the predicted alpha and the actual alpha, and compositional loss is the error between the predicted alpha and the ground truth RGB image after it is fused with the corresponding background and foreground through formula (1). Only the prediction error of the unknown region in the trimap will be back-propagated (this is the difference between matting and background removal. Matting requires trimap as a user interaction interface and only needs to predict the uncertain region in the matting, while backgournd removal requires the error of the entire image to be back-propagated). That is, the pure background and pure foreground parts in the trimap are not included in the model prediction range, and these two parts only need to be directly copied to the output alpha.

[0079] Step 4: Prediction and generation of the main part of the car image

[0080] It can accurately extract the target object from the picture. In order to achieve this high-quality automatic cutout effect, we use a self-developed algorithm to ensure the integrity of the cutout object and the accuracy of local details. Now the tool has obvious automation features. You only need to upload the picture to extract the target from the picture with high precision. If you are not satisfied, you can make adjustments with a few strokes.

[0081] Step 5: Output the cutout results

[0082] According to the preset background requirements, the result image can be output with a solid color or transparent background.

[0083] The present invention constructs an instance segmentation model for extracting the rough outline of the main body of the car image based on the instance segmentation data set. Then, through the morphological operation of the image, a three-element map for image cutout is generated. Then, based on the cutout data of the open source object, a cutout model for accurate cutout of the car image is synthesized. After the three-element map is input into the cutout model, a high-quality segmentation map can be produced. Practice has verified that the manual verification of the algorithm proposed in the present invention can reach a satisfaction rate of 98%. On average, a picture can return the cutout result within 1 second, which improves the automation efficiency and greatly saves labor costs.

[0084] Figure 6 FIG. 1 is a schematic diagram of the structure of the vehicle cutout system based on a neural network of the present invention. Figure 6 As shown, the vehicle cutout system 5 based on neural network of the present invention comprises:

[0085] The region extraction module 51 inputs the image to be identified into a first trained convolutional neural network to extract multiple feature information of each pixel in the image information, wherein the multiple feature information includes at least color information of each pixel and object identification information;

[0086] The area determination module 52 determines whether there is a pixel point whose object identification information is a vehicle, and if so, executes the morphological operation module;

[0087] A morphological operation module 53 performs a morphological operation on the region of the pixel set whose object identification information is a vehicle, and obtains a region to be identified by corroding and dilating the edge of the region;

[0088] A cutout trajectory module 54 inputs multiple feature information of pixels in the area to be identified into a trained second convolutional neural network to output at least a cutout trajectory of the vehicle;

[0089] The region cutout module 55 cuts out the closed graphic region surrounded by the to-be-cutout trajectory from the to-be-recognized image.

[0090] The vehicle cutout system based on neural network of the present invention can greatly improve the processing efficiency in large-scale image processing and ensure the cutout quality, and has high application value.

[0091] The embodiment of the present invention further provides a vehicle cutout device based on a neural network, comprising a processor and a memory, wherein executable instructions of the processor are stored, wherein the processor is configured to execute the steps of the vehicle cutout method based on a neural network by executing the executable instructions.

[0092] As shown above, the vehicle cutout system based on neural network of the embodiment of the present invention can greatly improve the processing efficiency in large-scale image processing and ensure the cutout quality, and has high application value.

[0093] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0094] Figure 7 Schematic diagram of the structure of the vehicle cutout device based on neural network of the present invention. Figure 7 The electronic device 600 according to this embodiment of the present invention is described. Figure 7 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0095] like Figure 7 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0096] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of various exemplary embodiments of the present invention described in the electronic prescription circulation processing method section of this specification. For example, the processing unit 610 can execute the following steps: Figure 1 Follow the steps shown in .

[0097] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0098] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0099] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0100] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0101] The embodiment of the present invention also provides a computer-readable storage medium for storing a program, and the steps of the vehicle cutout method based on a neural network are implemented when the program is executed. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above-mentioned electronic prescription circulation processing method section of this specification.

[0102] As shown above, the vehicle cutout system based on neural network of the embodiment of the present invention can greatly improve the processing efficiency in large-scale image processing and ensure the cutout quality, and has high application value.

[0103] Figure 8 Schematic diagram of the structure of the computer-readable storage medium of the present invention. Figure 8 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0104] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0105] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0106] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0107] In summary, the purpose of the present invention is to provide a vehicle cutout method, system, device and storage medium based on a neural network. The vehicle cutout system based on a neural network of the present invention can greatly improve the processing efficiency in large-scale image processing and ensure the cutout quality, and has a high application value.

[0108] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A vehicle cutout method based on a neural network, characterized in that: The following steps are involved: S110, inputting the image to be identified into a trained first convolutional neural network to perform image segmentation based on the image; obtaining object identification information D corresponding to each segmented image region in the image through the first convolutional neural network; the multiple feature information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located; S120, determining whether there is a pixel point whose object identification information is a vehicle, if so, executing step S130, if not, ending; S130, performing a morphological operation on the region of the pixel set whose object identification information is a vehicle, corroding and dilating the edge of the region to obtain a region to be identified, and using the background region and the foreground region that remain unchanged after the morphological operation as the region to be identified and the background region, respectively; S140, inputting multiple feature information of the pixel points in the to-be-recognized area into a trained second convolutional neural network, obtaining contour trajectories of the vehicles in each of the to-be-recognized areas, and outputting the contour trajectory with the largest total pixel area enclosed by the contour in the image as the to-be-cutout trajectory, and outputting at least one to-be-cutout trajectory of the vehicle; S150, performing cropping on a closed graphic area surrounded by the to-be-cropped trajectory in the to-be-recognized image.

2. The vehicle cutout method based on neural network according to claim 1, characterized in that: Before step S110, the following steps are also included: Acquire a first predetermined number of learning images at least including vehicles and recognition results of each graphic area in the learning images; The learning image and the recognition results of each graphic area are input into the convolutional network, the extended network and the Softmax layer of the original convolutional neural network model, the original neural network model is trained, and the first convolutional neural network is obtained.

3. The vehicle cutout method based on neural network according to claim 1, characterized in that: Before step S110, the method also includes: using a large number of vehicle images with multiple feature information to train the second convolutional neural network, the second convolutional neural network outputs the category of the vehicle and the trajectory of the vehicle to be cut out, and the trajectory to be cut out is a closed pattern.

4. A vehicle cutout system based on a neural network, used to implement the vehicle cutout method based on a neural network as claimed in claim 1, characterized in that: include: A region extraction module, inputting the image to be identified into a trained first convolutional neural network to perform image segmentation based on the image; Obtaining object identification information D corresponding to each segmented image region in the image through a first convolutional neural network; the multiple feature information of each pixel includes RGB information of the pixel and object identification information D in the image region where the pixel is located; A region determination module determines whether there is a pixel point whose object identification information is a vehicle, and if so, executes a morphological operation module; A morphological operation module performs a morphological operation on the region of the pixel set whose object identification information is a vehicle, obtains the region to be identified after corroding and dilating the edge of the region, and uses the background region and the foreground region that remain unchanged after the morphological operation as the region to be identified and the background region respectively; A cutout trajectory module, inputting multiple feature information of pixel points in the to-be-identified area into a trained second convolutional neural network, obtaining contour trajectories of vehicles in each of the to-be-identified areas, and outputting the contour trajectory with the largest total pixel area enclosed by the contour in the image as the to-be-cutout trajectory, and outputting at least one to-be-cutout trajectory of the vehicle; The area cutout module cuts out the closed graphic area surrounded by the to-be-cutout trajectory from the to-be-recognized image.

5. A vehicle cutout device based on a neural network, characterized in that: include: processor; a memory storing executable instructions of the processor; Wherein, the processor is configured to perform the steps of the neural network-based vehicle cutout method according to any one of claims 1 to 3 by executing the executable instructions.

6. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the vehicle cutout method based on a neural network as described in any one of claims 1 to 3 are implemented.

Images