Image processing method, processing device and computer readable storage medium

Abstract

The application provides an image processing method, an image processing device and a computer readable storage medium. The image processing method comprises: acquiring an image to be processed, the resolution of the image being a first resolution; dividing the image into a plurality of sub-images, the resolution of each sub-image being a second resolution, the second resolution being less than the first resolution; inputting the plurality of sub-images into a neural network model for parallel operation to identify road traffic targets in the sub-images. The method first acquires an image to be processed, then divides the image to be processed into a plurality of sub-images, and then simultaneously inputs the plurality of sub-images into a neural network model for parallel operation. Although the resolution of the sub-images is less than the resolution of the image to be processed, all the sub-images include all the features of the image to be processed. The neural network model performs parallel operation on the plurality of sub-images, thereby ensuring the real-time performance of image processing while ensuring the resolution.

Description

Technical Field

[0001] This application relates to the field of image recognition for autonomous driving, and more specifically, to an image processing method, processing device, computer-readable storage medium, and processor. Background Technology

[0002] In recent years, deep learning neural network technology has significantly improved the performance of target recognition and detection in camera images. Therefore, deep learning target detection technology is increasingly being applied to target detection in autonomous driving camera perception systems. In addition to network model design, data training is a crucial aspect of deep learning neural network target detection applications. Training with massive amounts of data is essential to obtaining the weight parameters of tens or even hundreds of millions of neurons in the trained network model. The quality of the training data directly determines the accuracy of the neural network when used for real-time vehicle detection.

[0003] The applicant, through research on deep learning neural network models for object detection in the field of target perception from autonomous driving camera images, discovered the following contradictions and shortcomings in data processing:

[0004] 1. The conflict between resolution and real-time performance:

[0005] Due to the complexity of traffic scenarios, the distribution of targets is always random, from near to far and from left to right, and the size and type of targets are also randomly distributed. Therefore, to achieve good detection accuracy in autonomous driving perception, the resolution of the camera output image should be as high as possible so that targets at a distance from the vehicle can be accurately detected. However, because the bandwidth and computing power of onboard transmission and computing equipment are limited, the resolution of the camera output image cannot be increased indiscriminately. Doing so would crowd out the transmission bandwidth of other sensors and increase the time it takes for the onboard computing equipment to process an image by a factor of two, causing the autonomous driving perception processing to fail to meet the required real-time requirements (for cameras, a processing frequency of 30Hz is generally required).

[0006] 2. Resource Waste: To ensure a long field of view, cameras are typically mounted on vehicles at a near-horizontal angle, resulting in images where the upper half usually contains a large area of ​​sky. The actual targets to be detected are primarily located in the lower half of the image. Therefore, the image data representing the upper half wastes valuable resources of the vehicle's transmission and computing equipment. Summary of the Invention

[0007] The main objective of this application is to provide an image processing method, processing device, computer-readable storage medium, and processor to solve the problem of the contradiction between image resolution and real-time performance in autonomous driving perception in the prior art.

[0008] To achieve the above objectives, according to one aspect of this application, an image processing method is provided, comprising: acquiring an image to be processed, the image having a first resolution; segmenting the image into multiple sub-images, each sub-image having a second resolution, the second resolution being less than the first resolution; inputting the multiple sub-images into a neural network model for parallel computation to identify road traffic targets in the sub-images.

[0009] Furthermore, after segmenting the image into multiple sub-images, and before inputting the multiple sub-images into a neural network model for parallel computation to identify the road traffic target in the sub-images, the method further includes: discarding some sub-images in the sub-images, wherein the discarded sub-images do not include the road traffic target.

[0010] Furthermore, the method also includes overlapping portions of any two adjacent sub-images to form an overlapping region.

[0011] Furthermore, the method further includes: determining whether the road traffic target is within the overlapping area; and performing overlap processing on the road traffic target if the road traffic target is within the overlapping area.

[0012] Further, the overlapping processing of the road traffic target includes: defining two adjacent sub-images as a left image and a right image; obtaining a first bounding box, the first bounding box being the bounding box of the portion of the road traffic target in the left image; obtaining a second bounding box, the second bounding box being the bounding box of the portion of the road traffic target in the right image; determining whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target based at least on the sum of the areas and the overlapping area, wherein the sum of the areas is the sum of the areas of the first bounding box and the second bounding box, and the overlapping area is the area of ​​the overlapping portion of the first bounding box and the second bounding box.

[0013] Further, determining whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target, at least based on the sum of their areas and the overlapping area, includes: determining that the road traffic target in the left image and the road traffic target in the right image are the same road traffic target if the area ratio is greater than or equal to a predetermined value, wherein the area ratio is the ratio of the overlapping area to the sum of their areas; and determining that the road traffic target in the left image and the road traffic target in the right image are not the same road traffic target if the area ratio is less than the predetermined value.

[0014] Further, determining whether the road traffic target is within the overlapping area includes: obtaining the coordinates of the four vertices of the first bounding box; obtaining the coordinates of the four vertices of the second bounding box; and determining whether the road traffic target is within the overlapping area based on the coordinates of the four vertices of the first bounding box, the coordinates of the four vertices of the second bounding box, the size of the sub-image, and the size of the overlapping area.

[0015] According to another aspect of this application, an image processing apparatus is provided, comprising: an acquisition unit for acquiring an image to be processed, the image having a first resolution; a segmentation unit for segmenting the image into multiple sub-images, each sub-image having a second resolution, the second resolution being less than the first resolution; and a computation unit for inputting the multiple sub-images into a neural network model for parallel computation to identify road traffic targets in the sub-images.

[0016] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the image processing methods described above.

[0017] According to another aspect of this application, a processor is provided for running a program, wherein the program, when running, performs any of the image processing methods described above.

[0018] By applying the technical solution of this application, the image to be processed is first acquired, then the image to be processed is divided into multiple sub-images, and then the multiple sub-images are simultaneously input into the neural network model for parallel operation. Although the resolution of the sub-images is smaller than that of the image to be processed, all the sub-images include all the features of the image to be processed. The neural network model performs parallel operation on multiple sub-images, which ensures both resolution and real-time image processing. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 A flowchart illustrating an image processing method according to an embodiment of this application is shown;

[0021] Figure 2 A schematic diagram illustrating the overlapping processing of road traffic targets according to an embodiment of this application is shown;

[0022] Figure 3A schematic diagram of an image processing apparatus according to an embodiment of this application is shown. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element, or there may be an intermediate element present. Furthermore, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element, or "connected" to the other element via a third element.

[0027] As described in the background section, there is a contradiction between image resolution and real-time performance in autonomous driving perception in the prior art. To solve the above-mentioned problem of the contradiction between image resolution and real-time performance in autonomous driving perception, embodiments of this application provide an image processing method, processing device, computer-readable storage medium, and processor.

[0028] According to embodiments of this application, an image processing method is provided.

[0029] Figure 1 This is a flowchart of an image processing method according to an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0030] Step S101: Obtain the image to be processed, wherein the resolution of the image is the first resolution;

[0031] Step S102: Divide the above image into multiple sub-images, and the resolution of each sub-image is a second resolution, which is smaller than the first resolution.

[0032] Step S103: Input multiple of the above sub-images into the neural network model for parallel processing to identify road traffic targets in the above sub-images.

[0033] Specifically, multiple sub-images mentioned above are input into the neural network model for parallel computation, including: inputting the GPU's parallel data processing mode (or CPU multi-process mode) into the neural network through the algorithm system call for parallel processing.

[0034] Specifically, the images to be processed are acquired by cameras installed on autonomous vehicles. In order to ensure high detection accuracy, the images captured by the cameras have high resolution. However, high-resolution images will occupy the transmission bandwidth of other sensors and increase the time it takes for the on-board computing device to process an image by a factor of two, resulting in the autonomous driving perception processing failing to meet the required real-time requirements.

[0035] Specifically, assuming the original output image resolution of the camera installed on an autonomous vehicle is 2048×1536 (aspect ratio of 4:3), and the deep learning neural network model requires an input image resolution (i.e., size) of 640×640, conventional methods typically reduce the original 2048×1536 image to 640×640 before inputting it into the neural network for computation. However, this solution can extract five 640×640 sub-images from the original output image, specifically from the bottom edge of the original output image. These sub-images are divided into two layers: the first layer consists of two sub-images positioned in the middle of the image; the second layer consists of three sub-images positioned at the bottom of the image.

[0036] In the above scheme, the image to be processed is first acquired, then the image to be processed is divided into multiple sub-images, and then the multiple sub-images are simultaneously input into the neural network model for parallel operation. Although the resolution of the sub-images is smaller than that of the image to be processed, all the sub-images include all the features of the image to be processed. The neural network model performs parallel operation on multiple sub-images, which ensures both resolution and real-time image processing.

[0037] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0038] In one embodiment of this application, after segmenting the image into multiple sub-images, and before inputting the multiple sub-images into a neural network model for parallel computation to identify road traffic targets in the sub-images, the method further includes: discarding a portion of the sub-images, wherein the discarded sub-images do not include the road traffic targets. Generally, to ensure a long field of view, cameras are typically mounted on vehicles at near-horizontal shooting angles, and the upper half of the captured image usually contains a large area of ​​sky. The actual road traffic targets to be detected are primarily distributed in the lower half of the image. Therefore, the image data equivalent to the upper half wastes valuable resources of the onboard transmission and computing equipment. By discarding sub-images that do not include road traffic targets, valuable resources of the onboard transmission and computing equipment are saved. Furthermore, discarding a portion of the sub-images increases the speed of image processing and indirectly reduces the resolution of the image to be processed. In other words, the method of this invention maintains the resolution of the image input to the neural network computation, discards the resource waste caused by computation in invalid areas, and ensures that real-time performance is not affected by parallel computation. This significantly improves the accuracy of neural network target detection.

[0039] In one embodiment of this application, to avoid losing small targets near the boundaries of sub-images and to ensure that the small targets have a relatively complete shape in at least one sub-image, the method further includes overlapping portions of any two adjacent sub-images to form an overlapping region. Specifically, the adjacent boundaries of any two adjacent sub-images can overlap by 30 pixels. Existing solutions reduce both the width and height of the image. For some small road traffic targets, the reduction in image size leads to severe feature loss, hindering accurate detection by image recognition algorithms. This solution, however, simply segments the original image without losing the features of small road traffic targets.

[0040] In one embodiment of this application, the method further includes: determining whether the road traffic target is within the overlapping area; and performing overlap processing on the road traffic target if it is within the overlapping area. That is, when it is determined that the road traffic target is within the overlapping area, overlap processing is required to achieve accurate identification of the road traffic target.

[0041] In one embodiment of this application, such as Figure 2As shown, the overlapping processing of the aforementioned road traffic targets includes: defining two adjacent sub-images as a left image and a right image; obtaining a first bounding box, which is the bounding box of the portion of the road traffic target in the left image; obtaining a second bounding box, which is the bounding box of the portion of the road traffic target in the right image; determining whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target based at least on the sum of their areas and the overlapping area, where the sum of their areas is the sum of the areas of the first bounding box and the second bounding box, and the overlapping area is the area of ​​the overlapping portion of the first bounding box and the second bounding box. Since there may be multiple road traffic targets in the overlapping area, to achieve accurate identification of the road traffic targets, it is necessary to determine whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target. A preferred method to determine whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target within the overlapping area of ​​two adjacent sub-images is through the sum of their areas and the overlapping area.

[0042] In one embodiment of this application, determining whether a road traffic target in the left image and a road traffic target in the right image are the same road traffic target, at least based on their area sum and overlapping area, includes: determining that the road traffic target in the left image and the road traffic target in the right image are the same road traffic target when the area ratio is greater than or equal to a predetermined value, wherein the area ratio is the ratio of the overlapping area to the sum of the areas; and determining that the road traffic target in the left image and the road traffic target in the right image are not the same road traffic target when the area ratio is less than the predetermined value. Specifically, the predetermined value can be set to 0.5; of course, those skilled in the art can set appropriate predetermined values ​​according to actual circumstances.

[0043] In one embodiment of this application, determining whether the road traffic target is within the overlapping area includes: obtaining the coordinates of the four vertices of the first bounding box; obtaining the coordinates of the four vertices of the second bounding box; and determining whether the road traffic target is within the overlapping area based on the coordinates of the four vertices of the first bounding box, the coordinates of the four vertices of the second bounding box, the size of the sub-image, and the size of the overlapping area.

[0044] In one specific embodiment of this application, the specific implementation method for determining whether the above-mentioned road traffic target is within the above-mentioned overlapping area is as follows: the bounding box of the target car detected in the left image is ABCD (A is the upper left corner, B is the upper right corner, C is the lower right corner, and D is the lower left corner), and the bounding box detected in the right image is EFGH (E is the upper left corner, F is the upper right corner, G is the lower right corner, and H is the lower left corner).

[0045] First, determine whether bounding box ABCD is entirely or partially within the overlapping area. The determination condition is:

[0046] Bx > (640 - 30) or Cx > (640 - 30)

[0047] The left and right images are both 640×640 in size, the width of the overlapping area is 30, Bx represents the x-coordinate of point B, and Cx represents the x-coordinate of point C. When Cx>(640-30), it means that the entire bounding box ABCD is within the overlapping area. When Bx>(640-30) and Cx<(640-30), it means that part of the bounding box ABCD is within the overlapping area. When Bx<(640-30), it means that the bounding box ABCD is not within the overlapping area.

[0048] Similarly, determine whether the right image bounding box is within the same overlapping region:

[0049] Ex<30 or Hx<30

[0050] Where Ex represents the x-coordinate of point E, and Hx represents the x-coordinate of point H.

[0051] If bounding boxes ABCD and EFGH satisfy the overlapping region condition, it can be determined that the aforementioned road traffic target is within the overlapping region. Then, it is further determined whether the two bounding boxes represent the same target. This invention designs the following calculation method to obtain the metric Object_one for determining whether they represent the same target:

[0052] Object_one=Area_and / Area_all

[0053] Where Area_all is the sum of the areas of bounding boxes ABCD and EFGH:

[0054] Area_all=((Bx–Ax)×(Cy–By))+((Fx–Ex)×(Gy–Ey))

[0055] Where Area_and is the area of ​​the overlapping part of the two rectangles, Ax represents the x-coordinate of point A, Bx represents the x-coordinate of point B, By represents the y-coordinate of point C, Fx represents the x-coordinate of point F, Ex represents the x-coordinate of point E, Gy represents the y-coordinate of point G, and Ey represents the y-coordinate of point E.

[0056] The calculation method is as follows:

[0057] Area_and=(min(Bx,Fx)–max(Ax,Ex))×(min(Dy,Gy)–max(Ay,Ey))

[0058] Area_and = max(0, Area_and)

[0059] The final result of Object_one is:

[0060] Object_one=Area_and / Area_all

[0061] The metric Object_one calculated using the above method will be a floating-point number ranging from 0 to 1. We set the threshold T = 0.5.

[0062] If the two bounding boxes in the overlapping region of two image patches satisfy:

[0063] Object_one>T

[0064] We then consider this a target, merge its bounding box, and output it as the target detection result of the neural network.

[0065] The solution of this invention utilizes 5100 real-time road traffic images captured by cameras, including real street scenes, vehicles, pedestrians, and bicycles, motorcycles, and tricycles. We employ the latest YOLOv5 neural network model released in 2020 for image target recognition.

[0066] Target detection rate (in the image set: the total number of detected targets divided by the total number of all labeled targets):

[0067] The original image was reduced to 640×640 and input into the neural network, with a detection rate of 86.6%.

[0068] After data processing according to this invention, the data is input into a neural network, achieving a detection rate of 96.5%.

[0069] Therefore, by employing the data processing method of this invention, the target detection rate for autonomous driving is significantly increased from 86.6% to 96.5%. Furthermore, some small, distant targets that are not detected by conventional methods can be detected by this invention.

[0070] This application also provides an image processing apparatus. It should be noted that the image processing apparatus of this application can be used to execute the image processing method provided in this application. The image processing apparatus provided in this application will be described below.

[0071] Figure 3 This is a schematic diagram of an image processing apparatus according to an embodiment of this application. Figure 3 As shown, the device includes:

[0072] The acquisition unit 10 is used to acquire the image to be processed, wherein the resolution of the image is a first resolution;

[0073] The segmentation unit 20 is used to segment the image into multiple sub-images, each of the sub-images having a second resolution, which is smaller than the first resolution.

[0074] The processing unit 30 is used to input multiple of the above-mentioned sub-images into the neural network model for parallel processing, and to identify road traffic targets in the above-mentioned sub-images.

[0075] In the above scheme, the acquisition unit acquires the image to be processed, the segmentation unit segments the image to be processed into multiple sub-images, and the operation unit simultaneously inputs multiple sub-images into the neural network model for parallel operation. Although the resolution of the sub-images is smaller than that of the image to be processed, all sub-images include all the features of the image to be processed. The neural network model performs parallel operation on multiple sub-images, ensuring both resolution and real-time image processing.

[0076] In one embodiment of this application, the device further includes a discarding unit. This discarding unit is used to discard a portion of the sub-images after the image is segmented into multiple sub-images, and before inputting the multiple sub-images into a neural network model for parallel computation to identify road traffic targets in the sub-images. The discarded sub-images do not include the road traffic targets. Generally, to ensure a long field of view, cameras are typically mounted on vehicles at near-horizontal shooting angles. The upper half of the captured image usually contains a large area of ​​sky. The actual road traffic targets to be detected are primarily distributed in the lower half of the image. Therefore, the image data equivalent to the upper half wastes valuable resources of the onboard transmission and computing equipment. By discarding sub-images that do not include road traffic targets, valuable resources of the onboard transmission and computing equipment are saved. Furthermore, discarding a portion of the sub-images increases the speed of image processing and indirectly reduces the resolution of the image to be processed. In other words, the solution of this invention maintains the resolution of the image input to the neural network computation while discarding the resource waste caused by computation in invalid areas, and the real-time performance is not affected by parallel computation. This significantly improves the accuracy of neural network target detection.

[0077] In one embodiment of this application, to avoid losing small targets near the boundaries of sub-images and to ensure that the small targets have a relatively complete shape in at least one sub-image, the aforementioned apparatus further includes an overlapping unit. The overlapping unit is used to overlap portions of any two adjacent sub-images to form an overlapping region. Specifically, the adjacent boundaries of any two adjacent sub-images can overlap by a width of 30 pixels. With existing solutions, both the width and height of the image are reduced. For some small road traffic targets, the feature loss is severe due to the image reduction, which is detrimental to the correct detection by image recognition algorithms. This solution, however, only segments the original image without losing the features of small road traffic targets.

[0078] In one embodiment of this application, the apparatus further includes a determining unit and a processing unit. The determining unit is used to determine whether the road traffic target is within the overlapping area. The processing unit is used to perform overlap processing on the road traffic target if it is within the overlapping area. That is, when it is determined that the road traffic target is within the overlapping area, overlap processing is required to achieve accurate identification of the road traffic target.

[0079] In one embodiment of this application, such as Figure 2As shown, the processing unit includes a definition module, a first acquisition module, a second acquisition module, and a determination module. The definition module defines two adjacent sub-images as a left image and a right image, respectively. The first acquisition module acquires a first bounding box, which is the bounding box of the portion of the road traffic target in the left image. The second acquisition module acquires a second bounding box, which is the bounding box of the portion of the road traffic target in the right image. The determination module determines whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target, based at least on the sum of their areas and the overlapping area. The sum of their areas is the sum of the areas of the first bounding box and the second bounding box, and the overlapping area is the area of ​​the overlapping portion of the first and second bounding boxes. Since there may be multiple road traffic targets in the overlapping area, to achieve accurate identification of the road traffic target, it is necessary to determine whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target. The preferred way to determine whether a road traffic target in the left image and a road traffic target in the right image are the same road traffic target within the overlapping area of ​​two adjacent sub-images is by using the area and the overlapping area.

[0080] In one embodiment of this application, the determining module is further configured to determine that the road traffic target in the left image and the road traffic target in the right image are the same road traffic target if the area ratio is greater than or equal to a predetermined value, wherein the area ratio is the ratio of the overlapping area to the sum of the areas; and to determine that the road traffic target in the left image and the road traffic target in the right image are not the same road traffic target if the area ratio is less than the predetermined value. Specifically, the predetermined value can be set to 0.5; of course, those skilled in the art can set an appropriate predetermined value according to the actual situation.

[0081] In one embodiment of this application, the determining unit is further configured to obtain the coordinates of the four vertices of the first bounding box; obtain the coordinates of the four vertices of the second bounding box; and determine whether the road traffic target is within the overlapping area based on the coordinates of the four vertices of the first bounding box, the coordinates of the four vertices of the second bounding box, the size of the sub-image, and the size of the overlapping area.

[0082] The image processing device includes a processor and a memory. The acquisition unit, segmentation unit, and arithmetic unit are all stored in the memory as program units, and the processor executes the program units stored in the memory to achieve the corresponding functions.

[0083] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and their parameters can be adjusted to ensure both resolution and real-time image processing.

[0084] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0085] This invention provides a computer-readable storage medium including a stored program, wherein the program, when running, controls the device where the computer-readable storage medium is located to execute the image processing method.

[0086] This invention provides a processor for running a program, wherein the program executes a method for processing the image during runtime.

[0087] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0088] Step S101: Obtain the image to be processed, wherein the resolution of the image is the first resolution;

[0089] Step S102: Divide the above image into multiple sub-images, and the resolution of each sub-image is a second resolution, which is smaller than the first resolution.

[0090] Step S103: Input multiple of the above sub-images into the neural network model for parallel processing to identify road traffic targets in the above sub-images.

[0091] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0092] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0093] Step S101: Obtain the image to be processed, wherein the resolution of the image is the first resolution;

[0094] Step S102: Divide the above image into multiple sub-images, and the resolution of each sub-image is a second resolution, which is smaller than the first resolution.

[0095] Step S103: Input multiple of the above sub-images into the neural network model for parallel processing to identify road traffic targets in the above sub-images.

[0096] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0098] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0100] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0101] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0102] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0103] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0104] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0105] 1) The image processing method of this application first obtains the image to be processed, then divides the image to be processed into multiple sub-images, and then inputs the multiple sub-images into the neural network model for parallel operation. Although the resolution of the sub-images is smaller than the resolution of the image to be processed, all the sub-images include all the features of the image to be processed. The neural network model performs parallel operation on multiple sub-images, which ensures the resolution and the real-time performance of image processing.

[0106] 2) The image processing apparatus of this application includes an acquisition unit that acquires an image to be processed, a segmentation unit that segments the image to be processed into multiple sub-images, and a computation unit that simultaneously inputs the multiple sub-images into a neural network model for parallel computation. Although the resolution of the sub-images is lower than that of the image to be processed, all the sub-images include all the features of the image to be processed. The neural network model performs parallel computation on the multiple sub-images, ensuring both resolution and real-time image processing.

[0107] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, include: Acquire the image to be processed, wherein the resolution of the image is a first resolution; The image is divided into multiple sub-images, each sub-image having a second resolution, which is smaller than the first resolution; Multiple sub-images are input into a neural network model for parallel processing to identify road traffic targets in the sub-images; The method further includes: Overlapping portions of any two adjacent sub-images forms an overlapping region, wherein the overlapping region is determined based on a preset pixel width; In response to the road traffic target being in the overlapping area, overlapping processing of the road traffic target is performed, including: The two adjacent sub-images are defined as the left image and the right image, respectively; Obtain a first bounding box, which is the bounding box of the portion of the road traffic target in the left image; Obtain a second bounding box, which is the bounding box of the portion of the road traffic target in the right image; The area ratio is determined based on the ratio of the sum of areas and the overlapping area. The road traffic target in the left image and the road traffic target in the right image are determined based on the area ratio and a predetermined value. The sum of areas is the sum of the area of ​​the first bounding box and the area of ​​the second bounding box, and the overlapping area is the area of ​​the overlapping part of the first bounding box and the second bounding box. In response to the fact that the road traffic target in the left image and the road traffic target in the right image are the same road traffic target, the first bounding box and the second bounding box are merged.

2. The processing method according to claim 1, characterized in that, After segmenting the image into multiple sub-images, and before inputting the multiple sub-images into a neural network model for parallel computation to identify road traffic targets in the sub-images, the method further includes: A portion of the sub-images are discarded, and the discarded sub-images do not include the road traffic target.

3. The processing method according to claim 1, characterized in that, The method further includes: Determine whether the road traffic target is within the overlapping area.

4. The processing method according to claim 1, characterized in that, The area ratio is determined based on the ratio of the sum of areas and the overlapping area. The area ratio, combined with a predetermined value, determines whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target. The sum of areas is the sum of the areas of the first bounding box and the second bounding box, and the overlapping area is the area of ​​the overlapping portion of the first bounding box and the second bounding box, including: If the area ratio is greater than or equal to the predetermined value, it is determined that the road traffic target in the left image and the road traffic target in the right image are the same road traffic target; If the area ratio is less than the predetermined value, it is determined that the road traffic target in the left image and the road traffic target in the right image are not the same road traffic target.

5. The processing method according to claim 3, characterized in that, Determining whether the road traffic target is within the overlapping area includes: Obtain the coordinates of the four vertices of the first bounding box; Obtain the coordinates of the four vertices of the second bounding box; Based on the coordinates of the four vertices of the first bounding box, the coordinates of the four vertices of the second bounding box, the size of the sub-image, and the size of the overlapping region, it is determined whether the road traffic target is within the overlapping region.

6. An image processing apparatus, characterized in that, include: An acquisition unit is used to acquire an image to be processed, wherein the resolution of the image is a first resolution; A segmentation unit is used to segment the image into multiple sub-images, each sub-image having a second resolution, the second resolution being smaller than the first resolution; The computing unit is used to input multiple sub-images into a neural network model for parallel computation to identify road traffic targets in the sub-images; The device is further configured to overlap portions of any two adjacent sub-images to form an overlapping region, wherein the overlapping region is determined based on a preset pixel width; in response to the road traffic target being in the overlapping region, to perform overlapping processing on the road traffic target, including: defining two adjacent sub-images as a left image and a right image; obtaining a first bounding box, the first bounding box being the bounding box of the portion of the road traffic target in the left image; obtaining a second bounding box, the second bounding box being the bounding box of the portion of the road traffic target in the right image; determining an area ratio based on the ratio of the sum of areas and the overlapping area, and determining whether the road traffic target in the left image and the road traffic target in the right image are the same road traffic target based on the area ratio and a predetermined value, wherein the sum of areas is the sum of the area of ​​the first bounding box and the area of ​​the second bounding box, and the overlapping area is the area of ​​the overlapping portion of the first bounding box and the second bounding box; in response to the road traffic target in the left image and the road traffic target in the right image being the same road traffic target, merging the first bounding box and the second bounding box.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the image processing method according to any one of claims 1 to 5.

8. A processor, characterized in that, The processor is used to run a program, wherein the program executes the image processing method according to any one of claims 1 to 5.

Images