Pixel row level parallelized image segmentation method, system and readable storage medium

Abstract

The present application relates to the technical field of software engineering, and particularly relates to a pixel row level parallel image segmentation method, system and readable storage medium, the method comprising: dividing a high-resolution original image into multiple segmentation images along the horizontal and vertical directions, and obtaining the position of the segmented region; according to the position of the segmented region, locating the first address of the non-continuous storage of the row image data of the segmented region in the original image corresponding to each row image data of each segmentation image as the starting read pointer for preprocessing and copying of the row image data; according to the position of the segmented region, calculating the starting write pointer of each row image data in each segmentation image; in parallel processing hardware, using the characteristics of parallel processing of multiple processing cores to preprocess each row image data after reading from the corresponding starting read pointer, and copy to the starting write pointer. Thus, in the scene of ultra-high resolution, the effect of reducing the image segmentation and preprocessing delay and finally improving the inference frame rate is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of software engineering, and particularly relates to a pixel row level parallel image segmentation method, a system and a readable storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, more and more industrial industries begin to widely apply artificial intelligence technology, so as to reduce the dependence on artificial labor, improve production efficiency, improve product quality and reduce artificial errors. Machine vision based on artificial intelligence is a relatively common application field. This field consists of multiple links, which are image acquisition, image preprocessing, image reasoning, reasoning result judgment, action feedback, etc. According to the public research data, the average frame rate limit of human eyes can distinguish objects is about 60 frames, and the resolution of object motion recognition is 6000x4000. If converted into an image of about 4GB of image data per second in RGB format. Therefore, if the function of the human eye needs to be replaced, the intelligent recognition ability of 4K or 8K and above high-resolution images needs to be performed under the premise of low delay, so the preprocessing algorithm of super high resolution image is essential. In the image preprocessing calculation task, the super high resolution image segmentation and normalization processing belong to the calculation intensive processing, which cannot be simply solved by improving the frequency of general CPU, but needs to rely on the calculation characteristics of certain parallel computing hardware and clever parallel algorithm design to complete.

[0003] Image preprocessing includes image normalization, channel conversion, image segmentation, median filtering, edge detection, distortion correction, stretching and shrinking, image sharpening and other operations. Among them, normalization, channel conversion and image segmentation are often used in industrial machine vision applications. Since these operations need to calculate and copy memory for each pixel point, if the artificial intelligence application has to face super high resolution images, a large number of calculation tasks will be encountered. Therefore, it is an effective path to shorten the calculation delay and improve the throughput to divide a large number of calculation tasks into a large number of independent calculation simple subtasks and process them with hardware suitable for parallel processing.

[0004] Image segmentation is an effective method to solve the problem of insufficient memory and computational redundancy when processing high-resolution images. It is an application of the finite element method in the field of image processing. The original concept of the finite element method comes from solving problems in elasticity mechanics using structural mechanics methods. The original finite element method is mainly used for engineering calculations. With the continuous improvement of computer performance and the further demand for image realism, the finite element method has been gradually applied to the field of graphics. Image segmentation is based on the principle that the farther the Euclidean distance between pixel points, the lower the relevance of the detection instance represented by the pixel. High-resolution images are segmented into multiple relatively small images for reasoning, and the reasoning results are aggregated according to certain rules to obtain the reasoning results of the original high-resolution image.

[0005] Figure 1 Figure 1 is a schematic diagram of the image segmentation process in the prior art, Figure 1 A is the storage mode of the original high-resolution image before segmentation in the memory. The image data is stored continuously, and the arrangement format is HWC, i.e., the first dimension is the color three channels, the second dimension is the width direction, and the third dimension is the height direction. The picture display software can parse this continuous data block into a three-color rectangular image with a height of H and a width of W. Figure 1 B represents that the high-resolution image is first divided into multiple small resolution images at equal intervals before segmentation, and finally stored in Figure 1 D mode. Figure 1 C represents that the data of each segmented image region in the high-resolution image is moved to the corresponding region in Figure 1 D in the order indicated by the arrow. Since the final merging needs to reverse the corresponding position of the segmented image in the high-resolution image from the order of the segmented image arrangement, the order of each segmented image in Figure 1 D needs to be fixed. In addition, whether it is a CPU-based inference framework or a GPU-based inference framework, the picture data for inference needs to be arranged continuously in sequence when batch reasoning is performed, so Figure 1 The storage space of each segmented image in Figure 1 D is connected at the head and tail, and if the data block of D is parsed by the picture display software, a picture with a width of W and a height of the batch number of H can be obtained.

[0006] At present, the image segmentation of high-resolution images is mainly based on CPU, and the steps are as follows:

[0007] 1. Use the slicing operation of numpy based on CPU to cut the high-resolution image into segmented images;

[0008] 2. Use numpy to perform the following operations on each segmented image:

[0009] a) integer to floating point;

[0010] b) divide by 255.0;

[0011] c) subtract the average value of the corresponding BGR channel respectively;

[0012] d) divide by the standard deviation of the corresponding BGR channel respectively;

[0013] e) interchange the values of R and B channels to complete the conversion from BGR to RGB;

[0014] 3. Use the concatenate method of numpy to connect multiple segmentation maps into a batch of data and send it to the network inference.

[0015] Where numpy represents a mathematical calculation library of python language. Since the pixel amount of super high resolution image is huge, if the super high resolution image is segmented and preprocessed using the traditional method, a huge time delay will be generated, and the throughput of image processing and recognition will be greatly reduced, resulting in that even if the accuracy and inference speed of the model meet the requirements, the practical application value is lost due to the slow preprocessing speed. Therefore, how to decompose these huge amount of calculation tasks into a large number of independent and simple sub-tasks is the premise of realizing parallel calculation and high-speed preprocessing. SUMMARY

[0016] The purpose of the present application is to provide a pixel row level parallel image segmentation method, system and readable storage medium, which at least solves the problem that the existing image segmentation method will generate a huge time delay when segmenting and preprocessing a super high resolution image, and greatly reduces the throughput of image processing and recognition.

[0017] To achieve the above purpose, the present application provides a pixel row level parallel image segmentation method, comprising:

[0018] The high-resolution original image is equally divided into multiple segmentation maps along the horizontal and vertical directions, and the positions of the segmented regions are obtained;

[0019] According to the positions of the segmented regions, the starting read pointer of each row image data of each segmentation map is located as the starting read pointer of the pre-processing and copying of the non-continuous storage row image data of the segmented region in the original image corresponding to the row image data.

[0020] According to the positions of the segmented regions, the starting write pointer of each row image data in each segmentation map is calculated, so that after the row image data is pre-processed and copied, the adjacent two rows of data in the same segmentation map are continuously stored in the memory space, and the adjacent two segmentation maps vertically segmented are also continuously stored in the memory space.

[0021] In parallel processing hardware, the known respective line image data is pre-processed after being read from the corresponding start read pointer using the parallel processing characteristics of the multi-processing core, and copied to the start write pointer.

[0022] Optionally, the method for locating each line image data of each split image according to the location of the split region comprises:

[0023] locating the row position of the split image planned to be processed by the processing core in the longitudinal equal-division grid of the original image;

[0024] locating the column position of the split image planned to be processed by the processing core in the transverse equal-division grid of the original image;

[0025] locating the row position of the line image data planned to be processed by the processing core in the split image.

[0026] Optionally, the method for locating each line image data of each split image according to the location of the split region specifically comprises:

[0027] when the location of the split image to which the line image data planned to be processed by the processing core belongs is the nth row planned to be split in the original image, offsetting the read pointer by n*j*(W*3) values, wherein j represents the number of rows needed to be offset in the longitudinal direction for each split line split image, and (W*3) represents the number of pixel data in the width of the original image of three channels;

[0028] when the location of the split image to which the line image data planned to be processed by the processing core belongs is the mth column planned to be split in the original image, offsetting the read pointer by m*i*3 values, wherein i represents the number of columns needed to be offset in the transverse direction for each split split image;

[0029] when the processing core plans to copy the rth row of the split image, offsetting the read pointer by r*W*3 values.

[0030] Optionally, the method for calculating the start write pointer of each line image data in each split image according to the location of the split region specifically comprises:

[0031] when the location of the split image to which the line image data planned to be processed by the processing core belongs is the nth row planned to be split in the original image, offsetting the read pointer by n*p*(h*w*3) values, wherein p represents the number of rows planned to be split in the longitudinal direction of the original image, and (h*w*3) represents the image size of the split image;

[0032] when the location of the split image to which the line image data planned to be processed by the processing core belongs is the mth column planned to be split in the original image, offsetting the read pointer by m*(h*w*3) values.

[0033] The processing core is known to plan to copy the rth row of the corresponding partitioned image, so as to offset the read pointer by r*(w*3) values, wherein (w*3) represents the pixel size of the row image data.

[0034] Optionally, the image partitioning preprocessing step specifically comprises the following steps.

[0035] Three Uint8 type data are read each time, and integer-to-floating point conversion, normalization and channel conversion processing are performed.

[0036] Optionally, the parallel computing framework is a CUDA computing framework of a GPU, and under the CUDA computing framework, each grid comprises a plurality of processing core blocks, and each processing core block comprises a plurality of processing cores, and the processing core is the most basic execution unit.

[0037] Based on the same technical concept, the application further provides a pixel row level parallel image partitioning system, comprising:

[0038] A splitting module is configured to equally divide a high-resolution original image into a plurality of partitioned images along the horizontal and vertical directions, and obtain the position of the partitioned region;

[0039] A positioning module is configured to position, according to the position of the partitioned region, the starting read pointer of each row image data of each partitioned image as the starting read pointer of the row image data of the partitioned region in the original image for preprocessing and copying of the row image data;

[0040] A calculation module is configured to calculate, according to the position of the partitioned region, the starting write pointer of each row image data in each partitioned image, so that after the row image data is preprocessed and copied, the adjacent two rows of data in the same partitioned image are continuously stored in the memory space, and the adjacent two partitioned images vertically partitioned are also continuously stored in the memory space.

[0041] A preprocessing module is configured to utilize the multi-processing core parallel processing characteristics in the parallel processing hardware to preprocess each row image data after the row image data is read from the corresponding starting read pointer, and copy the row image data to the starting write pointer.

[0042] Based on the same technical concept, the application further provides a readable storage medium, which stores a computer program, and the computer program can implement the pixel row level parallel image partitioning method when executed.

[0043] In the pixel row level parallelization image segmentation method, system and readable storage medium provided by the present application, the high-resolution original image is equally divided into a plurality of small-resolution segmentation images along the horizontal and vertical directions, and each row of pixel data of each segmentation image to be read in the original image is stored continuously in the memory address, and the storage address of each row of pixel data of each segmented image is also a continuous address. When planning the parallelization granularity of the segmentation task, the present application tries to make the most of the characteristics of the multiple processing cores of the parallel processing hardware to process simultaneously, and uses the multiple processing cores of the parallel processing hardware to cleverly address and locate the source address and target address of each row of data of the segmented image, while completing the normalization, channel conversion, copying and other image segmentation and preprocessing work, so as to achieve the purpose of dividing a large image into small images and completing image preprocessing in a very short time, and achieve the effect of reducing the image segmentation and preprocessing delay and finally improving the inference frame rate in the scene of ultra-high resolution. BRIEF DESCRIPTION OF DRAWINGS

[0044] Those skilled in the art will understand that the provided drawings are for the purpose of better understanding the present application and do not constitute any limitation on the scope of the present application. Among them:

[0045] Figure 1 It is a schematic diagram of the image segmentation process in the prior art;

[0046] Figure 2 It is a flowchart of the pixel row level parallelization image segmentation method provided by an embodiment of the present application;

[0047] Figure 3 It is a storage schematic diagram of a high-resolution image provided by an embodiment of the present application;

[0048] Figure 4 It is a schematic diagram of CUDA computing framework segmentation provided by an embodiment of the present application;

[0049] Figure 5A It is a schematic diagram of positioning the starting read pointer of each processing core provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] As described in the background, how to split these huge number of computing tasks into a large number of independent computing simple subtasks is the premise of parallel computing and then high-speed preprocessing. As the condition of parallel subtask splitting, the mutual independence and time economy between each subtask are problems that must be considered. According to the arrangement mode of the result data segmented by the traditional method, it can be found that each segmented image storage location is arranged in reading order and stored in a head-to-tail manner, so that the segmented tasks of each segmented image are directly independent of each other, meeting the independence requirement of parallel task segmentation. In addition, when the total number of segmented images that need to be segmented is sufficient, the time folding benefit brought by parallelization is much greater than the time loss caused by data replication loss, system overhead, and subsystem frequency difference at the start of each subtask, so the improvement of picture-level parallelization meets the time economy.

[0051] Although the picture-level parallelization method can achieve a certain degree of parallelization of the segmentation task, the parallelization degree is limited by the total number of segmented images, and when the parallelization capability of the parallel processing hardware far exceeds the total number of segmented images, the performance of the parallelization hardware cannot be fully utilized, resulting in limited effect of preprocessing acceleration. In view of the problem of the picture-level parallelization method, the present application proposes a pixel row-level finer granularity parallelization method. Since an original image needs to be divided into multiple segmented images, and each row of pixel data of each segmented image needs to be read in the original image and stored continuously in the memory address, and the storage address of each row of pixel data of each segmented image is also a continuous address, each row of pixels of each segmented image can be segmented as a subtask. Since each row of image is also independent in the result data, it also meets the independence requirement of subtask splitting. In addition, since the number of subtasks = the number of segmented images x the number of segmented image rows, it will generally exceed the parallel capability of the parallel processing device, so the hardware resources of the parallel processing device can be fully utilized.

[0052] To make the purpose, advantages and characteristics of the present application clearer, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the drawings are very simplified and all use non-precise proportions, only to facilitate and clarify the purpose of assisting the description of the embodiments of the present application. In order to make the purpose, characteristics and advantages of the present application more obvious and easy to understand, please refer to the drawings. It should be noted that the structure, proportion, size, etc. shown in the drawings attached to the present specification are only used to cooperate with the content disclosed in the specification, so that people skilled in the art can understand and read, and are not used to limit the conditions of the implementation of the present application. Any modification of structure, change of proportion relationship or adjustment of size, as long as it is the same or similar to the effect and purpose that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application.

[0053] As used in the present application, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. As used in the present application, the term "or" is generally employed in its sense of "and / or" unless the content clearly dictates otherwise. As used in the present application, the term "several" is generally employed in its sense of "at least one" unless the content clearly dictates otherwise. As used in the present application, the term "at least two" is generally employed in its sense of "two or more" unless the content clearly dictates otherwise. In addition, the terms "first," "second," "third," etc. are used merely to describe a particular feature and do not imply or suggest a relative importance or an implicit indication of the number of the indicated technical features. Thus, features qualified with "first," "second," "third," etc. can explicitly or implicitly include one or at least two of the features.

[0054] In the description of the present application, unless otherwise clearly and definitely specified and limited, the terms "mounting", "connection", "connecting", "fixing" should be understood in a broad sense, for example, can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be direct connection, or indirect connection through intermediate medium; can be internal communication of two elements or interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0055] Please refer to Figure 2 , Figure 2 The flow chart of the pixel row level parallel image segmentation method provided by an embodiment of the present application. The embodiment of the present application provides a pixel row level parallel image segmentation method, which comprises the following steps:

[0056] S1, equally dividing a high-resolution original image along the horizontal direction and the vertical direction into a plurality of segmentation images, and obtaining the positions of the segmented areas;

[0057] S2, according to the positions of the segmented areas, locating the start read pointer of each row image data of each segmentation image as the start address of the non-continuous storage of the row image data of the segmented area in the original image for pre-processing and copying of the row image data;

[0058] S3, according to the positions of the segmented areas, calculating the start write pointer of each row image data in each segmentation image, so that after the row image data is pre-processed and copied, the adjacent two row data in the same segmentation image are continuously stored in the memory space, and the adjacent two segmentation images vertically segmented are also continuously stored in the memory space;

[0059] S4, in parallel processing hardware, using the characteristics of parallel processing of multiple processing cores, pre-processing each row image data from the corresponding starting read pointer after reading, and copying to the starting write pointer.

[0060] Due to the characteristics that each row of pixel data of each split image needs to be read in the original image is stored continuously in the memory address and the storage address of each row of pixel data of each split image is also a continuous address, the application can maximize the simultaneous processing characteristics of multiple processing cores of parallel processing hardware when planning the parallelization granularity of the split task, and can use the multiple processing cores of parallel processing hardware to cleverly address and locate the source address and target address of each row of image data of each split image, while completing the normalization, channel conversion, copying and other image splitting and preprocessing work, so as to achieve the purpose of splitting a large image into small images and completing image preprocessing in a very short time, and to achieve the effect of reducing the time delay of image splitting and preprocessing and finally improving the inference frame rate in the scene of ultra-high resolution.

[0061] Firstly, step S1 is performed, the high-resolution original image is equally divided into multiple split images along the horizontal and vertical directions, and the positions of the split regions are obtained, each split image has a plurality of row image data.

[0062] Please combine Figure 3 In the embodiment, the original image is a high-resolution or ultra-high-resolution image, the height of the original image is represented by H, the width is represented by W, and the image size is represented by H*W*3. The arrangement mode of the original image is HWC, wherein H represents the first dimension as height, W represents the second dimension as width, and C represents the third dimension as color channel. The storage order in the memory is BGRBGR……BGR. The height of the split image output after cutting is represented by h, the width is represented by w, and the image size is represented by h*w*3. The image arrangement mode is also HWC.

[0063] In the embodiment, it is assumed that the high-resolution original image needs to be equally divided in the vertical direction q and the horizontal direction p to cut the image into q*p small split images, each split image has h rows of image data, and q*p*h processing cores are needed for copying. Since the original image is 3-channel data, each processing core needs to copy w*3 pixel data.

[0064] Then, step S2 is performed, according to the positions of the split regions, the starting read pointer of each row of image data of each split image is located as the starting read pointer for pre-processing and copying of the first address of the non-continuous storage of the row image data of the split region in the original image.

[0065] In the embodiment, the method for locating each row of image data of each split image according to the positions of the split regions comprises:

[0066] locating a column position of the split image to be processed by the processing core in the longitudinal equal-division grid of the original image, i.e. which column of the split image to be processed by the processing core in the longitudinal equal-division grid of the original image;

[0067] locating a column position of the split image to be processed by the processing core in the longitudinal equal-division grid of the original image, i.e. which column of the split image to be processed by the processing core in the longitudinal equal-division grid of the original image;

[0068] locating a column position of the split image to be processed by the processing core in the longitudinal equal-division grid of the original image, i.e. which column of the split image to be processed by the processing core in the longitudinal equal-division grid of the original image;

[0069] Further, the method of locating each row image data of each split image according to the position of the split region specifically comprises:

[0070] when the position of the split image to which the row image data to be processed by the processing core belongs is the nth row of the original image to be split, offsetting the read pointer by n*j*(W*3) values, where j represents the number of rows to be offset in the longitudinal direction for each split row of the split image, and (W*3) represents the number of pixel data in the width of the original image of three channels, as shown in Figure 5A ;

[0071] when the position of the split image to which the row image data to be processed by the processing core belongs is the mth column of the original image to be split, offsetting the read pointer by m*i*3 values, where i represents the number of columns to be offset in the transverse direction for each split image, and at this time the read pointer points to the first pixel of the split image region, as shown in Figure 5B ;

[0072] when the processing core plans to copy the rth row of the split image to which the processing core belongs, offsetting the read pointer by r*W*3 values, and at this time the read pointer points to the first pixel of the row of the split image, as shown in Figure 5C .

[0073] Then, step S3 is performed, and the starting write pointer of each row image data in each split image is calculated according to the position of the split region, so that after the row image data is preprocessed and copied, the adjacent two rows of data in the same split image are stored continuously in the memory space, and the adjacent two split images in the longitudinal direction are also stored continuously in the memory space.

[0074] Further, the method of calculating the starting write pointer of each row image data in each split image according to the position of the split region specifically comprises:

[0075] It is known that the position of the split image to which the row image data processed by the processing core belongs is the nth row of the split image planned by the original image, so the read pointer is offset by n*p*(h*w*3) values, wherein p represents the number of rows in which the original image is split in the vertical direction, and (h*w*3) represents the image size of the split image.

[0076] It is known that the position of the split image to which the row image data processed by the processing core belongs is the mth column of the split image planned by the original image, so the read pointer is offset by m*(h*w*3) values.

[0077] It is known that the processing core plans to copy the rth row of the split image, so the read pointer is offset by r*(w*3) values, wherein (w*3) represents the pixel size of the row image data.

[0078] In the embodiment, W represents the width of the original image, H represents the height of the original image, w represents the width of the split image, h represents the height of the split image, p represents the number of columns of split images planned to be split from the original image, q represents the number of rows of split images planned to be split from the original image, m represents the column of the split image to which a certain pixel row to be processed by a certain processing core belongs, and n represents the row of the split image to which a certain pixel row to be processed by a certain processing core belongs.

[0079] It should be understood that the calculation of the start write pointer is similar to the positioning sequence of the start read pointer, and only the pointer offset is different. In addition, the application does not limit the sequence in which the start read pointer and the start write pointer of each processing core are positioned.

[0080] Finally, step S4 is performed, and in the parallel processing hardware, the known row image data is preprocessed after being read from the corresponding start read pointer and copied to the start write pointer by using the parallel processing characteristics of the multiple processing cores.

[0081] In the embodiment, the start read pointer and the start write pointer of each processing core are positioned in the following sequence: Figure 4, the parallel processing hardware is preferably the CUDA computing framework of NVIDIA GPU, under which each processing grid contains multiple processing blocks, and each processing block contains multiple processing cores (threads), which are the most basic execution units. Since NVIDIA GPU is good at concurrent processing of multiple tasks, a logically simple but computationally large and less dependent task can be split into thousands or tens of thousands of concurrent tasks for calculation to improve computational efficiency and shorten calculation time, so based on the characteristics of NVIDIA GPU, the timing of the four processes of original image segmentation, integer to floating point conversion, normalization, segmentation map inference and result aggregation can be designed as follows: first, use parallel processing to complete image segmentation, integer to floating point conversion and normalization, then divide each segmented image data that has completed segmentation, integer to floating point conversion and normalization into reasonable batches and send it to the deep neural network for inference, and then collect and aggregate the results.

[0082] In which, the idea of using parallel processing for image segmentation is as follows: since an original image needs to be segmented into multiple segmented images, and each row of pixel data of each segmented image needs to be read in the original image and stored continuously in the memory address, and the storage address of each row of pixel data of each segmented image is also a continuous address, each processing core of the NVIDIA GPU can be responsible for copying one row of image data of one segmented image, and at the same time, the Uint8 type data is converted to a floating point number and normalized, thereby improving the parallelism of image segmentation, integer to floating point conversion, normalization and channel conversion, and shortening the processing time.

[0083] Preferably, since 32 processing cores in the NVIDIA GPU are divided into a warp and executed simultaneously, the number of processing cores needs to be divisible by 32 when designing the grid and processing core block.

[0084] In this embodiment, the image segmentation preprocessing step specifically includes:

[0085] Each time three Uint8 type data are read, integer to floating point conversion, normalization and channel conversion processing are performed.

[0086] Using the characteristics of multi-processing core parallel processing, each time three Uint8 type data are read from the corresponding starting read pointer when reading each row of image data, and then integer to floating point conversion, normalization and channel conversion processing are performed and written to the target pointer, and the same operation is performed for w rounds, thereby achieving the operation of cutting a high-resolution image into small images, and also completing the non-complex but large number of preprocessing operations brought by the high-resolution image.

[0087] Since the NVIDIA GPU can execute thousands or tens of thousands of processing cores at the same time, when planning the parallelization granularity of the segmentation task, the characteristics of the multiple processing cores of the parallel processing hardware can be utilized as much as possible to perform parallel addressing, normalization, channel conversion, copying, etc. on each row of image data to achieve the purpose of dividing a large image into small images and completing image preprocessing in an extremely short time, and to achieve the effect of reducing the image segmentation and preprocessing delay and ultimately improving the inference frame rate in the scene of ultra-high resolution.

[0088] The present application designs the following experimental scenarios to compare the performance and effect of the traditional segmentation method:

[0089] 1) 80 128*128 segmentation images are cut from a 1000*400 3-channel original image, and the time consumption of the image from the start of segmentation to the completion of image segmentation is compared.

[0090] 2) The inference time of a 1000*400 3-channel original image is compared, and the frame rate per second from the start of preprocessing to the inference result is obtained, wherein the preprocessing includes image segmentation.

[0091] Experimental conditions:

[0092] 1) CPU: Intel(R) Xeon(R) Gold 5117 CPU @ 2.00GHz, 55 cores

[0093] 2) GPU: NVIDIA Titan-V

[0094] 3) Memory: DDR4 ECC 220G

[0095] 4) Hard disk: SSD 4T

[0096]

[0097] Table 1

[0098] Table 1 is a comparison of the time delay and frame rate of the traditional image segmentation method and the image segmentation method provided by the present application. According to the experimental results, it can be found that the preprocessing time delay of the image segmentation method of the present application is shortened from 85ms to 4.8ms, which is improved by 17.7 times, and the inference frame rate is improved from 12.5 frames to 74.3 frames, which is improved by 5.9 times. Therefore, it can be judged that the image segmentation method of the present application can also achieve the effect of reducing the time delay and improving the frame rate in the scene of ultra-high resolution, and the application value of the inference model is improved.

[0099] Based on the same inventive concept, the embodiments of the present application also provide a pixel row level parallel image segmentation system, comprising:

[0100] a segmentation module configured to divide a high-resolution original image into a plurality of segmented images along a horizontal direction and a vertical direction, and to obtain positions of segmented regions;

[0101] a positioning module configured to position, according to the positions of the segmented regions, a start read pointer of each row image data of each segmented image corresponding to a start address of non-continuous storage of the row image data of the segmented region in the original image, for pre-processing and copying of the row image data;

[0102] a calculation module configured to calculate, according to the positions of the segmented regions, a start write pointer of each row image data in each segmented image, so that, after the row image data is pre-processed and copied, adjacent two rows of data in the same segmented image are stored continuously in a memory space, and adjacent two segmented images vertically segmented are also stored continuously in the memory space;

[0103] a pre-processing module configured to, in parallel processing hardware, pre-process each row image data after reading from a corresponding start read pointer by using a multi-processing core parallel processing feature, and copy to the start write pointer.

[0104] Based on the same inventive concept, the embodiment of the present application further provides a readable storage medium having a computer program stored thereon, and the computer program is executable to implement the pixel row level parallel image segmentation method as above.

[0105] A readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The readable storage medium can be, for example but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched-tape, a holographic storage medium, and any suitable combination of the foregoing. A computer program product, which can include a computer program tangibly embodied in a readable storage medium, can be downloaded to the respective computer / processor from an external computer or external storage device or to an external computer or an external storage device from the respective computer / processor. A network or a data cloud can include a copper transmission cable, a fiber-optic transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or a server. A network adapter of a computer / processor receives computer programs from a network and forwards the computer programs to a readable storage medium of the respective computer / processor. The computer programs can execute entirely on the user's computer / processor, partly on the user's computer / processor, as a stand-alone software package, partly on the user's computer / processor and partly on a remote computer / processor or entirely on the remote computer / processor or server. In the latter scenario, the remote computer / processor can be connected to the user's computer / processor through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0106] The computer program can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the computer program running on the computer, other programmable data processing apparatus, or other device implements the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0107] The computer program can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the computer program running on the computer, other programmable data processing apparatus, or other device implements the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0108] In summary, the embodiments of the present application provide a pixel row level parallel image segmentation method, system and readable storage medium. The high-resolution original image is equally divided into a plurality of small resolution segmentation images along the horizontal and vertical directions. Each row of pixel data of each segmentation image to be read in the original image is stored in a continuous memory address. The storage address of each row of pixel data of each segmented image is also a continuous address. When planning the parallel granularity of the segmentation task, the embodiments of the present application can maximize the characteristics of the multiple processing cores of the parallel processing hardware to simultaneously process. The multiple processing cores of the parallel processing hardware are used to cleverly address and locate the source address and target address of each row of data of each segmentation image. At the same time, the normalization, channel conversion, copying and other image segmentation and preprocessing work are completed. The purpose of dividing a large image into small images and completing image preprocessing in a very short time is achieved. In the scene of ultra-high resolution, the effect of reducing the image segmentation and preprocessing delay and finally improving the inference frame rate is achieved.

[0109] The above description is only the description of the preferred embodiments of the present application, and does not limit the scope of the present application in any way. Any modification or change made by those skilled in the art according to the above disclosure is within the scope of the present application. Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations are within the scope of the present application and equivalent technology thereof, the present application is intended to include these modifications and variations.

Claims

1. A pixel-row-level parallel image segmentation method, characterized in that: include: Divide the high-resolution original image into multiple segmentation maps in the horizontal and vertical directions, and obtain the positions of the segmented areas; According to the position of the segmented area, locating the first address of each row of image data of each segmented image corresponding to the non-continuously stored row of image data of the segmented area in the original image as a starting read pointer for preprocessing and copying the row of image data; Calculating a start write pointer for each row of image data in each segmented image based on the position of the segmented area, so that after the row of image data is pre-processed and copied and saved, two adjacent rows of data in the same segmented image are stored continuously in the memory space, and two adjacent segmented images that are vertically segmented are also stored continuously in the memory space; In the parallel processing hardware, utilizing the parallel processing characteristics of multiple processing cores, pre-processing is performed on each row of known image data after reading it from the corresponding starting read pointer, and copying it to the starting write pointer; The method of calculating the start write pointer of each row of image data in each segmentation map according to the position of the segmented area specifically includes: Knowing that the position of the segmentation map to which the row image data to be processed by the processing core belongs is located at the nth row of the original image to be segmented, the read pointer is offset by n*p*(h*w*3) values, where p represents the number of rows into which the original image is to be segmented in the vertical direction, and (h*w*3) represents the image size of the segmentation map; It is known that the position of the segmentation map to which the row image data to be processed by the processing core belongs is located in the mth column of the original image to be segmented, so that the read pointer is shifted by m*(h*w*3) values; It is known that the processing core plans to copy the rth row of the segmentation map, so that the read pointer is offset by r*(w*3) values, where (w*3) represents the pixel size of the row image data.

2. The pixel row level parallel image segmentation method according to claim 1, characterized in that: The method of locating each row of image data of each segmentation image according to the position of the segmented area includes: Locating the row positions of the segmentation graph processed by the processing core plan in the vertically equally divided grid of the original image; Locating the column positions of the segmentation map processed by the processing core plan in the horizontally equally divided grid of the original image; Locate the row position of the row image data that the processing core plans to process in the segmentation map.

3. The pixel row level parallel image segmentation method according to claim 2, characterized in that: The method of locating each row of image data in each segmentation image according to the position of the segmented area specifically includes: Given that the segmentation map to which the row image data to be processed by the processing core belongs is located at the nth row of the original image to be segmented, the read pointer is offset by n*j*(W*3) values, where j represents the number of rows to be offset vertically for each segmentation row, and (W*3) represents the number of pixel data in the width of the three-channel original image. It is known that the position of the segmentation map to which the row image data to be processed by the processing core belongs is located in the mth column of the planned segmentation of the original image, so that the read pointer is offset by m*i*3 values, where i represents the number of columns to be offset horizontally for each segmentation; It is known that the processing core plans to copy the rth row of the corresponding segmentation map, so that the read pointer is offset by r*W*3 values.

4. The pixel row level parallel image segmentation method according to claim 1, characterized in that: The steps for image segmentation preprocessing specifically include: Each time, three Uint8 data are read and converted to floating point, normalized, and channel converted.

5. The pixel row level parallel image segmentation method according to claim 1, characterized in that: The parallel processing hardware is a CUDA computing framework of a GPU. Under the CUDA computing framework, each grid includes multiple processing core blocks, each processing core block includes multiple processing cores, and the processing core is the most basic execution unit.

6. A pixel-row-level parallel image segmentation system, characterized in that: include: A segmentation module is configured to divide the high-resolution original image into multiple segmentation maps in the horizontal and vertical directions, and obtain the positions of the segmented areas; a positioning module configured to locate, according to the position of the segmented area, a first address of each row of image data of each segmentation graph corresponding to the non-continuously stored row of image data of the segmented area in the original image as a starting read pointer for preprocessing and copying the row of image data; a calculation module configured to calculate a start write pointer for each row of image data in each segmented image according to the position of the segmented area, so that after the row of image data is pre-processed and copied and saved, two adjacent rows of data in the same segmented image are stored continuously in the memory space, and two adjacent segmented images that are vertically segmented are also stored continuously in the memory space; a pre-processing module configured to pre-process each row of known image data after reading it from the corresponding start read pointer and copy it to the start write pointer by utilizing the parallel processing characteristics of multiple processing cores in the parallel processing hardware; The method for the calculation module to calculate the start write pointer of each row of image data in each segmentation map according to the position of the segmented area specifically includes: Knowing that the position of the segmentation map to which the row image data to be processed by the processing core belongs is located at the nth row of the original image to be segmented, the read pointer is offset by n*p*(h*w*3) values, where p represents the number of rows into which the original image is to be segmented in the vertical direction, and (h*w*3) represents the image size of the segmentation map; It is known that the position of the segmentation map to which the row image data to be processed by the processing core belongs is located in the mth column of the original image to be segmented, so that the read pointer is shifted by m*(h*w*3) values; It is known that the processing core plans to copy the rth row of the segmentation map, so that the read pointer is offset by r*(w*3) values, where (w*3) represents the pixel size of the row image data.

7. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the pixel row level parallel image segmentation method according to any one of claims 1 to 5 can be implemented.

Images