An image data processing method, device and storage medium

Abstract

The application provides an image data processing method, device and storage medium. The method comprises: obtaining first image data of N different scales of an image to be processed with light and shadow edges, N being an integer greater than or equal to 2; obtaining the nth second image data after smoothing processing of the nth first image data, n being an integer greater than or equal to 1 and less than or equal to N; performing color mapping on the nth second image data to obtain the nth third image data, wherein the color channel component value of each pixel in the nth second image data and the color component value of the corresponding pixel in the nth third image data have a mapping relationship; and performing data fusion on the N third image data to obtain target image data with the same scale as the first image data of the largest scale, wherein the image represented by the target image data does not have light and shadow edges.

Description

Technical Field

[0001] This application relates to the field of image data processing technology in financial technology (Fintech), and includes, but is not limited to, an image data processing method, device, and storage medium. Background Technology

[0002] With the development of computer computing, more and more technologies are being applied in the financial sector, and the traditional financial industry is gradually transforming into Fintech. However, due to the security and real-time requirements of the financial industry, Fintech also places higher demands on technology. In the field of Fintech, various financial systems, such as bank management systems, need to monitor the area around the bank to obtain video images, and then perform edge detection on objects in the video images.

[0003] One method for edge detection of objects in video images in related technologies involves processing each frame of the video image into grayscale to obtain a grayscale image, and then performing edge detection based on the grayscale image. However, when objects in the image create shadows due to lighting, the significant grayscale difference at the edges of the shadows can lead to the shadow edges being mistakenly identified as the object edges during edge detection. Therefore, the above methods suffer from at least one problem: inaccurate detection. Summary of the Invention

[0004] This application provides an image data processing method, device, and storage medium to solve the problem in related technologies where light and shadow edges are mistakenly identified as the edges of target objects, resulting in inaccurate detection.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an image data processing method, including:

[0007] Obtain first image data at N different scales of the image to be processed that has light and shadow edges, where N is an integer greater than or equal to 2;

[0008] Obtain the nth second image data after smoothing the nth first image data, where n is an integer greater than or equal to 1 and less than or equal to N;

[0009] Color mapping is performed on the nth second image data to obtain the nth third image data, wherein the color channel component values ​​of each pixel in the nth second image data have a mapping relationship with the color component values ​​of the corresponding pixels in the nth third image data;

[0010] Data fusion is performed on N third image data to obtain target image data with the same scale as the first image data with the largest scale, wherein the image represented by the target image data does not have light and shadow edges.

[0011] This application provides a data processing device, including:

[0012] Memory, used to store executable instructions;

[0013] The processor implements the above method when executing executable instructions stored in memory.

[0014] This application provides a storage medium storing executable instructions for inducing a processor to execute the above-described method.

[0015] The embodiments of this application have the following beneficial effects:

[0016] First, first image data at different scales of the image to be processed is acquired. Then, the first image data is smoothed to obtain second image data. Next, color mapping is performed on the second image data to obtain third image data. The color channel component values ​​of each pixel in the second image data are mapped to the corresponding color component values ​​of the pixels in the third image data. That is, the color channel component values ​​of each pixel in the second image data are converted into color component values ​​in other color spaces to obtain the third image data. This preserves the color attributes of objects and light / shadow edges in the image data, allowing for the elimination of light and shadow by using the difference in color attributes between objects and light / shadow edges. Finally, data fusion is performed based on the third image data at different scales to obtain target image data with the same scale as the first image data at the largest scale, and the target image data represents an image without light / shadow edges. Thus, data fusion of third image data at different scales achieves the merging and fusion of multi-scale images, further highlighting the main object regions, eliminating light and shadow interference, and ensuring that the edges of the identified objects no longer contain light / shadow edges. Therefore, edge detection of objects in the image is performed based on the image with eliminated light / shadow edges, improving the accuracy of edge detection. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating image data processing methods in related technologies;

[0018] Figure 2 This is a schematic diagram of an optional architecture of the terminal provided in an embodiment of this application;

[0019] Figure 3 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of the image to be processed provided in an embodiment of this application;

[0021] Figure 5 This is a schematic diagram of images represented by first image data at three different scales provided in the embodiments of this application;

[0022] Figure 6 This is a schematic diagram of images represented by second image data at three different scales provided in the embodiments of this application;

[0023] Figure 7 This is a schematic diagram of the image represented by the target image data provided in the embodiments of this application;

[0024] Figure 8 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0025] Figure 9 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0026] Figure 10 This is a schematic diagram illustrating the process of dimensionality reduction processing of the image to be processed, provided in an embodiment of this application.

[0027] Figure 11 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0028] Figure 12 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0029] Figure 13 This is a schematic diagram illustrating the process of smoothing using a sparse matrix provided in an embodiment of this application;

[0030] Figure 14 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0031] Figure 15 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0032] Figure 16 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0033] Figure 17 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0034] Figure 18 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0035] Figure 19 This is a schematic diagram illustrating the dimensionality expansion processing of the selected third image data provided in an embodiment of this application;

[0036] Figure 20 This is an optional flowchart illustrating the image data processing method provided in an embodiment of this application;

[0037] Figure 21 This is a schematic diagram illustrating the nonmaximum suppression process provided in an embodiment of this application;

[0038] Figure 22 This is a schematic diagram of the edge detection result of an object obtained by the image data processing method provided in the embodiments of this application;

[0039] Figure 23 This is an optional flowchart illustrating the image data processing method provided in the embodiments of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit the application.

[0042] To better understand the image data processing method provided in the embodiments of this application, the image data processing methods in related technologies are first described:

[0043] See Figure 1 , Figure 1This is a structural block diagram of an image data processing method provided by related technologies. First, the image to be processed is obtained and then grayscale processing is performed on the image to be processed to obtain grayscale image data. Second, a two-dimensional Gaussian filter is used to remove noise from the grayscale image data to obtain smoothed grayscale image data. Then, based on the obtained first-order finite difference horizontal axis gradient calculation matrix and vertical axis gradient calculation matrix, the gradient magnitude matrix and gradient direction matrix of the smoothed grayscale image data are calculated. Finally, based on the gradient magnitude matrix and gradient direction matrix, edge detection of objects is performed.

[0044] The above image grayscale processing process is as follows: The RGB values ​​of each pixel in the image to be processed are read sequentially to obtain the image RGB matrix M. RGB , the image RGB matrix M RGB Convert to image grayscale matrix M GRAY The image grayscale conversion formula is as follows:

[0045] Gray (i,j) =0.299×r (i,j) +0.587×g (i,j) +0.114×b (i,j) (Formula 1)

[0046] Where i and j are the coordinate positions in the image matrix, i is an integer greater than or equal to 1 and less than or equal to 1, 1 is the total number of rows in the image matrix, and j is an integer greater than or equal to 1 and less than or equal to J, 1 is the total number of columns in the image matrix; Gray (i,j) Let r be the grayscale value of the pixel at coordinate (i,j). (i,j) g represents the red (r) color channel component value of the pixel at coordinate (i,j). (i,j) Let b be the green (g) color channel component value of the pixel at coordinate (i,j). (i,j) Let (i,j) be the blue (b) color channel component value of the pixel at coordinate (i,j).

[0047] In (Formula 1), the RGB values ​​of each pixel are weighted and summed to obtain the grayscale value of that pixel, and then the grayscale values ​​of all pixels are calculated. (i,j) The grayscale matrix M of the image is composed of GRAY .

[0048] The process of using a two-dimensional Gaussian filter to remove noise from a grayscale image is as follows: First, a filter matrix template is generated, according to the definition of a two-dimensional Gaussian filter: This generates a two-dimensional Gaussian filter matrix template with dimensions (2t+1)×(2t+1). The formulas for calculating each element of the matrix are as follows:

[0049]

[0050] Where i and j are the coordinates in the filter matrix, π is a constant value of pi, sigma is the standard deviation of the Gaussian distribution, and t is the size of the template used to define the two-dimensional Gaussian filter matrix. It should be noted that a larger sigma value results in a more dispersed Gaussian distribution, increasing the influence of distant pixels on the central pixel, and thus a smoother filtering result; conversely, a smaller sigma value reduces the influence of distant pixels on the central pixel.

[0051] According to the Gaussian distribution (μ, sigma) 2 Where u is the mean (also known as the mathematical expectation) of the Gaussian distribution; the area under the curve accounts for 68% of the total area under the curve in the interval (μ-sigma, μ+sigma), 95% in the interval (μ-2sigma, μ+2sigma), and 99.7% in the interval (μ-3sigma, μ+3sigma). Therefore, the length and width of the two-dimensional Gaussian filter matrix template should be within a 3 to 4 times relationship with the value of sigma to be considered reasonable. In related technologies, the length and width of the two-dimensional Gaussian filter matrix template are generally 5×5, so the value of t is 2 and the value of sigma is 1.4; further, substituting t and sigma into (Formula 2) to calculate, we obtain the 5×5 Gaussian filter matrix G. 2×5 Finally, the 5×5 Gaussian filter matrix G is... 5×5 Compared with the original image grayscale matrix M GRAY The convolution operation is performed to smooth the image and remove noise, ultimately yielding the smoothed grayscale image data, i.e., the smoothed grayscale matrix GM. GRAY .

[0052] For example, a 5×5 Gaussian filter matrix G 5×5 It can be:

[0053] G 5×5 =[[0.00111797,0.00400291,0.00860487,0.01110539,0.00860487],

[0054] [0.00400291,0.01433255,0.03080999,0.03976317,0.03080999],

[0055] [0.00860487,0.03080999,0.06623077,0.08547699,0.06623077],

[0056] [0.01110539,0.03976317,0.08547699,0.11031603,0.08547699],

[0057] [0.00860487,0.03080999,0.06623077,0.08547699,0.06623077]].

[0058] The process of calculating the horizontal and vertical gradient matrices of a grayscale image using first-order finite difference, and then obtaining the magnitude and direction matrices, is as follows: First, obtain the horizontal gradient calculation matrix as x_kernel and the vertical gradient calculation matrix as y_kernel. For example, the horizontal gradient calculation matrix x_kernel = [[-1,1],[-1,1]]; the vertical gradient calculation matrix y_kernel = [[1,1],[-1,-1]]. Second, use GM... GRAY Perform a convolution operation with the horizontal gradient calculation matrix x_kernel to calculate the horizontal gradient value g of each pixel. x (i,j), using GM GRAY Perform a convolution operation with the y_kernel matrix (which calculates the gradient along the vertical axis) to obtain the vertical gradient value g for each pixel. y (i,j). Then, the lateral gradient value g of each pixel. x (i,j) and longitudinal gradient value g y (i,j), the smoothed grayscale matrix GM of the image is obtained by calculating it as follows (Formula 3). GRAY The gradient magnitude matrix A g The horizontal gradient value g of each pixel x (i,j) and longitudinal gradient value g y (i,j), the smoothed grayscale matrix GM of the image is obtained by calculating it as follows (Formula 4). GRAY The sum of the gradient direction matrix D g It should be noted that each pixel in the matrix represents the gradient magnitude and gradient direction at its corresponding coordinates. The formula for calculating the gradient magnitude is:

[0059]

[0060] The formula for calculating the gradient direction is:

[0061] D(i,j)=arctan(g y(i,j) / g x(i,j) )×180 / π (Formula 4)

[0062] Where A(i,j) is the gradient magnitude of the pixel at coordinate (i,j), D(i,j) is the gradient direction of the pixel at coordinate (i,j), and gx (i,j) represents the horizontal gradient value of the pixel at coordinate (i,j), g y (i,j) represents the vertical gradient value of the pixel at coordinate (i,j), and arctan is the arctangent function.

[0063] When performing edge detection on objects using the above method, at least the following problems exist:

[0064] (1) The two-dimensional Gaussian filter smooths image noise by calculating the influence of the central pixel on the local pixel weights, and the weight value is larger the closer to the center. When objects in the image produce light and shadow due to illumination, the gray-level difference at the edge of the light and shadow will be more obvious. Then, under the effect of the two-dimensional Gaussian filter principle, the gray-level difference at the edge of the light and shadow will be further amplified, forming more obvious false edge lines. This will cause a lot of interference when performing edge detection on the object, mistaking the edge of the light and shadow as the edge of the object, resulting in the object range being judged to be larger than it actually is.

[0065] (2) The basis for judging the edge of an object in an image is the gradient change of the gray value of the image. The larger the gradient amplitude of the gray value, the greater the gray value difference between the two sides of the place. This gray value difference is the edge of the object in visual presentation. That is, there is still a problem of mistaking the edge of light and shadow as the edge of the object, which leads to inaccurate detection of the edge of the object.

[0066] The following describes exemplary applications of the image data processing device provided in the embodiments of this application. This image data processing device can be implemented as any terminal with a screen display function, such as a laptop, tablet, desktop computer, mobile device (e.g., mobile phone, portable music player, personal digital assistant, dedicated messaging device, portable gaming device), or intelligent robot, or as a server. The following will describe exemplary applications when the image data processing device is implemented as a terminal.

[0067] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the terminal 100 provided in the embodiments of this application. Figure 2 The terminal 100 shown includes at least one processor 110, at least one network interface 120, a user interface 130, and a memory 150. The various components in the terminal 100 are coupled together via a bus system 140. It is understood that the bus system 140 is used to implement communication between these components. In addition to a data bus, the bus system 140 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 1 The general labeled all buses as Bus System 140.

[0068] The processor 110 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] User interface 130 includes one or more output devices 131 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 130 also includes one or more input devices 132, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0070] Memory 150 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Memory 150 may optionally include one or more storage devices physically located remote from processor 110. Memory 150 may include volatile memory or non-volatile memory, or both. Non-volatile memory may be read-only memory (ROM), and volatile memory may be random access memory (RAM). The memory 150 described in this application embodiment is intended to include any suitable type of memory. In some embodiments, memory 150 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

[0071] Operating system 151 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0072] The network communication module 152 is used to reach other computing devices via one or more (wired or wireless) network interfaces 120, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0073] The input processing module 153 is used to detect and translate one or more user inputs or interactions from one or more input devices 132.

[0074] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An image data processing device 154 stored in memory 150 is shown. This image data processing device 154 can be an image data processing device in terminal 100, and can be software in the form of programs and plug-ins, including the following software modules: an acquisition module 1541 and a processing module 1542. These modules are logically linked and can therefore be arbitrarily combined or further split according to the functions they implement. The functions of each module will be described below.

[0075] In other embodiments, the apparatus provided in this application can be implemented in hardware. For example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image data processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0076] The image data processing method provided in this application will be described below with reference to exemplary applications and implementations of the terminal 100 provided in the embodiments of this application. See also Figure 3 , Figure 3 This is an optional flowchart illustrating an image data processing method provided in an embodiment of this application, which will be combined with... Figure 3 The steps shown are explained below.

[0077] Step 301: Obtain first image data at N different scales of the image to be processed that has light and shadow edges.

[0078] Where N is an integer greater than or equal to 2.

[0079] In this embodiment of the application, the image to be processed includes an image of an object with light and shadow edges caused by light illumination.

[0080] In this embodiment, the first image data includes image data with multiple color channels, wherein the multiple color channels can be r-channel components, g-channel components, and b-channel components. Here, the first image data can be represented in matrix form; it should be noted that the image data described below will all be represented in matrix form.

[0081] In this embodiment of the application, considering the computing resources of the terminal when processing image data and the amount of computation for processing image data, the value range of N can be [2,4]. That is to say, the terminal can process the first image data of the image to be processed with two different scales that have light and shadow edges, the terminal can process the first image data of the image to be processed with three different scales that have light and shadow edges, and the terminal can process the first image data of the image to be processed with four different scales that have light and shadow edges. This application does not impose specific limitations on this.

[0082] Optionally, N can be 3, which ensures that the acquired image data is valid and that the terminal uses minimal computing resources, has a small computational load, and processes image data quickly.

[0083] In this embodiment, after obtaining the image to be processed containing light and shadow edges, the color channel component values ​​corresponding to the multi-color channel components of the pixels in the image to be processed are sequentially traversed from left to right and from top to bottom to obtain the image data to be processed. Further, the image data to be processed is transformed at different scales to obtain N first image data with different scales. It should be noted that the N first image data with different scales may include the image data to be processed. Here, in one feasible scenario, taking N=3 as an example, see [link to relevant documentation]. Figure 4 and Figure 5 As shown, Figure 4 This is a schematic diagram of the image to be processed provided in an embodiment of this application. Figure 5 This is a schematic diagram of images represented by first image data at three different scales provided in the embodiments of this application. Figure 5 In this context, A represents the image represented by the first image data at the largest of the three different scales. Figure 5 In this context, B represents the image represented by the first image data at the second largest scale among three different scales. Figure 5 C in the image represents the first image data at the smallest of the three different scales.

[0084] Step 302: Obtain the nth second image data after smoothing the nth first image data.

[0085] Where n is an integer greater than or equal to 1 and less than or equal to N.

[0086] In this embodiment, the first image data can be smoothed using a two-dimensional Gaussian filter matrix or a predefined sparse matrix. This application does not impose any specific limitations on these methods.

[0087] In this embodiment of the application, the first image data is smoothed for noise reduction and to blur the light and shadow edges, thereby eliminating the light and shadow edges. Furthermore, the light and shadow edges are not included when identifying the edges of objects.

[0088] In this embodiment, when the terminal obtains N first image data of different scales of an image to be processed containing light and shadow edges, it performs smoothing processing on the nth first image data to obtain the nth second image data. In one feasible scenario, taking N=3 as an example, see [link to relevant documentation]. Figure 5 and Figure 6 As shown, Figure 6 This is a schematic diagram of images represented by second image data at three different scales provided in the embodiments of this application. Figure 6 In this context, A represents the image obtained by smoothing the first image data (which is the largest of the three different scales) to form the second image data. Figure 6 In this context, B represents the image obtained by smoothing the second image data, which is the second largest image data among three different scales. Figure 6 In this context, C represents the image obtained by smoothing the first image data, which is the smallest of the three different scales, and is then used as the second image data.

[0089] Step 303: Perform color mapping on the nth second image data to obtain the nth third image data.

[0090] Among them, the color channel component values ​​of each pixel in the nth second image data have a mapping relationship with the color component values ​​of the corresponding pixels in the nth third image data.

[0091] In this embodiment, the color channel component values ​​of each pixel include the r color channel component value, g color channel component value, and b color channel component value of each pixel; the color component values ​​include, but are not limited to, chroma component values, saturation component values, and the sum of chroma component values ​​and saturation component values.

[0092] In this embodiment of the application, after the terminal obtains the nth second image data after the nth first image data has been smoothed, the color channel component values ​​of each pixel in the nth second image data are used to perform color mapping on the nth second image data to ensure that there is a mapping relationship between the color component values ​​of each pixel in the nth third image data and the RGB color channel component values ​​of the corresponding pixels in the nth second image data.

[0093] Step 304: Perform data fusion on N third image data to obtain target image data with the same scale as the first image data with the largest scale.

[0094] The target image data represents an image in which there are no light and shadow edges.

[0095] In this embodiment, the maximum scale is the original scale of the image data to be processed corresponding to the image to be processed, that is, the scale of the image data to be processed before data dimensionality reduction processing is performed.

[0096] In this embodiment, when color mapping is performed on the nth second image data to obtain the nth third image data, the N third image data are fused to obtain target image data with eliminated light and shadow edge data. It should be noted that the scale of the target image data is the same as the scale of the largest first image data. In one feasible scenario, see [link to relevant documentation]. Figure 7 As shown, Figure 7 This is a schematic diagram of the image represented by the target image data provided in the embodiments of this application, that is, the image represented by the target image data has had its light and shadow eliminated.

[0097] This application provides an image data processing method. First, it acquires first image data of different scales of the image to be processed. Then, it smooths the first image data, blurring the edges of light and shadow to a certain extent, resulting in second image data. Next, it performs color mapping on the second image data to obtain third image data. The color channel component values ​​of each pixel in the second image data are mapped to the color component values ​​of the corresponding pixels in the third image data. That is, the color channel component values ​​of each pixel in the second image data are converted into color component values ​​in other color spaces to obtain the third image data. This preserves the color attributes of objects and light and shadow edges in the image data, allowing for the elimination of light and shadow by utilizing the difference in color attributes between the objects and light and shadow edges. Finally, data fusion is performed on third image data at different scales to obtain target image data with the same scale as the first image data at the largest scale, and the target image data represents an image without light and shadow edges. In this way, data fusion of third image data at different scales realizes the merging and fusion of multi-scale images, further highlighting the main object region, eliminating the interference of light and shadow, and the edges of the identified objects no longer contain light and shadow edges. Then, edge detection of objects in the image is performed based on the image with light and shadow edges eliminated, which improves the accuracy of edge detection.

[0098] See Figure 8 , Figure 8 This is an optional flowchart illustrating an image data processing method provided in an embodiment of this application, which will be combined with... Figure 8 The steps shown are explained below.

[0099] Step 801: Obtain first image data at N different scales of the image to be processed that has light and shadow edges.

[0100] Step 802: Obtain a sparse matrix with a first preset scale.

[0101] Among them, the first preset scale is smaller than the smallest scale among different scales.

[0102] In this embodiment, the first preset scale is much smaller than the smallest scale among different scales. Here, the first preset scale can be 3×3, 5×5, or 7×7. This application does not impose specific restrictions on the first preset scale.

[0103] In this embodiment, the sparse matrix is ​​a matrix composed of elements 1 and 0 arranged with intervals. For example, if the first preset size is 3×3, then the 3×3 sparse matrix m can be m = [[1,0,1],[0,1,0],[1,0,1]]; if the first preset size is 5×5, then the 5×5 sparse matrix m can be m = [[1,0,1,0,1],[0,1,0,1,0], ...1],[1,0,1,1] 1]]; If the first preset scale is 7×7, then the sparse matrix m of 7×7 can be m=[[1,0,1,0,1,0,1],[0,1,0,1,0,1,0],[1,0,1,0,1,0,1],[0,1,0,1,0,1,0],[1,0,1,0,1,0],[1,0,1,0,1,0,1],[0,1,0,1,0,1,0],[1,0,1,0,1,0,1]].

[0104] Optionally, the first preset scale can be 5×5. In this way, when the sparse matrix performs smoothing processing on the first image data, it blurs the light and shadow and the edges of the light and shadow. The degree of blurring of the light and shadow and the edges of the light and shadow meets the blurring condition, which lays the foundation for eliminating the edges of the light and shadow. The blurring condition can be understood as the ability to eliminate the blurred edges of the light and shadow.

[0105] Step 803: Based on the sparse matrix, smooth the nth first image data to obtain the nth second image data.

[0106] In this embodiment of the application, after obtaining a sparse matrix m of a first preset scale, the sparse matrix is ​​used to smooth the nth first image data to obtain the nth second image data that satisfies the smoothing condition, wherein the smoothing condition includes the aforementioned blurring condition.

[0107] Here, the terminal uses a sparse matrix to smooth the first image data. The purpose is twofold: first, to distribute the weights of the color channel components of each pixel in the first image data through the sparse matrix, thereby achieving noise reduction and smoothing of the first image data; and second, to blur the light and shadow edges in the image to a certain extent, thus ensuring that the obtained second image data is smoother, and at the same time, to perform preliminary processing for eliminating light and shadow edges.

[0108] Step 804: Based on the color channel component values ​​of all pixels in the nth second image data, perform colorimetric component mapping on the nth second image data to obtain the nth colorimetric image data.

[0109] Step 805: Based on the color channel component values ​​of all pixels in the nth second image data, perform saturation component mapping on the nth second image data to obtain the nth saturation image data.

[0110] In this embodiment of the application, image data can be represented using RGB color channel components. Image data can also be represented using chroma components and saturation components. The chroma component is used to represent the brightness of the image color; the saturation is used to represent the density of the image color. The higher the saturation, the more saturated the color. The lower the saturation, the more saturated the color will appear old. When the saturation is 0, the image is a grayscale image.

[0111] In this embodiment, the terminal performs chroma component mapping on each pixel in the nth second image data based on the color channel component values ​​of all pixels in the nth second image data to obtain the chroma component corresponding to each pixel; based on the chroma components corresponding to all pixels, the nth chroma image data is obtained. Further, based on the color channel component values ​​of all pixels in the nth second image data, the terminal performs saturation component mapping on each pixel in the nth second image data to obtain the saturation component corresponding to each pixel; based on the saturation components corresponding to all pixels, the nth saturation image data is obtained. It should be noted that the chroma and saturation of the image at the edge of light and shadow are different from those of the image at the edge of an object. By using color mapping, the second image data is decomposed to obtain chroma image data and saturation image data. Then, according to certain rules, chroma image data containing light and shadow edges and saturation image data containing light and shadow edges are removed. Thus, the elimination of image color data containing light and shadow and light and shadow edges is achieved.

[0112] Step 806: Linearly fuse the nth chroma image data and the nth saturation image data to obtain the nth third image data.

[0113] In this embodiment of the application, the nth chroma image data and the nth saturation image data are linearly fused to obtain the nth third image data. This can be understood as linearly adding the chroma component values ​​of all pixels in the nth chroma image data to the saturation component values ​​of the corresponding pixels in the nth saturation image data to obtain the nth third image data. Each pixel in the nth third image data corresponds to a chroma component value.

[0114] Step 807: Perform data fusion on N third image data to obtain target image data with the same scale as the first image data with the largest scale.

[0115] The target image data represents an image in which there are no light and shadow edges.

[0116] As described above, in this embodiment, firstly, first image data of different scales of the image to be processed is obtained. Then, sparse matrix processing is used to process the first image data, and the image data is smoothed by weight distribution, blurring the light and shadow edges to a certain extent, resulting in second image data of different scales. Next, based on the color channel component values ​​of all pixels in the second image data of different scales, chroma component mapping and saturation component mapping are performed on the second image data respectively to obtain chroma image data and saturation image data of different scales. Then, the chroma image data and saturation image data of the same scale are linearly fused to obtain third image data of different scales. Finally, the third image data of different scales are fused to obtain target image data with the same scale as the first image data of the largest scale, and the image represented by the target image data does not have light and shadow edges. In this way, the color and saturation attributes of objects and light and shadow edges in the image data are preserved. Based on the difference in color and saturation between objects and light and shadow edges, light and shadow are eliminated. Furthermore, data fusion is performed on third image data of different scales, realizing the merging and fusion of multi-scale images, highlighting the main object area, eliminating the interference of light and shadow, and the edges of the identified objects no longer contain light and shadow edges.

[0117] In some embodiments, the process of obtaining N different scales of first image data of the image to be processed containing light and shadow edges in step 801 is combined with... Figure 9 To provide further explanation,

[0118] Step 901: Obtain the image data to be processed corresponding to the image to be processed.

[0119] In this embodiment, the terminal reads the image to be processed and obtains the color channel component values ​​corresponding to different color channel components of each pixel in the image to be processed, thereby obtaining the image data to be processed. Here, the color channel component values ​​corresponding to different color channel components include: the r color channel component value corresponding to the r color channel component, the g color channel component value corresponding to the g color channel component, and the b color channel component value corresponding to the b color channel component. Here, the scale of the image data to be processed is the maximum scale.

[0120] Step 902: Obtain the first template matrix with the second preset scale.

[0121] In this embodiment, the second preset scale is much smaller than the smallest scale among different scales. For example, the second preset scale can be 3×3, 5×5, or 7×7. This application does not impose specific restrictions on the second preset scale.

[0122] In this embodiment of the application, the first template matrix is ​​an identity matrix composed of elements 1. The first template matrix is ​​used to perform dimensionality reduction processing on the image data to be processed in order to obtain image data of different scales.

[0123] Step 903: Based on the first template matrix, perform dimensionality reduction processing on the data of different color channel components in the image data to be processed, and obtain the dimensionality-reduced image data corresponding to each color channel component.

[0124] In this embodiment of the application, dimensionality reduction can be achieved by downsampling. Downsampling can be understood as performing pixel sampling on image data, thereby reducing the image size and resolution.

[0125] In this embodiment, since the image data to be processed includes the r-channel component value, g-channel component value, and b-channel component value corresponding to the r-channel component of each pixel in all pixels, the terminal first extracts the color channel component value corresponding to each color channel component from the image data to be processed, thereby obtaining the image data corresponding to each color channel component, i.e., obtaining the r-channel image data, g-channel image data, and b-channel image data. Secondly, from left to right and from top to bottom, the image data corresponding to each color channel component is subjected to N-1 dimensionality reduction processes sequentially, using the first template matrix as the unit, to obtain N-1 dimensionality-reduced image data at different scales corresponding to each color channel component. For example, if N=2, the first template matrix is ​​used to perform one dimensionality reduction process on the image data corresponding to each color channel component to obtain one dimensionality-reduced image data corresponding to each color channel component; if N=3, the first template matrix is ​​used to perform two consecutive dimensionality reduction processes on the image data corresponding to each color channel component to obtain two dimensionality-reduced image data at different scales corresponding to each color channel component.

[0126] It should be noted that the first template matrix is ​​used to perform a non-repeating traversal of the image data corresponding to each color channel component. Each time, the first template matrix frames a fixed number of pixels in the image data corresponding to each color channel component. The color channel component values ​​corresponding to the fixed number of pixels are weighted and averaged with the unit values ​​corresponding to the pixels in the first template matrix. The average value is then placed into a new matrix, thereby obtaining the dimensionality-reduced image data corresponding to each color channel component.

[0127] Step 904: Merge the dimensionality-reduced image data corresponding to all color channel components to obtain the first image data.

[0128] Among them, the first image data at N different scales includes the first image data and the image data to be processed.

[0129] In this embodiment of the application, the dimensionality-reduced image data corresponding to different color channel components at the same scale are merged to obtain the first image data, and the first image data at N different scales includes the first image data and the image data to be processed.

[0130] In one feasible scenario, taking a second preset scale of 3×3 as an example, the first template matrix is ​​[[1,1,1],[1,1,1],[1,1,1]], see [link to documentation]. Figure 10As shown, a first template matrix is ​​used to iterate through the image data of each color channel component without repetition. Each time, the first template matrix frames 9 pixels within the image data corresponding to each color channel component. The color channel component values ​​corresponding to these 9 pixels are then weighted and averaged with the unit values ​​corresponding to those pixels in the first template matrix. The resulting average is placed into a new matrix, thus obtaining the dimensionality-reduced image data for each color channel component. Finally, the dimensionality-reduced image data corresponding to all color channel components are merged to obtain the first image data.

[0131] As described above, in this embodiment, by performing dimensionality reduction processing on the image data to be processed using different color channel components, dimensionality-reduced image data corresponding to different color channel components at multiple scales are obtained. Then, the dimensionality-reduced image data corresponding to different color channel components at the same scale are merged to obtain the first image data obtained through dimensionality reduction. Here, the image data to be processed and the first image data obtained through dimensionality reduction constitute N first image data at different scales. Thus, by discarding the detailed parts of the image data through dimensionality reduction while retaining the main content area of ​​the image data, and by introducing first image data at different scales, a guarantee is provided for eliminating light and shadow edges.

[0132] In some embodiments, given N first image data of different scales and a sparse matrix of a first preset scale, the process of smoothing the nth first image data based on the sparse matrix in step 803 to obtain the nth second image data is combined with... Figure 11 To provide further explanation,

[0133] Step 1101: Using a sparse matrix, perform a first smoothing process on the nth first image data to obtain the nth initial smoothed image data after eliminating the detail data information in the nth first image data.

[0134] In this embodiment of the application, the first smoothing process is used to eliminate detailed data information in the nth first image data.

[0135] In this embodiment of the application, step 1101 involves using a sparse matrix to perform a first smoothing process on the nth first image data, resulting in the nth initially smoothed image data with the detail data information in the nth first image data eliminated. This is combined with... Figure 12 To provide further explanation,

[0136] Step A1: Use a sparse matrix to traverse the nth first image data row by row and column by column. After the kth traversal, the kth part of the image data is obtained.

[0137] Where k is an integer greater than or equal to 1 and less than or equal to K, and K is the total number of traversals.

[0138] In this embodiment of the application, the k-th part of the image data is the first image data C obtained by using a sparse matrix m in the k-th traversal. n The scale of the k-th part of the image data is the same as the first preset scale.

[0139] In a feasible scenario, referencing Figure 13 As shown, Figure 13 The diagram shows how to obtain the k-th part of the image data by using a sparse matrix to iterate through the nth first image data over the kth time.

[0140] Step A2: Obtain the mean value of the color channel component of any pixel in the k-th part of the image data, and the first difference between the mean value of the color channel component of any pixel and the sparse value corresponding to any pixel in the sparse matrix.

[0141] In this embodiment of the application, the terminal obtains the mean value of the r-color channel component value, g-color channel component value and b-color channel component value of any pixel in all pixels of the k-th part of the image data, and calculates the first difference between the mean value of the pixel and the sparse value corresponding to the pixel in the sparse matrix.

[0142] Step A3: Determine the k-th pixel corresponding to the smallest first difference among all pixels.

[0143] Step A4: In the nth first image data, replace the color channel component value of the center pixel of the kth part of the image data with the color channel component value corresponding to the kth pixel.

[0144] Step A5: After the Kth traversal, in the nth first image data, replace the color channel component value of the center pixel of the Kth part of the image data with the color channel component value corresponding to the Kth pixel to obtain the nth initial smoothed image data.

[0145] In the embodiments of this application, reference is made to Figure 13 As shown, after obtaining the first difference corresponding to all pixels in the k-th part of the image data, the terminal selects the pixel corresponding to the smallest first difference as the first target pixel in the k-th part of the image data, i.e., the k-th pixel; then, in the nth first image data C n In this process, the color channel component value of the center pixel of the k-th part of the image data is replaced with the color channel component value corresponding to the k-th pixel. Furthermore, after the k-th traversal, in the n-th first image data C... n In the process, the color channel component value of the center pixel of the Kth part of the image data is replaced with the color channel component value corresponding to the Kth pixel to obtain the nth initial smoothed image data. Thus, the nth first image data C is eliminated. n Detailed data information.

[0146] Step 1102: Using a sparse matrix, perform a second smoothing process on the nth initial smoothed image data to obtain the nth second image data that highlights the main data information in the nth initial smoothed image data.

[0147] In this embodiment of the application, the second smoothing process is used to highlight the main data information in the nth initial smoothed image data.

[0148] In this embodiment, step 1102 involves using a sparse matrix to perform a second smoothing process on the nth initial smoothed image data, resulting in the nth second image data that highlights the main data information in the nth initial smoothed image data. This process is combined with... Figure 14 To provide further explanation,

[0149] Step B1: Use a sparse matrix to traverse the nth initial smoothed image data. After the lth traversal, the lth part of the image data is obtained.

[0150] Where l is an integer greater than or equal to 1 and less than or equal to L, and L is the total number of iterations. Here, L is the same as K.

[0151] In this embodiment of the application, the l-th part of the image data is the initial smoothed image data of the nth traversal using a sparse matrix m in the l-th iteration. The image data framed at that time, the scale of the l-th part of the image data is the same as the first preset scale.

[0152] Step B2: Obtain the mean value of the color channel component of any pixel in all pixels of the l-th part of the image data, and the first sum between the mean value of the color channel component of any pixel and the sparse value corresponding to any pixel in the sparse matrix.

[0153] In this embodiment of the application, the terminal obtains the mean value of the r-color channel component value, g-color channel component value and b-color channel component value of any pixel in all pixels of the l-part image data, and calculates the first sum value between the mean value of the pixel and the sparse value corresponding to the pixel in the sparse matrix.

[0154] Step B3: Determine the l-th pixel corresponding to the largest first sum among all pixels.

[0155] Step B4: In the nth initial smoothed image data, replace the color channel component value of the center pixel of the l-th part of the image with the color channel component value corresponding to the l-th pixel.

[0156] Step B5: After the Lth traversal is completed, in the nth initial smoothed image data, replace the color channel component value of the center pixel of the Lth part of the image data with the color channel component value corresponding to the Lth pixel to obtain the nth second image data.

[0157] In this embodiment, after obtaining the first sum value corresponding to all pixels in the l-th part of the image data, the terminal selects the pixel corresponding to the largest first sum value as the second target pixel in the l-th part of the image data, i.e., the l-th pixel; then, in the n-th initial smoothed image data... In this process, the color channel component value of the center pixel of the l-th part of the image data is replaced with the color channel component value corresponding to the l-th pixel. Furthermore, after the L-th traversal, in the n-th initial smoothed image data... In the process, the color channel component value of the center pixel of the Lth part of the image data is replaced with the color channel component value corresponding to the Lth pixel to obtain the nth second image data. In this way, the initial smoothed image data of the nth time is highlighted. The main data information in it.

[0158] In some embodiments, when smoothing the nth first image data based on a sparse matrix to obtain the nth second image data, the process of step 804, which maps the colorimetric components of the nth second image data based on the color channel component values ​​of all pixels in the nth second image data to obtain the nth colorimetric image data, is combined with... Figure 15 To provide further explanation,

[0159] Step 1501: If the multiple color channel component values ​​of any pixel in all pixels are equal, determine the chromaticity component value of any pixel as the first chromaticity component value.

[0160] In this embodiment of the application, the fact that the multiple color channel component values ​​of any pixel are equal can be understood as the r color channel component value, g color channel component value, and b color channel component value of any pixel being equal.

[0161] In this embodiment of the application, the first colorimetric component value can be a preset value, for example, the preset value can be 0.

[0162] Step 1502: If the multiple color channel component values ​​of any pixel are not equal, determine the target color channel corresponding to the color channel component value that is greater than the first threshold among the multiple color channel component values ​​of any pixel, select the color degree mapping formula corresponding to the target color channel component, and obtain the second color degree component value of any pixel.

[0163] The nth chromaticity image data includes the first chromaticity component values ​​and / or the second chromaticity component values ​​of all pixels.

[0164] In this embodiment, the first threshold can be a value that is dynamically adjusted based on the color channel component value of each pixel. Alternatively, the first threshold can be a threshold that allows the acquisition of the maximum color channel component value when multiple color channel component values ​​are not equal. This application does not impose any specific restrictions on this.

[0165] In this embodiment, when the multiple color channel component values ​​of any pixel in the nth second image data are all equal, the terminal determines the chromaticity component value of any pixel as the first chromaticity component value; when the multiple color channel component values ​​of any pixel in the nth second image data are not equal, the terminal determines the target color channel corresponding to the color channel component value greater than a first threshold among the multiple color channel component values ​​of any pixel, selects the chromaticity mapping formula corresponding to the target color channel, obtains the second chromaticity component value of any pixel, and then obtains the nth chromaticity image data based on the chromaticity component values ​​of all pixels.

[0166] In this embodiment of the application, the value of the second colorimetric component can be determined by any one of formulas 5-1 to 5-4:

[0167]

[0168] Where h is the second colorimetric component value, max(r,g,b) is the maximum component value among the multiple color channel component values ​​of any pixel, and min(r,g,b) is the minimum component value among the multiple color channel component values ​​of any pixel.

[0169] It should be noted that if the target color channel is r, the calculation of the second chromaticity component value uses either formula (5-1) or formula (5-2). Specifically, when the target color channel is r, if the g color channel component value is greater than or equal to the b color channel component value, the calculation of the second chromaticity component value uses formula (5-1); if the g color channel component value is less than the b color channel component value, the calculation of the second chromaticity component value uses formula (5-2). If the target color channel is g, the calculation of the second chromaticity component value uses formula (5-3); if the target color channel is b, the calculation of the second chromaticity component value uses formula (5-4). Thus, based on the color channel component values ​​of all pixels in the nth second image, the chromaticity component values ​​of all pixels in the nth third image data are obtained, achieving the elimination of light and shadow edges.

[0170] It should be noted that steps 1501 and 1502 can be executed simultaneously, or steps 1501 can be executed before or after steps 1502. This application does not impose any specific restrictions on this.

[0171] In some embodiments, when smoothing the nth first image data based on a sparse matrix to obtain the nth second image data, the process of step 805, which maps the saturation components of the nth second image data based on the color channel component values ​​of all pixels in the nth second image data to obtain the nth saturated image data, is combined with... Figure 16 To provide further explanation,

[0172] Step 1601: If the multiple color channel component values ​​of any pixel in all pixels are preset component values, determine the saturation component value of any pixel as the first saturation component value.

[0173] In this embodiment, the preset component value can be 0. The fact that multiple color channel component values ​​of any pixel are all preset component values ​​can be understood as the r-channel component value, g-channel component value, and b-channel component value of any pixel being equal and all being 0.

[0174] Step 1602: If the multiple color channel component values ​​of any pixel are not preset component values, determine the second saturation component value of any pixel using the saturation mapping formula.

[0175] The nth saturation image data includes the first saturation component value and / or the second saturation component value of all pixels.

[0176] In this embodiment, when the multiple color channel component values ​​of any pixel in the nth second image data are all preset component values, the terminal determines the saturation component value of any pixel as the first saturation component value; when the multiple color channel component values ​​of any pixel in the nth second image data are not preset component values, the second saturation component value of any pixel is determined by a saturation mapping formula, and then the nth saturation image data is obtained based on the saturation component values ​​of all pixels.

[0177] In this embodiment of the application, if none of the multiple color channel component values ​​of any pixel are preset component values, the saturation mapping formula is:

[0178]

[0179] Where s is the second saturation component value, max(r,g,b) is the maximum component value among the multiple color channel component values ​​of any pixel, and min(r,g,b) is the minimum component value among the multiple color channel component values ​​of any pixel. The multiple color channel component values ​​include the red (r) color channel component value, the green (g) color channel component value, and the blue (b) color channel component value. Thus, based on the color channel component values ​​of all pixels in the nth second image, the saturation component values ​​of all pixels in the nth third image data are obtained, achieving the elimination of light and shadow edges.

[0180] It should be noted that steps 1601 and 1602 can be executed simultaneously, or steps 1601 can be executed before or after steps 1602. This application does not impose any specific restrictions on this.

[0181] In some embodiments, when the nth chroma image data and the nth saturation image data are linearly fused to obtain the nth third image data, the process of fusing N third image data in step 807 to obtain target image data with the same scale as the largest scale first image data is combined with... Figure 17 To provide further explanation,

[0182] Step 1701: From N third image data, filter out the third image data whose scale is smaller than the maximum scale.

[0183] In this embodiment, the terminal needs to perform dimensionality expansion processing on the third image data whose scale is not equal to the minimum scale among the N third image data in order to perform logical operations on the multiple image data of the same scale obtained after dimensionality expansion. Therefore, it is necessary to first filter out the third image data whose scale is smaller than the maximum scale from the N third image data.

[0184] Step 1702: Perform dimensional expansion processing on the selected third image data to obtain the fourth image data.

[0185] The scale of the fourth image data is equal to the maximum scale.

[0186] In this embodiment of the application, the dimension expansion process can be achieved by upsampling. Upsampling can be understood as performing pixel filling processing on image data, thereby enlarging the image size and resolution.

[0187] In this embodiment of the application, the process of performing dimension expansion processing on the selected third image data in step 1702 to obtain the fourth image data is combined with... Figure 18 To provide further explanation,

[0188] Step C1: Obtain the second template matrix with the third preset scale.

[0189] The third preset scale is smaller than the second preset scale.

[0190] In this embodiment, the third preset scale is smaller than the second preset scale. For example, if the second preset scale can be 3×3, then the third preset scale is 2×2. This application does not impose specific restrictions on the third preset scale.

[0191] In this embodiment, the second template matrix is ​​an identity matrix composed of elements 1. The second template matrix is ​​used to perform dimensionality expansion processing on the selected third image data to obtain image data at the maximum scale.

[0192] Step C2: Based on the second template matrix, perform dimensional expansion processing on the selected third image data to obtain the expanded third image data.

[0193] Among them, the scale of the expanded third image data is smaller than the maximum scale.

[0194] In one feasible application scenario, taking a third preset scale of 2×2 as an example, the second template matrix is ​​[[1,1],[1,1]], see [link to relevant documentation]. Figure 19 As shown, the second template matrix is ​​used to perform a non-repeating traversal of the selected third image data. Each time, the second template matrix frames an image data set containing 4 pixels within the selected third image data. The color component values ​​corresponding to these 4 pixels are then upsampled using the upsampling algorithm (Formula 7) to obtain the expanded image data. After the traversal using the second template matrix, the expanded third image data is obtained. Since the scale of the second template matrix used for expansion is smaller than the scale of the first template matrix used for dimensionality reduction, the scale of the expanded third image data is still smaller than the maximum scale. Here, the upsampling algorithm (Formula 7) is:

[0195]

[0196]

[0197]

[0198]

[0199]

[0200]

[0201] Where P0, P1, P2, and P3 are the color component values ​​of the pixels obtained by framing the filtered third image data using the second template matrix, and E[P] is the mean of P0, P1, P2, and P3. 01P is the color component value inserted between P0 and P1. 02 P represents the color component values ​​inserted between P0 and P2. 23 P represents the color component values ​​inserted between P2 and P3. 13 P is the color component value inserted between P1 and P3. c The color component values ​​inserted at the center position.

[0202] Step C3: Continue to expand the dimensions of the expanded third image data based on the second template matrix until the fourth image data with the same maximum scale is obtained.

[0203] In this embodiment of the application, after the third image data is obtained after one dimension expansion process, the second template matrix is ​​used to perform another dimension expansion process on the third image data until the fourth image data with the same maximum scale is obtained.

[0204] Step 1703: Perform logical operations on the third and fourth image data, which are at the same maximum scale, to obtain the target image data.

[0205] In this embodiment, the terminal performs binary data conversion on the color component values ​​of all pixels in the third image data with the same maximum scale to obtain the first binary color component value. It then performs data conversion on the color component values ​​of all pixels in the fourth image data to obtain the second binary color component value. The terminal then performs a bitwise AND operation on the first binary color component value of any pixel in the third image data with the second binary color component value of the corresponding pixel in the fourth image data to obtain the target binary color component value of that pixel. This target binary color component value is then converted to decimal data to obtain the target color component value of that pixel. This process is repeated to obtain the target color component values ​​of all pixels, thus obtaining the target image data. In this way, the third image data with a scale smaller than the maximum scale is expanded to obtain the fourth image data with the same maximum scale, achieving scale normalization. Furthermore, logical operations are performed on the color component values ​​of the same pixel in the third and fourth image data with the same scale as the maximum scale to obtain the target image data after shadow removal. This eliminates light and shadow edges in the image. Finally, edge detection of objects in the image is performed based on the image with the light and shadow edges removed, improving the accuracy of edge detection.

[0206] See Figure 20 , Figure 20 This is an optional flowchart illustrating an image data processing method provided in an embodiment of this application, which will be combined with... Figure 20 The steps shown are explained below.

[0207] Step 2001: Obtain first image data at N different scales of the image to be processed that has light and shadow edges.

[0208] Where N is an integer greater than or equal to 2.

[0209] Step 2002: Obtain the nth second image data after smoothing the nth first image data.

[0210] Where n is an integer greater than or equal to 1 and less than or equal to N.

[0211] Step 2003: Perform color mapping on the nth second image data to obtain the nth third image data.

[0212] Among them, the color channel component values ​​of each pixel in the nth second image data have a mapping relationship with the color component values ​​of the corresponding pixels in the nth third image data.

[0213] Step 2004: Perform data fusion on N third image data to obtain target image data with the same scale as the first image data with the largest scale.

[0214] The target image data represents an image in which there are no light and shadow edges.

[0215] Step 2005: Obtain the gradient magnitude data and gradient direction data of the target image data.

[0216] In this embodiment, the terminal obtains the horizontal gradient calculation matrix as x_kernel and the vertical gradient calculation matrix as y_kernel. For example, the horizontal gradient calculation matrix is ​​x_kernel = [[-4,0,4],[-20,0,20],[-4,0,4]] and the vertical gradient calculation matrix is ​​y_kernel = [[-4,-20,-4],[0,0,0],[4,20,4]]. Then, the target image data U is convolved with the horizontal gradient calculation matrix x_kernel to calculate the horizontal gradient value g of each pixel. x (i,j) Perform a convolution operation between the target image data U and the vertical gradient calculation matrix y_kernel to calculate the vertical gradient value g of each pixel. y (i,j). Further, based on the lateral gradient value g of each pixel... x (i,j) and longitudinal gradient value g y (i,j) can be calculated using (Formula 3) and (Formula 4) above, which can be used to calculate the gradient magnitude A(i,j) and gradient direction D(i,j) of each pixel in the target image data, thereby obtaining the gradient magnitude data and gradient direction data of the target image data.

[0217] Step 2006: Based on gradient magnitude data and gradient direction data, perform non-maximum suppression on the gradient magnitude of the target image data to obtain initial edge image data.

[0218] In this embodiment of the application, the terminal is based on the horizontal gradient value g of each pixel. x (i,j), longitudinal gradient value g y Given the gradient magnitude A(i,j) and gradient direction D(i,j), we obtain the lateral gradient data g of the target image data, which includes all pixels. x Longitudinal gradient data g y and gradient magnitude data A g and gradient direction data D g Furthermore, simultaneously iterate through these four data points, determining whether the gradient magnitude of coordinate (i,j) is maximized along the gradient direction within its eight neighborhood. Figure 21 As shown, if the maximum gradient magnitude (e.g., the gradient magnitude at point P0 is greater than that at points Ptmp1 and Ptmp2) is obtained, then the gradient magnitude data A is retained. g If the gradient magnitude value at the inner coordinate (i,j) is not the maximum, then the gradient magnitude data A is... g The gradient magnitude at the inner coordinate (i,j) is set to 0. The gradient magnitude at point Ptmp1 is calculated as follows:

[0219] A(Ptmp1)=w×A(P2)+(1-w)×A(P1) (Formula 8)

[0220] Where A(Ptmp1) is the gradient magnitude at point Ptmp1, A(P2) is the gradient magnitude at point P2, A(P1) is the gradient magnitude at point P1, and w is the weight of the gradient magnitudes at points P1 and P2. It should be noted that since point Ptmp1 is the intersection of the gradient direction at point P0 and the lines connecting points P1 and P2, according to Formula 8, the gradient magnitude at point Ptmp1 is calculated by summing the weights of the gradient magnitudes at points P1 and P2. The formula for calculating the weight w is as follows:

[0221]

[0222] It is important to emphasize that, given the gradient direction, horizontal gradient value, and vertical gradient value at point P0, the weight w can be calculated by substituting them into Formula 9. Then, the gradient magnitude at Ptmp1 can be calculated based on w. The gradient magnitude at Ptmp2 can be obtained similarly. Using this method, the gradient magnitude data A can be calculated iteratively. g By analyzing all non-border pixels, we obtain Non-Maximum Suppression (NMS) image data, which is the initial edge image data.

[0223] Step 2007: Based on the obtained second and third thresholds, perform edge detection on the initial edge image data to obtain the target edge image data.

[0224] In this embodiment, the second threshold can be a high threshold maxVal, and the third threshold can be a low threshold minVal. For example, the high threshold maxVal can be 15% to 17% of the maximum gradient magnitude in the Non-Maximum Suppression (NMS) image data, and the low threshold minVal can be 9% to 11% of the maximum gradient magnitude in the NMS image data. For instance, the high threshold maxVal can be 16% of the maximum gradient magnitude in the NMS image data, and the low threshold minVal can be 10% of the maximum gradient magnitude in the NMS image data. This application does not impose specific limitations on these limitations.

[0225] Here, the process iterates through each pixel in the Non-Maximum Suppression (NMS) image data. If the gradient magnitude of the current pixel is greater than or equal to the high threshold `maxVal`, it is considered an edge point, and the current pixel is set to 255. If the gradient magnitude of the current pixel is less than or equal to the low threshold `minVal`, it is not an edge point, and the current pixel is suppressed, i.e., set to 0. If the gradient magnitude of the current pixel is between the high threshold `maxVal` and the low threshold `minVal`, the process iterates through the eight neighbors of the current pixel. First, it checks if there is an edge point. If there is, the current pixel is an extension of the edge point, and the current pixel is classified as an edge point, i.e., set to 255. If there is no edge point, it checks if there is a pixel greater than the low threshold `minVal`. If there is, the process searches its eight neighbors again. This process is repeated until an edge point is encountered. If so, it indicates that this series of pixels forms an image edge closure with the edge point and is an extension of the edge point, and this series of pixels is set to 255. If no edge point is encountered and no pixel greater than the low threshold `minVal` is encountered, this series of pixels is set to 0. After completing the above steps, edge detection of the initial edge image data is complete. At this point, the binary matrix of the image edges is obtained, which is the target edge image data. For example... Figure 22 As shown, Figure 22 The image shows the detection results of edge detection on the object. Clearly, this embodiment of the application achieves the elimination of light and shadow edges, more accurately detects the edges of objects, and reduces interfering edges in the image to a certain extent.

[0226] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0227] Here, we'll illustrate with a real-world scenario: a car dealership takes out a loan from a bank to purchase a new car. The bank monitors the entry and exit of these new cars from its warehouse or storage area. Because the vehicles are parked on the ground, they are affected by lighting, resulting in light and shadow edges. Therefore, in this scenario, we obtain multiple frames of images, and for each frame, we perform edge detection on the vehicle, referring to... Figure 23 As shown, this can be achieved through the following steps:

[0228] Step 2301: Obtain the image to be processed.

[0229] Step 2302: Downsample the image to be processed twice to obtain RGB pixel matrices of three different scales.

[0230] In this embodiment, the image to be processed is read, and the RGB pixel matrix of the image is obtained. From left to right and from top to bottom, the RGB pixel matrix of the image is traversed in sequence using a 3×3 first template matrix [[1,1,1],[1,1,1],[1,1,1]]. Two consecutive downsampling operations are performed to obtain three RGB pixel matrices of different scales: pixel matrix C1, pixel matrix C2, and pixel matrix C3, with their length and width decreasing sequentially. Pixel matrix C1 is the RGB pixel matrix of the image to be processed, pixel matrix C2 is obtained by downsampling C1, and pixel matrix C3 is obtained by sampling C2.

[0231] The downsampling process is as follows: Figure 10 As shown, the 3×3 matrix on the left is traversed in the middle RGB pixel matrix in blocks without repetition. Each bounding area contains 9 pixel coordinates. The weighted average of these 9 pixel coordinates and the corresponding values ​​on the 3×3 matrix is ​​calculated and put into the new matrix on the right, generating a new image RGB pixel matrix.

[0232] Step 2303: Use a sparse matrix to calculate and smooth the RGB pixel matrix of the image to obtain the second image matrix.

[0233] In this embodiment of the application, firstly, a sparse matrix m of size 5×5 is defined as the unit structuring element, that is: m=[[1,0,1,0,1],[0,1,0,1,0],[1,0,1,0,1],[0,1,0,1,0],[1,0,1,0,1]).

[0234] Secondly, the pixel matrix C1 is processed by traversing the pixels of C1 row by row and column by column using a sparse matrix m. The neighborhood of the center pixel P0 defined by m in C1 is the matrix block M1 used in each calculation. m After each calculation, only point P0 is retained, and the result is obtained after traversal. Figure 7The final result of point P0 in each iteration is calculated according to the following formula:

[0235]

[0236]

[0237] in, For matrix block M1 m The RGB r component value of the pixel with coordinates (i,j). For matrix block M1 m The RGB g component value of the pixel with coordinates (i,j). For matrix block M1 m The RGB b-component value at the pixel with coordinates (i,j), m (i,j) Let be the sparse values ​​at coordinates (i,j) in the sparse matrix m.

[0238] In (Formula 10), the matrix block M1 is calculated. m The set of differences between the average RGB component values ​​of each coordinate point and the corresponding coordinate points of the sparse matrix m is then used to find the minimum value of the set using min[]. (x,y) For m and M1 m After calculating the difference between corresponding coordinate points, obtain the coordinates (i,j) of the smallest difference in the set of differences as the min value. (x,y) In (Formula 11), matrix block M1 m The corresponding coordinate point min (x,y) The RGB value at point P0 is assigned to the center pixel P0. After this process is repeated, the initial smoothed image matrix is ​​obtained.

[0239] Furthermore, the initial smoothed image matrix is ​​traversed row by row and column by column again using the sparse matrix m. Given m in the initial smooth image matrix The neighborhood of the center pixel P0 defined in the bounding box is the matrix block M2 that participates in each calculation. m The calculation formula is:

[0240]

[0241]

[0242] In (Formula 13), the matrix block M2 is calculated. m The set of average RGB component values ​​at each coordinate point and the sum of values ​​at the corresponding coordinate points of the sparse matrix m, where max[] represents the maximum value. (x,y) For m and M2 mAfter calculating the sum of the corresponding coordinates, obtain the coordinates (i,j) of the maximum sum in the set of sums as the max value. (x,y) In (Formula 13), matrix block M2 m The corresponding coordinate point max (x,y) The RGB value at point P0 is assigned to the center pixel P0. After this process of calculation is completed, the second image matrix is ​​obtained.

[0243] Finally, similarly, pixel matrix C2 and pixel matrix C3 are processed in the same way described above to obtain the second image matrix. Second image matrix Thus, three second image matrices of different scales are obtained.

[0244] Step 2304: Based on the second image matrix, obtain the saturation feature matrix and the chroma feature matrix respectively, and linearly fuse the saturation feature matrix and the chroma feature matrix to obtain the third image matrix.

[0245] In this embodiment of the application, the second image matrix is ​​obtained. The RGB component values ​​r, g, b are used to calculate the colorimetric value of each pixel using formula 14, resulting in the second image matrix. Colorimetric feature matrix Substituting the component value of each pixel into colorimetric calculation formula 15 yields the second image matrix. saturation characteristic matrix Similarly, the second image matrix can be obtained. Colorimetric feature matrix and saturation characteristic matrix pixel matrix Colorimetric feature matrix and saturation characteristic matrix Here, the formulas for calculating chroma and saturation are as follows:

[0246]

[0247]

[0248] Furthermore, the second image matrix is ​​obtained through Formula 15. Colorimetric feature matrix and saturation characteristic matrix Perform linear addition to obtain the third image matrix. The second image matrix Colorimetric feature matrix and saturation characteristic matrix Perform linear addition to obtain the third image matrix. pixel matrix Colorimetric feature matrix and saturation characteristic matrix Perform linear addition to obtain the third image matrix. Here, Formula 15 is as follows:

[0249]

[0250]

[0251]

[0252] Step 2305: Scale normalize the third image matrices at different scales and perform logical fusion to obtain the target image matrix.

[0253] In this embodiment, the second template matrix [[1,1],[1,1]] of 2×2 is obtained as the unit, and the third image matrix is ​​traversed sequentially from left to right and from top to bottom. and the third image matrix Each defined area contains four coordinate points, namely P0, P1, P2, and P3, as follows: Figure 19 As shown, during each traversal, based on these four pixels, the third image matrix is ​​adjusted using Formula 7. Perform two upsampling operations to obtain the matrix of the third image. A scale-consistent fourth image matrix Each iteration is based on these four pixels, and the third image matrix is ​​processed using Formula 7. Perform six upsampling operations to obtain the result with the third image matrix. A scale-consistent fourth image matrix This ensures the third image matrix Fourth image matrix and the fourth image matrix The scale is made uniform. Furthermore, the third image matrix is ​​scaled using Equation 16. Fourth image matrix and the fourth image matrix Logical operations are performed to obtain the target image matrix U, which is the deshaded image matrix U. Here, Equation 16 is...

[0254] Step 2306: Based on the target image matrix, obtain the horizontal gradient matrix, vertical gradient matrix, gradient magnitude matrix, and gradient direction matrix.

[0255] In this embodiment, the terminal obtains the horizontal gradient calculation matrix as x_kernel and the vertical gradient calculation matrix as y_kernel. For example, the horizontal gradient calculation matrix is ​​x_kernel = [[-4,0,4],[-20,0,20],[-4,0,4]] and the vertical gradient calculation matrix is ​​y_kernel = [[-4,-20,-4],[0,0,0],[4,20,4]]. Then, the target image matrix U is convolved with the horizontal gradient calculation matrix x_kernel to calculate the horizontal gradient value g of each pixel. x (i,j) Perform a convolution operation between the target image matrix U and the vertical gradient calculation matrix y_kernel to calculate the vertical gradient value g for each pixel. y (i,j). Further, based on the lateral gradient value g of each pixel... x (i,j) and longitudinal gradient value g y Given (i,j), calculate the gradient magnitude A(i,j) and gradient direction D(i,j) of each pixel in the target image matrix, and then obtain the horizontal gradient matrix g. x Vertical gradient matrix g y Gradient magnitude matrix A g and gradient direction matrix D g .

[0256] Step 2307: Based on the horizontal gradient matrix, vertical gradient matrix, gradient magnitude matrix, and gradient direction matrix, perform non-maximum suppression to obtain the image non-maximum suppression matrix.

[0257] In this embodiment of the application, after obtaining the transverse gradient matrix g x The vertical gradient matrix g y Gradient magnitude matrix A g and gradient direction matrix D g In this case, simultaneously traverse these four matrices and determine whether the gradient magnitude of coordinate (i,j) in the gradient direction within its eight neighborhood is maximized, such as... Figure 21 As shown, if the maximum gradient magnitude at point P0 is greater than that at points Ptmp1 and Ptmp2, then matrix A is retained. g If the gradient magnitude at the inner coordinate (i,j) is not the maximum, then matrix A will be... g The gradient magnitude at the inner coordinate (i,j) is set to 0. The gradient magnitude at point Ptmp1 is calculated as follows:

[0258] A(Ptmp1)=w×A(P2)+(1-w)×A(P1) (Formula 8)

[0259] Where A(Ptmp1) is the gradient magnitude at point Ptmp1, A(P2) is the gradient magnitude at point P2, A(P1) is the gradient magnitude at point P1, and w is the weight of the gradient magnitudes at points P1 and P2. It should be noted that since point Ptmp1 is the intersection of the gradient direction at point P0 and the lines connecting points P1 and P2, according to Formula 8, the gradient magnitude at point Ptmp1 is calculated by summing the weights of the gradient magnitudes at points P1 and P2. The formula for calculating the weight w is as follows:

[0260]

[0261] It is important to emphasize that, given the gradient direction, horizontal gradient value, and vertical gradient value at point P0, the weight w can be calculated by substituting them into Formula 9. Then, the gradient magnitude at Ptmp1 can be calculated based on w. The gradient magnitude at Ptmp2 can be obtained similarly. Using this method, the gradient magnitude of the image data A can be calculated iteratively. g The image non-maximum suppression matrix (NMS) is obtained by analyzing all non-border pixels in the image.

[0262] Step 2308: Based on the image non-maximum suppression matrix, determine the image edge binary matrix through double thresholding.

[0263] In this embodiment, a high threshold maxVal and a low threshold minVal are calculated. The default value of maxVal is 16% of the maximum gradient magnitude in the Non-Maximum Suppression Matrix (NMS), and the default value of minVal is 10% of the maximum gradient magnitude in the NMS. For example, maxVal = 900.252911218 and minVal = 517.15937003377504.

[0264] Then, iterate through each pixel in the Non-Maximum Suppression (NMS) matrix. If the gradient magnitude of the current pixel is greater than or equal to maxVal, it is considered an edge point, and the pixel value is set to 255. If the gradient magnitude is less than or equal to minVal, it is not an edge point, and the pixel value is suppressed and set to 0. If the gradient magnitude is between maxVal and minVal, search the eight-neighborhood of the current pixel. First, determine if an edge point exists. If it does, the current pixel is an extension of the edge point, and the pixel value is assigned to the edge point and set to 255. If no edge point exists, determine if a pixel value greater than minVal exists. If it does, search the eight-neighborhood of this pixel. Repeat this process until an edge point is encountered. If an edge point is encountered, it indicates that this series of pixels forms an image edge closure with the edge point and is an extension of the edge point, so the pixel value is set to 255. If no edge point is encountered and no pixel value greater than minVal is encountered, the pixel value is set to 0. After completing the above steps, obtain the binary edge matrix, which is the final edge detection result. Thus, when a car casts a shadow under sunlight, the steps in this embodiment eliminate the edges of the shadow, thereby eliminating the influence of light and shadow on the vehicle. The edges of the car can be detected more accurately under illumination, and the interference edges in the image are reduced to a certain extent, thus enabling better monitoring of the vehicle.

[0265] The following continues to describe the exemplary structure of the image data processing apparatus 154 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 1 As shown, the software module stored in the image data processing device 154 in the memory 150 can be the image data processing device in the terminal 100, including:

[0266] The module 1541 is used to obtain first image data at N different scales of the image to be processed that has light and shadow edges, where N is an integer greater than or equal to 1;

[0267] The module 1541 is also used to obtain the nth second image data after the nth first image data has been smoothed, where n is an integer greater than or equal to 1 and less than or equal to N;

[0268] The processing module 1542 is used to perform color mapping on the nth second image data to obtain the nth third image data, wherein the color channel component values ​​of each pixel in the nth second image data have a mapping relationship with the color component values ​​of the corresponding pixels in the nth third image data.

[0269] The processing module 1542 is also used to perform data fusion on N third image data to obtain target image data with the same scale as the first image data with the largest scale, wherein there are no light and shadow edges in the image represented by the target image data.

[0270] In other embodiments of this application, the obtaining module 1541 is further configured to obtain a sparse matrix of a first preset scale, wherein the first preset scale is smaller than the smallest scale among different scales; the processing module 1542 is further configured to perform smoothing processing on the nth first image data based on the sparse matrix to obtain the nth second image data.

[0271] In other embodiments of this application, the processing module 1542 is further configured to use a sparse matrix to perform a first smoothing process on the nth first image data to obtain an nth initial smoothed image data that eliminates detail data information in the nth first image data; and to use a sparse matrix to perform a second smoothing process on the nth initial smoothed image data to obtain an nth second image data that highlights the main data information in the nth initial smoothed image data.

[0272] In other embodiments of this application, the processing module 1542 is further configured to traverse the nth first image data row by row and column by column using a sparse matrix, and after the kth traversal, obtain the kth part of the image data, where k is an integer greater than or equal to 1 and less than or equal to K, and K is the total number of traversals; the obtaining module 1541 is further configured to obtain the first difference between the mean of the color channel component values ​​of any pixel in all pixels in the kth part of the image data and the sparse value corresponding to any pixel in the sparse matrix; the processing module 1542 is further configured to determine the kth pixel corresponding to the smallest first difference among all pixels; in the nth first image data, replace the color channel component value of the center pixel of the kth part of the image data with the color channel component value corresponding to the kth pixel; when the Kth traversal ends, in the nth first image data, replace the color channel component value of the center pixel of the kth part of the image data with the color channel component value corresponding to the kth pixel to obtain the nth initial smoothed image data.

[0273] In other embodiments of this application, the processing module 1542 is further configured to traverse the nth initial smoothed image data using a sparse matrix, and after the lth traversal, obtain the lth part of image data, where l is an integer greater than or equal to 1 and less than or equal to L, and L is the total number of traversals; the obtaining module 1541 is further configured to obtain the first sum between the mean of the color channel component values ​​of any pixel in all pixels of the lth part of image data and the sparse values ​​corresponding to any pixel in the sparse matrix; the processing module 1542 is further configured to determine the lth pixel corresponding to the largest first sum among all pixels; in the nth initial smoothed image data, replace the color channel component value of the center pixel of the lth part of image data with the color channel component value corresponding to the lth pixel; if the Lth traversal ends, in the nth initial smoothed image data, replace the color channel component value of the center pixel of the Lth part of image data with the color channel component value corresponding to the Lth pixel to obtain the nth second image data.

[0274] In other embodiments of this application, the processing module 1542 is further configured to perform chroma component mapping on the nth second image data based on the color channel component values ​​of all pixels in the nth second image data to obtain nth chroma image data; perform saturation component mapping on the nth second image data based on the color channel component values ​​of all pixels in the nth second image data to obtain nth saturation image data; and linearly fuse the nth chroma image data and the nth saturation image data to obtain nth third image data.

[0275] In other embodiments of this application, the processing module 1542 is further configured to: if the multiple color channel component values ​​of any pixel in all pixels are equal, determine the chromaticity component value of any pixel as a first chromaticity component value; if the multiple color channel component values ​​of any pixel in all pixels are not equal, determine the target color channel corresponding to the color channel component value greater than a first threshold among the multiple color channel component values ​​of any pixel, select the chromaticity mapping formula corresponding to the target color channel component, and obtain the second chromaticity component value of any pixel, wherein the nth chromaticity image data includes the first chromaticity component values ​​and / or the second chromaticity component values ​​of all pixels.

[0276] In other embodiments of this application, the processing module 1542 is further configured to: if the multiple color channel component values ​​of any pixel among all pixels are preset component values, determine the saturation component value of any pixel as a first saturation component value; if the multiple color channel component values ​​of any pixel among all pixels are not preset component values, determine the second saturation component value of any pixel through a saturation mapping formula, wherein the nth saturation image data includes the first saturation component value and / or the second saturation component value of all pixels.

[0277] In other embodiments of this application, the saturation mapping formula is:

[0278]

[0279] Where s is the second saturation component value, max(r,g,b) is the maximum component value among the multiple color channel component values ​​of any pixel, and min(r,g,b) is the minimum component value among the multiple color channel component values ​​of any pixel. The multiple color channel component values ​​include the red r color channel component value, the green g color channel component value, and the blue b color channel component value.

[0280] In other embodiments of this application, the obtaining module 1541 is further configured to obtain image data to be processed corresponding to the image to be processed; obtain a first template matrix of a second preset scale; the processing module 1542 is further configured to, based on the first template matrix, reduce the dimensionality of the data of different color channel components in the image data to be processed, to obtain the dimensionality-reduced image data corresponding to each color channel component; merge the dimensionality-reduced image data corresponding to all color channel components to obtain first image data; wherein, the first image data of N different scales includes the first image data and the image data to be processed.

[0281] In other embodiments of this application, the processing module 1542 is further configured to: filter out third image data with a scale smaller than the maximum scale from N third image data; perform dimensional expansion processing on the filtered third image data to obtain fourth image data, wherein the scale of the fourth image data is equal to the maximum scale; and perform logical operations on the third image data and the fourth image data with the same maximum scale to obtain target image data.

[0282] In other embodiments of this application, the obtaining module 1541 is further configured to obtain a second template matrix with a third preset scale, wherein the third preset scale is smaller than the second preset scale; the processing module 1542 is further configured to perform dimensional expansion processing on the selected third image data based on the second template matrix to obtain dimensional expansion third image data, wherein the scale of the dimensional expansion third image data is smaller than the maximum scale; and continue to perform dimensional expansion processing on the dimensional expansion third image data based on the second template matrix again until a fourth image data with the same maximum scale is obtained.

[0283] In other embodiments of this application, the obtaining module 1541 is further configured to obtain gradient magnitude data and gradient direction data of the target image data; the processing module 1542 is further configured to perform non-maximum suppression on the gradient magnitude of the target image data based on the gradient magnitude data and gradient direction data to obtain initial edge image data; and perform edge detection on the initial edge image data based on the obtained second threshold and third threshold to obtain target edge image data.

[0284] This application provides a storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to perform the method provided in this application, for example... Figure 3 , Figures 8-9 , Figures 11-12 , Figures 14-18 , Figure 22 as well as Figure 23 The method shown.

[0285] In some embodiments, the storage medium may be a computer-readable storage medium, such as a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface memory, optical disc, or a compact disk-read-only memory (CD-ROM); or it may be a device that includes one or any combination of the above-mentioned memories.

[0286] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0287] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file containing other programs or data, for example, in one or more scripts within a Hyper Text Markup Language (HTL) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files storing one or more modules, subroutines, or code sections). As an example, executable instructions may be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0288] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An image data processing method, characterized in that, include: Obtain first image data at N different scales of the image to be processed that has light and shadow edges, where N is an integer greater than or equal to 2; Obtain the nth second image data after smoothing the nth first image data, where n is an integer greater than or equal to 1 and less than or equal to N; Color mapping is performed on the nth second image data to obtain the nth third image data, wherein the color channel component values ​​of each pixel in the nth second image data have a mapping relationship with the color component values ​​of the corresponding pixels in the nth third image data; Data fusion is performed on N third image data to obtain target image data with the same scale as the first image data with the largest scale, wherein the target image data represents an image in which the light and shadow edges do not exist; The step of color mapping the nth second image data to obtain the nth third image data includes: Based on the color channel component values ​​of all pixels in the nth second image data, colorimetric component mapping is performed on the nth second image data to obtain the nth colorimetric image data; Based on the color channel component values ​​of all pixels in the nth second image data, saturation component mapping is performed on the nth second image data to obtain the nth saturation image data; The nth chroma image data and the nth saturation image data are linearly fused to obtain the nth third image data; Specifically, the step of mapping the nth second image data to colorimetric components based on the color channel component values ​​of all pixels in the second image data to obtain the nth colorimetric image data includes: If the multiple color channel component values ​​of any pixel among all the pixels are equal, the colorimetric component value of any pixel is determined to be the first colorimetric component value; If the multiple color channel component values ​​of any pixel among all pixels are not equal, determine the target color channel corresponding to the color channel component value greater than a first threshold among the multiple color channel component values ​​of the pixel, select the colorimetric mapping formula corresponding to the target color channel, and obtain the second colorimetric component value of the pixel. The nth colorimetric image data includes the first colorimetric component values ​​and / or the second colorimetric component values ​​of all pixels. The first threshold is a value dynamically adjusted based on the color channel component values ​​of each pixel, or the first threshold is a threshold that can obtain the maximum color channel component value when the multiple color channel component values ​​are not equal.

2. The method according to claim 1, characterized in that, The process of obtaining the nth second image data after smoothing the nth first image data includes: Obtain a sparse matrix with a first preset scale, where the first preset scale is smaller than the smallest of the different scales; Based on the sparse matrix, the nth first image data is smoothed to obtain the nth second image data.

3. The method according to claim 2, characterized in that, The process of smoothing the nth first image data based on the sparse matrix to obtain the nth second image data includes: Using the sparse matrix, the nth first image data is subjected to a first smoothing process to obtain the nth initial smoothed image data after the detailed data information in the nth first image data has been eliminated; Using the sparse matrix, a second smoothing process is performed on the nth initial smoothed image data to obtain the nth second image data that highlights the main data information in the nth initial smoothed image data.

4. The method according to claim 3, characterized in that, The step of using the sparse matrix to perform a first smoothing process on the nth first image data to obtain an nth initial smoothed image data that has eliminated detail data information in the nth first image data includes: The sparse matrix is ​​used to traverse the nth first image data row by row and column by column. After the kth traversal, the kth part of the image data is obtained, where k is an integer greater than or equal to 1 and less than or equal to K, and K is the total number of traversals. The first difference between the mean value of the color channel component of any pixel in all pixels of the k-th part of the image data and the sparse value corresponding to any pixel in the sparse matrix is ​​obtained. Determine the k-th pixel corresponding to the smallest first difference among all the pixels; In the nth first image data, the color channel component value of the center pixel of the kth part of the image data is replaced with the color channel component value corresponding to the kth pixel; When the Kth traversal is completed, in the nth first image data, the color channel component value of the center pixel of the Kth part of the image data is replaced with the color channel component value corresponding to the Kth pixel to obtain the nth initial smoothed image.

5. The method according to claim 3, characterized in that, The step of using the sparse matrix to perform a second smoothing process on the nth initial smoothed image data to obtain the nth second image data that highlights the main data information in the nth initial smoothed image data includes: The sparse matrix is ​​used to traverse the nth initial smoothed image data, after which... The truncation yields the th... Partial image data, It is an integer greater than or equal to 1 and less than or equal to L, where L is the total number of iterations; Obtain the first The first sum between the mean of the color channel component values ​​of any pixel in all pixels of a partial image data and the sparse value corresponding to any pixel in the sparse matrix; Determine the first sum value among all the first sum values ​​corresponding to the largest first sum value. Pixel; In the nth initial smoothed image data, the nth The color channel component values ​​of the center pixel of a portion of the image data are replaced with the first... The color channel component values ​​corresponding to the pixel; When the Lth traversal ends, in the nth initial smoothed image data, the color channel component value of the center pixel of the Lth part of the image data is replaced with the color channel component value corresponding to the Lth pixel to obtain the nth second image data.

6. The method according to claim 1, characterized in that, The step of mapping the nth second image data to the saturation component based on the color channel component values ​​of all pixels in the second image data to obtain the nth saturation image data includes: If the multiple color channel component values ​​of any pixel among all the pixels are all preset component values, the saturation component value of any pixel is determined to be the first saturation component value. If none of the multiple color channel component values ​​of any pixel among all the pixels are the preset component values, the second saturation component value of any pixel is determined by the saturation mapping formula, wherein the nth saturation image data includes the first saturation component value and / or the second saturation component value of all pixels.

7. The method according to claim 6, characterized in that, The saturation mapping formula is: in, This is the value of the second saturation component. The maximum component value among the multiple color channel component values ​​of any pixel. It is the minimum component value among multiple color channel component values ​​of any pixel, wherein the multiple color channel component values ​​include red (r) color channel component value, green (g) color channel component value, and blue (b) color channel component value.

8. The method according to any one of claims 1 to 7, characterized in that, Before obtaining the first image data at N different scales of the image to be processed containing light and shadow edges, the method includes: Obtain the image data to be processed corresponding to the image to be processed; Obtain the first template matrix with the second preset scale; Based on the first template matrix, the data of different color channel components in the image data to be processed are dimensionality reduced respectively to obtain the dimensionality-reduced image data corresponding to each color channel component. The dimensionality-reduced image data corresponding to all color channel components are merged to obtain the first image data; wherein, the N first image data at different scales include: the first image data and the image data to be processed.

9. The method according to any one of claims 1 to 7, characterized in that, The process of fusing N third image data to obtain target image data with the same scale as the first image data at the largest scale includes: From the N third image data, select third image data with a scale smaller than the maximum scale; The selected third image data is subjected to dimensionality expansion processing to obtain fourth image data, wherein the scale of the fourth image data is equal to the maximum scale; The target image data is obtained by performing logical operations on the third image data and the fourth image data, which have the same maximum scale.

10. The method according to claim 9, characterized in that, The step of expanding the dimensions of the selected third image data to obtain the fourth image data includes: Obtain a second template matrix with a third preset scale, wherein the third preset scale is smaller than the second preset scale; Based on the second template matrix, the selected third image data is subjected to dimensional expansion processing to obtain dimensional expansion third image data, wherein the scale of the dimensional expansion third image data is smaller than the maximum scale; Based on the second template matrix, the dimension-expanded third image data is further expanded until the fourth image data with the same maximum scale is obtained.

11. The method according to any one of claims 1 to 7, characterized in that, After fusing N third image data to obtain target image data with the same scale as the largest scale first image data, the method further includes: Obtain the gradient magnitude data and gradient direction data of the target image data; Based on the gradient magnitude data and the gradient direction data, non-maximum suppression is performed on the gradient magnitude of the target image data to obtain initial edge image data; Based on the obtained second and third thresholds, edge detection is performed on the initial edge image data to obtain target edge image data.

12. An image data processing device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the method according to any one of claims 1 to 11.

13. A storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the method of any one of claims 1 to 11.

Images