A color coding method based on vision

Abstract

The present invention relates to the field of television image technology, and in particular to a color encoding method based on vision, including extracting frame images according to video segments; dividing the pixels of a single frame image into color zones using the RGB values ​​and grayscale values ​​of the pixels, and deconstructing the pixels of a single frame image according to the frequency of occurrence of the RGB values ​​of the pixels; dividing the RGB values ​​of the pixels of continuous frame images into subnetworks, and merging the subnetworks; performing intersection and union operations on the merged subnetworks to complete the deconstruction of the pixels of continuous frame images; and accumulating the fourth weight value and the fifth weight value of the RGB value of the pixel to obtain the total weight value of the RGB value of each pixel. The present invention reconstructs the WHV color encoding method based on the color distribution characteristics of existing images, so that the number of redundancies in color encoding is reduced from tens of millions to millions in actual use, and the original colors can still be restored, the image compression ratio is increased, and the image compression efficiency and transmission speed are improved.

Description

Technical Field

[0001] The present invention relates to the field of television image technology, and in particular to a color coding method based on vision. Background Art

[0002] Color coding includes RGB coding, YUV, NV21, etc., which can be used in technical fields such as television image generation and image compression.

[0003] However, existing color coding defines many color IDs. Taking the RGB color system as an example, each color is composed of different intensities of the three primary colors of red, green, and blue. The intensity of each primary color can be represented by an integer between 0 and 255, and 0 to 1023 for 10-bit color. Therefore, the number of color IDs that can be represented by the RGB color system is 256 cubed, that is, 16,777,216 different colors, and 107,374,1824 colors for 10-bit color. The use of linear generation of 16,777,216 different colors during image generation and image compression seriously affects image transmission, display, and compression efficiency. Summary of the Invention

[0004] In response to the shortcomings of existing methods, the present invention reconstructs the WHV color encoding method based on the color distribution characteristics of existing images, so that color encoding eliminates color redundancy in a single-frame or single-shot data structure when actually used. Tens of millions only need to use millions of levels, and the original colors can still be restored, thereby increasing the image compression ratio, improving image compression efficiency and transmission speed.

[0005] The technical solution adopted by the present invention is: a color coding method based on vision includes the following steps:

[0006] Step 1: Collect different clips of the standard video and extract frame images according to the video segments;

[0007] Step 2: Use the RGB value and grayscale value of the pixel point to divide the color area of ​​the single frame image, and deconstruct the pixel points of the single frame image according to the frequency of occurrence of the RGB value of the pixel point;

[0008] As a preferred embodiment of the present invention, step 2 specifically includes:

[0009] Step 21: Convert the RGB value of the pixel into a grayscale value, and divide the pixel into different color areas according to the RGB value and the grayscale value, and assign a first weight value to each color area to obtain a first weight matrix R 1 ;

[0010] As a preferred embodiment of the present invention, the color area includes: a color level area, a hue area, a color level and hue overlap area, and a non-hue mixed color area.

[0011] As a preferred embodiment of the present invention, the formula for the first weight value is:

[0012]

[0013] Among them, W1 is the first weight value of the non-hue and mixed color area, W2 is the first weight value of the color level area, W3 is the first weight value of the color level area, W4 is the first weight value of the color level and color level overlap area, R I is the red component value of the I-th pixel, G I is the green component value of the I-th pixel, B I is the blue component value of the I-th pixel, RGB I Grayscale value of RGB value.

[0014] Step 22: In R 1 The second weight matrix R is obtained according to the descending order of the RGB value of the pixel point 2 The first column of 2 The parameters in the first column include: the RGB value of the pixel and the corresponding second weight value W 1 , W 1 =W*a, a is the RGB value of the pixel in R 1 The frequency of occurrence in , W is the first weight value;

[0015] Step 23, with R 2 The RGB value of the pixel in the first column is the reference. 1 Find in R 2 The RGB values ​​of the first column of pixels are the same, and the pixel with the highest frequency of right adjacent pixels is the R 2 The second column of R 2 The mth column of 2 The parameters of the second to m columns include: the RGB value of the pixel and the corresponding third weight W 2 , W 2 =W*b, b is R 2 The RGB value of the pixel in the mth column is based on R 1 The frequency of occurrence of the RGB value of the pixel to the right;

[0016] Step 24, R 2 The first correction coefficients c2 to c3 are set for the second to mth column pixel points. m ;

[0017] Step 25, with R 2 The RGB value of the pixel in the first column is used as the reference to search in the upper right direction. 2 Is there any content in the second to mth columns that matches R 2The RGB values ​​of the pixels in the first column of the image are the RGB values ​​of the pixels adjacent to the right. If they exist, then W 1 and c m Multiplied as the fourth weight value W 3 and with R 2 The RGB values ​​of the pixels in the first column together construct the third weight matrix R 3 .

[0018] As a preferred embodiment of the present invention, the method further includes: a third weight matrix R of a plurality of frames of images 3 The fourth weight value W of the RGB value of the pixel point in 3 Perform cumulative summation.

[0019] Step 3: Divide the pixels of the continuous frame images into sub-networks, merge the sub-networks, and perform intersection and union operations on the merged sub-networks to complete the deconstruction of the pixels of the continuous frame images;

[0020] As a preferred embodiment of the present invention, step three specifically includes:

[0021] Step 31: Divide the pixels in the 6 frames into several sub-networks according to the set window size and step size;

[0022] Step 32: perform a union operation on the first subnetwork of the first and second frames to obtain the RGB value of the pixel and the corresponding frequency sum matrix A. (1,2),1 ; And so on, we get the RGB values ​​of the pixels of the first subnet of the 3rd and 4th frames and the 5th and 6th frames and the corresponding frequency sum matrix A (3,4),1 、A (5,6),1 ;

[0023] Step 33, with A (1,2),1 As a benchmark with A (3,4),1 Perform intersection and merge operations ;

[0024] Step 34, then with A (5,6),1 Perform intersection and union operations to obtain the matrix ;

[0025] Step 35: A (1,2),1 、A (3,4),1 、A (5,6),1 Perform the union operation to obtain the matrix ;

[0026] Step 36: Calculate C 总,1 The weight of the RGB value of the pixel in , is the matrix C 总,1 The frequency of the RGB values ​​of the pixels in is the matrix All总,1 The frequency of the RGB values ​​of the pixels in is the matrix C 总,1 The weight correction factor of

[0027] Step 37: According to steps 31 to 36, the weights of the RGB values ​​of the pixels of all sub-grids of the 6 frames are solved, and the RGB values ​​of the pixels of the 6 frames are aggregated and summed according to the weights; and so on, the fifth weight value corresponding to the RGB value of each pixel of all sub-grids of the 6 frames is obtained. The fifth weight value , where p is the p-th subgrid of the 6-frame image, and P is the total number of subgrids in each frame image.

[0028] Step 4: Add the fourth weight value and the fifth weight value of the pixel RGB value to obtain the total weight value of the RGB value of each pixel;

[0029] As a preferred embodiment of the present invention, the method further includes: step 5, constructing a plurality of first color containers, placing the RGB values ​​of different pixels and the corresponding total weight values ​​into different color containers, and obtaining a plurality of corresponding second color containers according to the descending order of the total weight values.

[0030] As a preferred embodiment of the present invention, step five specifically includes:

[0031] Step 51: construct a plurality of first color containers;

[0032] As a preferred embodiment of the present invention, the formula of the first color container is:

[0033]

[0034] Among them, O1 is the pixel color area where the color changes from black to gray; O2 is the pixel color area where the color changes from blue to purple; O3 is the pixel color area where the color changes from dark green to light green; O4 is the pixel color area where the color changes from dark cyan to light cyan; O5 is the pixel color area where the color changes from dark red to light red; O6 is the pixel color area where the color changes from purple to red; O7 is the pixel color area where the color changes from brown to yellow; O8 is the pixel color area where the color changes from gray to white; R I is the red component value of the I-th pixel, G I is the green component value of the I-th pixel, B I is the blue component value of the I-th pixel.

[0035] Step 52: Import the RGB value of each pixel and the corresponding total weight value into the corresponding first color container according to the RGB value of the pixel;

[0036] Step 53: sort the first color containers in descending order according to the sum of the total weights of the RGB values ​​of the pixels to obtain a second color container;

[0037] Step 54: Import the RGB values ​​of the pixels that have not appeared into the second color container to obtain a third color container.

[0038] As a preferred embodiment of the present invention, the method further includes: Step 6: constructing a WHV coordinate system, and using the WHV coordinate system to coordinate map the RGB values ​​of the pixels in the third color container.

[0039] As a preferred embodiment of the present invention, step six specifically includes:

[0040] Step 61: With the base point coordinates as the origin, construct a WHV coordinate system in which W, H, and V are perpendicular to each other; wherein W, H, and V correspond to the length, width, and height of the base point coordinates;

[0041] Step 62: Specify 128^3 coordinates for a plurality of third color containers to generate corresponding coordinate containers;

[0042] Step 63: Divide the coordinate container into a commonly used inner color area and an uncommon outer color area according to the RGB value of the pixel point in each coordinate container.

[0043] As a preferred embodiment of the present invention, the formulas for the commonly used inner color area and the uncommon outer color area are:

[0044]

[0045] Where W, H, and V are the length, width, and height of the corresponding base point coordinates; c is the base point coordinate; and q is the coordinate container number.

[0046] As a preferred embodiment of the present invention, the method further includes: Step 7: generating a corresponding color ID for the coordinate container in the WHV coordinate system 。

[0047] Beneficial effects of the present invention:

[0048] 1. The present invention uses a single-frame processing method to convert the pixel color weights according to the visual characteristics and strip them from the image to form a primary and secondary color probability;

[0049] 2. The present invention decomposes the color motion relationship of the pixels in the previous and next 6 frames, extracts the most effective color and assigns a value, calculates the probability of image motion from the color perspective, and can also avoid the interference of edge flicker on the final sampling value through subtraction processing;

[0050] 3. The WHV coordinate system proposed by this invention and the design of different color containers are used to store more than two million colors in commonly used color areas, greatly improving the efficiency of looking up color codes;

[0051] 4. The color encoding method proposed in this invention aims to provide efficient and low-cost video processing solutions for fields such as television images and network video transmission. Compared with traditional YUV and NV12 color encoding, WHV color encoding has significant advantages in color reproduction, compression efficiency and processing speed, and provides a basis for higher data compression rates and lossless image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of the vision-based color coding method of the present invention;

[0053] Figure 2 This is a flowchart of pixel deconstruction of a single-frame image according to the present invention;

[0054] Figure 3 is a schematic diagram of obtaining a third weight matrix through the first correction coefficient and the second weight matrix of the present invention;

[0055] Figure 4 It is a schematic diagram of a continuous 6-frame sub-network of the present invention;

[0056] Figure 5 is a schematic diagram of a first color container of the present invention;

[0057] Figure 6 It is a schematic diagram of the WHV coordinate system of the present invention;

[0058] Figure 7 This is a schematic diagram of the process of generating a coordinate container from a second color container of the present invention;

[0059] Figure 8 Schematic diagram of the positions of the coordinate containers Pr1 and Pr2 in the WHV coordinate system of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore only shows the components related to the present invention.

[0061] like Figure 1 As shown, a vision-based color coding method includes the following steps:

[0062] Step 1: Collect different clips of the standard video and extract frame images according to the video segments;

[0063] Among them, standard videos can be derived from digital originals, TV series, movies, self-media videos, etc.; for example, standard videos are divided into different segments such as plot part, action part, scene part, etc., and each segment is intercepted with about 10 video segments, and each segment is about 3-5 seconds. Several frames of images are obtained according to the frame rate. At a frame rate of 25fps, a 5-second video has 125 frames.

[0064] like Figure 2 As shown, the RGB value and gray value of the pixel point are used to divide the pixel points of a single frame image into color areas, and the pixel points of a single frame image are deconstructed according to the frequency of occurrence of the RGB value of the pixel point;

[0065] Step 21: Decompose each frame of the image into a number of pixels, obtain the RGB value of each pixel, and convert the RGB value into a grayscale value; divide all pixels of a frame of the image into different color areas according to the RGB value and the grayscale value of the pixel;

[0066] Assume that the RGB value of a pixel in a frame image is (R I ,G I ,B I ); where I is the pixel number;

[0067] For example, if the image resolution is 3840*2160, there are 3840 horizontal pixels and 2160 vertical pixels, for a total of 8294400 pixels, then 1≤I≤8294400.

[0068] Different color areas use brightness and grayscale values ​​to separate pixels into different tones. The color areas include: level area, hue area, level and hue overlap area, and non-hue noise area.

[0069] Separate the brightness and hue of each frame of the video; assign different weights to different color areas;

[0070] The formula for converting the RGB value of a pixel into a grayscale value is:

[0071]

[0072] Among them, R I is the red component value of the I-th pixel, G I is the green component value of the I-th pixel, B I is the blue component value of the I-th pixel, RGB I is the grayscale value of RGB value, the RGB value of pixel = (R I , G I 、B I ).

[0073] The calculation formula for the weight values ​​of different color areas is:

[0074]

[0075] Among them, W is the first weight value; W1 is the first weight value of the first color zone, and the first color zone is the non-hue and mixed color zone; W2 is the first weight value of the second color zone, and the second color zone is the color scale zone; W3 is the first weight value of the third color zone, and the third color zone is the color hue zone; W4 is the first weight value of the fourth color zone, and the fourth color zone is the area where the color scale and color hue overlap; in this embodiment, W1=1, W2=1.129, W3=1.245, and W4=1.374.

[0076] According to the RGB value of the pixel point and the first weight value, the first weight matrix R of each pixel point in each color area of ​​a frame image is obtained. 1 ; R 1 Each parameter includes the RGB value of the pixel and the first weight value W;

[0077]

[0078] Among them, RW ij is the first weight matrix R 1 The ijth pixel parameter in , I = i * j; for example, RW 11 R 1 The pixel parameters in row 1 and column 1 include RGB values ​​(8,8,8) and W2=1.129, then RW ij It is: (8,8,8), 1.129.

[0079] Table 1: R 1 data

[0080]

[0081] Step 22: Calculate R 1 The frequency of occurrence of the RGB value of each pixel in the , and in descending order to obtain the second weight matrix R 2 The first column of parameters; where R 2 The first column parameter is RW 1 1 to RW 1 k ; The second weight matrix R 2 Each parameter in the first column includes the RGB value of the pixel and the second weight value W 1 , W 1 =W*a, a is the RGB value of the pixel in R 1 The frequency of occurrence in is defined as the first frequency;

[0082] For example, for an image with a resolution of 3840*2160, the maximum value of k is 8294400; 1The first row and first column in the image are (8,8,8), and the first frequency a of 1.129 is 10,000, which is the highest frequency among all the RGB values ​​of pixels. Then R 2 The first row of the first column parameter RW 1 1 is (8,8,8), 1.129*10000; (8,8,6), 1.129 appears the first frequency a is 9000 times, the second most frequent occurrence, then RW 1 2 is (8,8,6), 1.129*9000; and so on to get RW 1 k ; Table 2 shows the first column is RW 1 1 example.

[0083] The second weight matrix R 2 The formula is:

[0084]

[0085] Step 23, with R 2 The pixels in the first column of parameters are the reference points. 1 Find in R 2 The pixel point with the most RGB value to the right of the first column of the parameters is defined as the second most frequent pixel point, and is used as the R 2 The second column parameter of R 2 The mth column parameter of R 2 Each parameter from the second column to the mth column includes the RGB value of the pixel point and the third weight value W 2 , W 2 =W*b, b is R 2 The pixel point in the mth column is the reference point in R 1 The frequency of occurrence of the pixel right adjacent to the pixel point, that is, the second frequency;

[0086] As shown in Table 2, R 2 The first row of the first column of RW 1 The RGB value of 1 pixel is (8,8,8). In Table 1, find the adjacent pixel to the right of (8,8,8) and calculate the RGB value of the pixel with the second highest frequency. For example, the second frequency b of (8,8,6) is 5000 times, which is the highest. Then R 2 The first row of the second column of RW 2 1 is (8,8,6), 1.129*5000; similarly, find R 2 The second row of the second column of RW 2 2 to the kth row of the second column RW 2 k , and so on, we get R 2 The mth column of .

[0087] Table 2: R 2 data

[0088]

[0089] Step 24, R 2 The pixels in the second column of all parameters are set with the same first correction coefficient c2, i.e. RW 2 1 to RW 2 k The first correction coefficient of is c2; and so on, for R 2 The same first correction coefficient c is set for all pixels in the mth column m ;

[0090] Step 25, with R 2 The pixels in the first column are used as the reference to search for pixels in the second to mth columns in the upper right direction. When there are consecutive pixels in the same upper right direction as the pixels in the first column, R 2 The second weight value W of all parameters in the first column 1 Multiply by the first correction coefficient to get the third weight matrix R 3 ; R 3 Each parameter in the first column includes the RGB value of the pixel and the fourth weight value W 3 ; The fourth weight value W 3 =R 2 The second weight value W of the first column 1 Or the fourth weight value W 3 =R 2 The second weight value W of the first column 1 *c2 or the fourth weight value W 3 =R 2 The second weight value W of the first column 1 *c2*c3 or the fourth weight value W 3 =R 2 The second weight value W of the first column 1 *c2*c3*c4 or the fourth weight value W 3 =R 2 The second weight value W of the first column 1 *c2*c3*c4...c m ;

[0091] The third weight matrix R 3 The formula is:

[0092]

[0093] like Figure 3 As shown, the direction of the arrow in the figure is R2 The upper right direction of the first column parameter; R 2 The first column and second row pixel is (0,0,3), the second column and first row pixel is (0,0,3), then RW 11 2 Fourth weight value W 3 =RW 1 The second weight value W of 2 1 *c2; and R 2 The first column and third row and the first row of the third column are both (0,0,0), which is different from the second column and second row of pixels (0,0,1). They are not in the same continuous upper right direction, so RW 11 3 Fourth weight value W 3 =RW 1 The second weight value W of 2 1 ; Another example is R 2 The pixels in the first column and fifth row, the second column and fourth row, the third column and third row, and the fourth column and second row are all (0,0,1), so RW 11 5 Fourth weight value W 3 =RW 1 The second weight value W of 2 1 *c2*c3*c4.

[0094] Taking Table 2 as an example, we can get R in Table 3 3 As shown below:

[0095] Table 3: R 3 data

[0096]

[0097] In Table 2, the pixel values ​​of the 4th column of the 1st row, the 3rd column of the 2nd row, the 3rd column of the 3rd row, and the 1st column of the 4th row are all (150, 150, 150), so RW 11 The fourth weight value of 4 = W3*a*c1*c2*c3, RW 11 The fourth weight value of 3 = W3*a, RW 11 The fourth weight value of 2 = W2*a*c2, RW 11 The corrected weight value of 1 = W2*a.

[0098] R 3 The RGB values ​​of the pixels in the image are sorted in descending order according to the fourth weight value. When there are several frames of images, the RGB values ​​of the pixels in the image are sorted in descending order according to the fourth weight value. 3 The fourth weight value of the RGB value of a pixel in is accumulated and summed, and then sorted in descending order.

[0099] For example, the matrix R obtained from the first frame image 3The fourth weight value of the RGB value (8,8,8) of the pixel in the middle is 1.129*10000; the matrix R obtained by the second frame image 3 The corrected weight value of (8,8,8) in the image is 1.129*9000, so the fourth weight value of (8,8,8) in the first and second frames is 1.129*10000+1.129*9000.

[0100] During filming and production, images such as movies and television are centered around key people, objects, and scenery, resulting in a clear primary-secondary relationship and color bias. Existing methods use too broad a color distribution when performing tonal separation, even including less visually noticeable color areas. The present invention, based on visual characteristics, converts regular patterns into pixel color weights, stripping them from the image to form a color probability with primary and secondary colors.

[0101] Step 3: Divide the pixels of the continuous frame images into sub-networks, merge the sub-networks, and perform intersection and union operations on the merged sub-networks to deconstruct the relationship between the pixels of the continuous frame images;

[0102] Step 31: First, perform frame segmentation on the video; take the 1st to 6th frame images, divide the pixels of each frame image into several sub-grids according to the n*n window size and step length n, and set the first sub-grid of the first frame as A 1,1 , the first subgrid of the second frame is A 2,1 , and so on, let the first grid of the 6th frame be A 6,1 ;

[0103] like Figure 4 As shown, taking the first frame of a 4K 3840*2160 resolution as an example: a sub-network with 1280 horizontal and 720 vertical 3*3 windows with a step size of 3 is formed, that is, the first frame has a total of A 1,(1280*720) A 3*3 sub-network.

[0104] Step 32: Sub-grid A of the first frame 1,1 The RGB value of the pixel in the second frame is the same as the first sub-grid A 2,1 The RGB values ​​of the pixels in the matrix are combined and the RGB values ​​of the pixels are summed according to the frequency of occurrence to obtain the matrix A. (1,2),1 ; The first sub-grid A of the third frame 3,1 The RGB value of the pixel in the 4th frame is the same as the 1st sub-grid A 4,1 The RGB values ​​of the pixels in the matrix are combined and the RGB values ​​of the pixels are summed according to the frequency of occurrence to obtain the matrix A. (3,4),1 ; The first sub-grid A of the 5th frame 5,1 The RGB value of the pixel in the middle is the same as the 1st subgrid A in the 6th frame 6,1The RGB values ​​of the pixels in the matrix are combined and the RGB values ​​of the pixels are summed according to the frequency of occurrence to obtain the matrix A. (5,6),1 ;

[0105] For example: , ,but .

[0106] Step 33: Matrix A (1,2),1 As the benchmark and matrix A (3,4),1 Perform intersection operation, the RGB values ​​of different pixels are not retained; then add (3,4),1 Perform a union operation to sum the frequencies of the RGB values ​​of the same pixel points, and retain the RGB values ​​of different pixels;

[0107] For example: , ,but .

[0108] Step 34, and then with matrix A (5,6),1 Perform intersection and union operations to obtain matrix C 总,1 ;Right now , C 总,1 Includes the RGB value of each pixel and the corresponding frequency;

[0109] Step 35: Matrix A (1,2),1 , matrix A (3,4),1 , matrix A (5,6),1 Perform the union operation to get the matrix All 总,1 ;Right now , All 总,1 The RGB value of the pixel and the corresponding frequency are included;

[0110] Step 36: Matrix C 总,1 Medium frequency and matrix All 总,1 Divide the medium frequency by the number of times and add the weight correction factor to get the matrix C 总,1 The weight of the RGB value of the pixel in ; that is, ;in, is the matrix C 总,1 The frequency of the RGB values ​​of the pixels in is the matrix All 总,1 The frequency of the RGB values ​​of the pixels in is the matrix C 总,1 The weight correction factor of

[0111] Step 37: Perform the same operation on the next subgrid of the 6-frame image according to steps 31 to 36 to obtain the matrix C 总,2 The weight corresponding to the RGB value of each pixel in ; Similarly, the fifth weight value corresponding to the RGB value of each pixel point of all sub-grids in 6 frames is obtained. The fifth weight value , where p is the p-th subgrid of the 6-frame image, and P is the total number of subgrids in each frame image;

[0112] Execute steps 31 to 38 for the next 6 frames of images to obtain the average value of the RGB values ​​of each pixel and the corresponding fifth weight value in two consecutive 6-frame images; and so on to obtain the average value of the RGB values ​​of each pixel and the corresponding fifth weight value in several consecutive 6-frame images.

[0113] For example: the fifth weight value W corresponding to the RGB value (0.0.0) of the pixel point in the first 6 frames of image is obtained 5 =0.5, the fifth weight value W corresponding to the RGB value (0.0.0) of the pixel point is obtained in the second 6-frame image 5 =0.3, then the weight of the RGB value (0.0.0) of the pixel point obtained from two consecutive word grids of two consecutive 6-frame images is 0.4.

[0114] The present invention decomposes the color motion relationship of the pixels in the previous and next 6 frames, extracts the most effective color for assignment, calculates the probability of image motion from the color perspective, and can avoid the interference of edge flicker on the final sampling value through subtraction processing; at the current stage, H264 and H265 perform 4-frame edge reconstruction on the image, which will lead to a large amount of calculation and incomplete decoding of highly complex motion images, thereby causing image distortion. After a large number of samples, the multi-directional sampling values ​​have a clear color range, and the extremely small range of the concentrated light and dark areas is biased towards the yellow area and the green-blue gradation area, with a total value of only 631220 colors; the colors required for subsequent video processing can be greatly reduced within this range; that is, the 6-frame color processing in the lens visual processing process is more suitable for the processing of motion images than the 4-frame edge reconstruction.

[0115] Step 4: Add the fourth weight value and the fifth weight value of the RGB values ​​of the pixel points in step 2 and step 3 to obtain the total weight value of the RGB values ​​of each pixel point;

[0116] For example, if the fourth weight of the pixel's RGB value (0.0.0) is 10,000, and the fifth weight of the pixel's RGB value (0.0.0) is 20,000, the total weight of the pixel's RGB value (0.0.0) is 30,000.

[0117] Step 5: Construct several first color containers, put the RGB values ​​of different pixels and the corresponding total weight values ​​into different color containers, and obtain several corresponding second color containers according to the descending order of the total weight values;

[0118] Step 51: In this embodiment, eight first color containers are constructed, and each of the first color containers has a certain color cast.

[0119] The construction formulas for the 8 first color containers are as follows:

[0120]

[0121] Among them, O1 is the pixel color area where the color changes from black to gray; O2 is the pixel color area where the color changes from blue to purple; O3 is the pixel color area where the color changes from dark green to light green; O4 is the pixel color area where the color changes from dark cyan to light cyan; O5 is the pixel color area where the color changes from dark red to light red; O6 is the pixel color area where the color changes from purple to red; O7 is the pixel color area where the color changes from brown to yellow; O8 is the pixel color area where the color changes from gray to white; R I is the red component value of the I-th pixel, G I is the green component value of the I-th pixel, B I is the blue component value of the I-th pixel.

[0122] like Figure 5 As shown, there are eight first color containers constructed using formula (6).

[0123] Step 52: Import the total weight value of the RGB value of each pixel into the corresponding 8 first color containers;

[0124] For example, the RGB value of the pixel is (0,0,10), and the total weight value is 10,000 times filled into the first color container O1;

[0125] Step 53: Reorder the eight first color containers from high to low based on the sum of the total weight values ​​of the RGB values ​​of the pixels to obtain eight second color containers. 1 1 to 0 1 8;

[0126] For example, 1 1 to 0 1 After reordering, the order corresponding to the first color container is O4, O5, O1, O8, O7, O2, O6, and O3, that is, the sum of the total weight values ​​of the RGB values ​​of all pixels in the first color container O4 is the highest.

[0127] Step 54: Import the RGB values ​​of pixels that have not appeared into the second color container O 1 1 to 0 1 8. Get the third color container O 2 1 to 0 2 8;

[0128] The RGB values ​​of pixels that have never appeared have a total weight of 0. Most colors in the 3600 hours of videos actually sampled have never appeared, so the RGB values ​​of pixels that have never appeared are placed in the third color container.

[0129] Step 6: Construct a WHV coordinate system, and use the WHV coordinate system to perform coordinate mapping on the RGB values ​​of the pixels in the third color container;

[0130] like Figure 6 , Step 61, WHV coordinate system schematic diagram, W, H, V correspond to the length, width, and height of the base point coordinates; W, H, V are perpendicular to each other;

[0131] like Figure 7 Step 62: For the third color container O 2 1 to 0 2 8. Perform 128^3 coordinate specification to generate eight coordinate containers from Pr1 to Pr8;

[0132] The base point coordinates c of Pr1 to Pr8 are (0,0,0), (0,0,1), (0,1,0), (0,1,1), (1,0,0), (1,0,1), (1,1,0), (1,1,1); the 128^3 coordinate specification means that the maximum value of W, H, and V is 128.

[0133] Step 63: Divide the RGB values ​​of the pixels in the coordinate container Prq into a commonly used inner color area and an uncommon outer color area;

[0134] like Figure 8 Taking Pr1 and Pr2 as examples, Pr1 is divided into the commonly used inner color area De1 and the very outer color area De9; Pr2 is divided into the commonly used inner color area De2 and the very outer color area De10. Pr1 to Pr2 are cube structures with exactly the same structure, De1 and De2 are also cube structures with exactly the same structure, the space of De9 is the remaining space after subtracting De1 from the cube Pr1, and De10 is the remaining space after subtracting De2 from the cube Pr2; only Pr1 and Pr2 are illustrated in the figure, and the positions of Pr3 to Pr8 are determined according to the coordinates of the base point. Finally, Pr1 to Pr8 form a large cube.

[0135] The commonly used inner color area and outer color area formulas of the coordinate container Prq are:

[0136]

[0137] Where W, H, V are the length, width, and height of the corresponding base point coordinates; c is the base point coordinate; q is the coordinate container number, 1= <q<=8。

[0138] The 64^3*8=2097152 colors in the commonly used inner color area are all included in the WHV coordinate values ​​of De1 to De8, and the WHV coordinate values ​​of De9 to De16 include the remaining unused color area, that is, 128^3*8-64^3*8=14680064 colors.

[0139] Step 64: Place the third color container O 2 1 to 0 2 The RGB values ​​of the pixels in 8 are mapped one-to-one to Pr1 to Pr8 according to the corresponding total weight values;

[0140] For example: 2 The highest total weight value of the RGB value of the pixel in 1 is (10,10,10), 10,000 times, then the RGB value (10,10,10) is mapped to the De1 of Pr1 WHV The position of (0,0,0); RGB value (10,10,9), 9 thousand times the total weight value is O 2 1 is ranked 3rd, then the RGB value (10,10,9) is mapped to the WHV (0,0,3) position of De1 of Pr1, and so on. 2 All pixels in 1 are mapped to De1 of Pr1; when De1 mapping is completed, they are mapped to De9 in descending order of total weight values ​​until De9 is mapped.

[0141] Similarly, O 2 The RGB values ​​of the pixels in 2 are mapped to De2 and De9 of Pr2 according to the total weight value until O 2 All 8 are mapped to Pr8.

[0142] Step 7: Generate a corresponding color ID for the coordinate container Prq in the WHV coordinate system;

[0143] The color IDs can be numbered sequentially according to the coordinates in Prq, or different color IDs can be assigned to different Prqs.

[0144] For example, the first WHV coordinate (0,0,0) in Pr1 is assigned a color ID of 0, and the last WHV coordinate (128,128,128) in Pr8 is assigned a color ID of 16777216.

[0145] Alternatively, the first WHV coordinate (0,0,0) in Pr1 is assigned a color ID of 1-0, and the last WHV coordinate (128,128,128) in Pr1 is assigned a color ID of 1-2097152; the first WHV coordinate (0,0,0) in Pr2 is assigned a color ID of 2-0, and the last WHV coordinate (128,128,128) in Pr2 is assigned a color ID of 2-2097152.

[0146] With the above-described preferred embodiments of the present invention as inspiration, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical scope of this invention. The technical scope of this invention is not limited to the contents of the specification and must be determined according to the scope of the claims.

Claims

1. A color coding method based on vision, characterized in that: The following steps are involved: Step 1: Collect different clips of the standard video and extract frame images according to the video segments; Step 2: Use the RGB value and gray value of the pixel point to divide the pixel points of the single frame image into color areas, and deconstruct the pixel points of the single frame image according to the frequency of occurrence of the RGB value of the pixel point; Step 2 specifically includes: Step 21: Convert the RGB value of the pixel point to a gray value, divide the pixel point into different color areas according to the RGB value and the gray value, and assign a first weight value to each color area to obtain a first weight matrix R 1 ; Step 22: In R 1 The second weight matrix R is obtained according to the descending order of the frequency of occurrence of the RGB values ​​of the pixels. 2 The first column of 2 The parameters in the first column include: the RGB value of the pixel and the corresponding second weight value W 1 , W 1 =W*a, a is the RGB value of the pixel in R 1 The frequency of occurrence in , W is the first weight value; Step 23, with R 2 The RGB value of the pixel in the first column is the reference. 1 Find in R 2 The first column of pixels have the same RGB value, and the pixel with the highest frequency of right adjacent pixels is taken as R 2 The second column of R 2 The mth column of 2 The parameters of the second to m columns include: the RGB value of the pixel and the corresponding third weight W 2 , W 2 =W*b, b is R 2 The RGB value of the pixel in the mth column is based on R 1 The frequency of occurrence of the RGB value of the pixel to the right; Step 24: R 2 The first correction coefficients c2 to c3 are set corresponding to the pixel points in the second to m columns. m ; Step 25, with R 2 The RGB value of the pixel in the first column is used as the reference to search in the upper right direction. 2 Is there any R in the second to mth columns? 2 The RGB values ​​of the pixels in the first column of the RGB value of the pixel adjacent to the right in sequence, if there is, then W 1 and c m Multiply as the fourth weight value W 3 , and with R 2 The RGB values ​​of the pixels in the first column together construct the third weight matrix R 3 ; Step 3: Divide the pixels of the continuous frame images into sub-grids, merge the sub-grids, and perform intersection and union operations on the merged sub-grids to complete the deconstruction of the pixels of the continuous frame images; Step three specifically includes: Step 31, dividing the pixels in the 6 frames of images into a number of sub-grids according to the set window size and step size; Step 32: perform a union operation on the first subgrid of the first and second frames to obtain the RGB value of the pixel and the corresponding frequency sum matrix A. (1,2),1 ; And so on, the RGB values ​​of the pixels of the first sub-grid of the 3rd and 4th frames and the 5th and 6th frames and the corresponding frequency sum matrix A are obtained (3,4),1 , A (5,6),1 ; Step 33: A (1,2),1 As a benchmark with A (3,4),1 Perform intersection and merge operations ; Step 34, and then (5,6),1 Perform intersection and union operations to obtain the matrix ; Step 35: A (1,2),1 , A (3,4),1 , A (5,6),1 Perform a union operation to obtain the matrix ; Step 36: Calculate C 总,1 The weight of the RGB value of the pixel in, , is the matrix C 总,1 The frequency of the RGB values ​​of the pixels in is the matrix All 总,1 The frequency of the RGB values ​​of the pixels in is the matrix C 总,1 The weight correction factor of Step 37, according to steps 31 to 36, the weights of the RGB values ​​of the pixels of all sub-grids of the 6 frames of images are solved, and the RGB values ​​of the pixels of the 6 frames of images are aggregated and summed according to the weights; and so on, the fifth weight value corresponding to the RGB value of each pixel of all sub-grids of the 6 frames is obtained. The fifth weight value , where p is the pth subgrid of the 6-frame image, and P is the total number of subgrids in each frame image; Step 4: Add the fourth weight value and the fifth weight value of the RGB value of the pixel point to obtain the total weight value of the RGB value of each pixel point; Step 5: construct several first color containers, put the RGB values ​​of different pixels and the corresponding total weight values ​​into different color containers, and obtain several corresponding second color containers in descending order according to the total weight values; Step 5 specifically includes: Step 51, constructing a plurality of first color containers; The formula for the first color container is: ； Among them, O1 is the pixel color area where the color changes from black to gray; O2 is the pixel color area where the color changes from blue to purple; O3 is the pixel color area where the color changes from dark green to light green; O4 is the pixel color area where the color changes from dark cyan to light cyan; O5 is the pixel color area where the color changes from dark red to light red; O6 is the pixel color area where the color changes from purple to red; O7 is the pixel color area where the color changes from brown to yellow; O8 is the pixel color area where the color changes from gray to white; R I is the red component value of the I-th pixel, G I is the green component value of the I-th pixel, B I is the blue component value of the I-th pixel; Step 52: Import the RGB value of each pixel and the corresponding total weight value into the corresponding first color container according to the RGB value of the pixel; Step 53: sort the first color containers in descending order according to the sum of the total weight values ​​of the RGB values ​​of the pixels to obtain a second color container; Step 54, importing the RGB values ​​of the pixels that have not appeared into the second color container to obtain a third color container; Step 6: construct a WHV coordinate system, and use the WHV coordinate system to perform coordinate mapping on the RGB values ​​of the pixels in the third color container; Step 6 specifically includes: Step 61: With the base point coordinates as the origin, construct a WHV coordinate system in which W, H, and V are perpendicular to each other; wherein W, H, and V correspond to the length, width, and height of the base point coordinates; Step 62, specify 128^3 coordinates for a plurality of third color containers to generate corresponding coordinate containers; Step 63, dividing the coordinate container into a commonly used inner color area and an uncommonly used outer color area according to the RGB value of the pixel point in each coordinate container; The formulas for the commonly used inner color area and the uncommon outer color area are: ； Where W, H, V are the length, width, and height of the corresponding base point coordinates; c is the base point coordinate; q is the coordinate container serial number; Step 7: Generate a corresponding color ID for the coordinate container in the WHV coordinate system.

2. The vision-based color coding method according to claim 1, characterized in that: The formula for the first weight value is: ； Wherein, W1 is the first weight value of the non-hue and mixed color area, W2 is the first weight value of the color scale area, W3 is the first weight value of the color scale area, W4 is the first weight value of the color scale and color overlap area, R I is the red component value of the I-th pixel, G I is the green component value of the I-th pixel, B I is the blue component value of the I-th pixel, RGB I Grayscale value of RGB value.

Images