Image color anomaly detection method and device
By performing channel separation and difference calculation on the image to form a difference matrix, and combining absolute value and statistical measures to judge image color anomalies, the problems of large error and noise influence in the existing technology are solved, and efficient and accurate color anomaly detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG DAHUA TECH CO LTD
- Filing Date
- 2022-12-06
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies suffer from large computational errors and significant noise impact in image color anomaly detection, resulting in low detection efficiency and poor accuracy.
The method of channel separation and difference calculation is adopted. By separating the image into channels, single-channel data is obtained. The gray value deviation between single-channel data is calculated to form a difference matrix. The degree of color difference is determined by absolute value and statistical quantity. Combined with preset threshold, it is judged whether the image is abnormal.
It reduces computational errors and noise, improves detection accuracy and efficiency, and can quickly identify color anomalies in images.
Smart Images
Figure CN115937137B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video image processing technology, and in particular to a method and apparatus for detecting image color anomalies. Background Technology
[0002] With the nationwide rollout of smart cities, the number of various monitoring systems and image sensors in cities exceeds tens of thousands. Video data, as a crucial resource for efficient and intelligent analysis, makes ensuring the quality of collected video data paramount. Errors in camera settings and other factors can cause color anomalies in the monitored images, significantly impacting image quality. Therefore, detecting color anomalies in surveillance video can promptly identify equipment malfunctions, facilitating timely repairs by troubleshooting personnel.
[0003] Currently, existing technologies mainly employ a method of calculating the variance of color channels in blocks on the central processing unit (CPU) for image color anomaly detection. This variance calculation method is prone to large computational errors; furthermore, block-based processing introduces significant noise, impacting image detection.
[0004] In summary, this application provides an image color anomaly detection method that can reduce the impact of noise and effectively reduce calculation errors. Summary of the Invention
[0005] This application provides an image color anomaly detection method and apparatus. The channel separation difference calculation method is relatively simple and reduces calculation error when performing image color anomaly detection calculation. The image to be detected is processed based on global features, which greatly reduces the impact of noise on the algorithm effect.
[0006] In a first aspect, embodiments of this application provide an image color anomaly detection method, including channel separation of the image to be detected to obtain K single-channel data, where any single-channel data is the grayscale value of the image to be detected under the corresponding primary color; K is an integer not less than 2; determining the difference matrix corresponding to the image to be detected based on the grayscale value deviation of each pixel in the image to be detected in at least two different single-channel data; and determining the degree of color difference according to the difference matrix, where the degree of color difference is used to represent the degree of color anomaly in the image to be detected.
[0007] In the above technical solution, the image to be detected is separated into channels to obtain K single-channel data, each of which exists in the form of a matrix. Then, after obtaining the K single-channel data, the single-channel difference matrix is determined by the deviation between pixel values in at least two of the single-channel data, thereby determining the degree of color difference. Compared with existing technologies that determine the degree of color difference through variance, determining it through deviation calculation using two single-channel data is simpler and reduces the impact of calculation errors. It can be seen that the detection method for the image to be detected in this application is based on a holistic calculation operation of the entire image to be detected, that is, processing based on the global features of the image to be detected, which also reduces the impact of noise on the detection of image color anomalies.
[0008] In one possible design, the difference matrix corresponding to the image to be detected is determined based on the gray value deviation of each pixel in the detected image in at least two different single-channel data sets. This includes: grouping any two single-channel data sets from K single-channel data sets into a group to obtain multiple channel groups; determining the gray value deviation of each pixel in any channel group under the channel group; obtaining the gray value deviation matrix of the channel group based on the gray value deviation of each pixel; and determining the difference matrix corresponding to the image to be detected based on the gray value deviation matrices of the multiple channel groups.
[0009] In the above technical solution, the image to be detected is first separated into channels to obtain K single-channel data. Multiple channel groups are then determined by grouping data from each pair of different channels within the K channels. A deviation matrix is determined based on the grayscale value deviation of each pixel within each channel group. Determining the difference matrix based on the grayscale value deviation matrix of the multiple channel groups improves accuracy and ensures the robustness of the image to be detected.
[0010] In one possible design, determining the grayscale value deviation of a pixel in a channel group includes: calculating the difference between the grayscale values of the pixel in the two single-channel data of the channel group to obtain the grayscale value deviation of the pixel; and determining the difference matrix corresponding to the image to be detected based on the grayscale value deviation matrices of multiple channel groups, including: summing the grayscale value deviation matrices of multiple channel groups to obtain the difference matrix corresponding to the image to be detected.
[0011] In the above technical solution, the deviation matrix for determining the grayscale value deviation in each single-channel data is mainly obtained by calculating the difference between the grayscale values of two single-channel data in the channel group. The difference calculation is a simple subtraction operation in mathematics, which is easy to perform. Determining the difference matrix based on multiple deviation matrices is mainly achieved by summing the respective deviation matrices. This summation operation effectively improves the calculation speed of the image to be detected, increasing its efficiency and accuracy, and thus enhancing its robustness.
[0012] In one possible design, after determining the degree of color difference based on the difference matrix, the method further includes: determining that there is a color anomaly in the image to be detected when the degree of color difference is less than a preset threshold.
[0013] In the above technical solution, after determining the difference matrix, the degree of color difference is then determined. The degree of color difference represents the extent of color anomalies in the image to be detected. The principle for determining the degree of color difference is to compare it with a preset threshold. When the degree of color difference is less than the preset threshold, a color anomaly is determined to exist in the image to be detected. This comparison with a preset threshold can quickly determine whether a color anomaly exists in the image to be detected, improving the detection efficiency.
[0014] In one possible design, the degree of color difference is determined based on the difference matrix, including: obtaining the absolute values of all elements in the difference matrix; and determining the degree of color difference based on the statistics of the absolute values of all elements.
[0015] In this process, the statistical measure involves statistically analyzing the pixel values in the obtained difference matrix and comparing the statistically derived data with a preset threshold. In the above technical solution, the degree of color difference is primarily determined through the difference matrix. Specifically, the absolute values of all elements in the difference matrix are first processed. After the absolute value processing, a statistical measure is calculated for each element in the difference matrix; the resulting statistical measure represents the degree of color difference. Using statistical measures and absolute values for calculation is mathematically simple, improving the computational efficiency of the image to be detected. Processing the data through absolute values and statistical measures provides a foundation for subsequent threshold comparisons.
[0016] In one possible design, the image to be detected is an RGB format image converted from a YUV format image; the K single-channel data include at least two of the R single-channel data, G single-channel data, and B single-channel data in the RGB color space.
[0017] In the above technical solution, multiple single channels include at least two of the following in the RGB color space: R single-channel data, G single-channel data, and B single-channel data. This is because the image to be detected is a color image, and all colors in a color image are represented by a mixture of the three primary colors "blue, green, and red." Since the image was originally in YUV format, it needs to be converted to RGB format before channel separation processing. This ensures that the image to be detected can be channel-separated using the separation model for subsequent calculations.
[0018] In one possible design, the method is applied to an image processor; channel separation is performed on the image to be detected to obtain K single-channel data, including: the image processor performs K-channel separation on the image to be detected in parallel using a channel separation model to obtain K single-channel data; based on the gray value deviation of each pixel in the image to be detected in at least two different single-channel data, the difference matrix corresponding to the image to be detected is determined, including: the image processor determines the gray value deviation matrix of each channel group in parallel; each pair of different single-channel data constitutes a channel group; the gray value deviation matrix of any channel group is obtained by the image processor calculating the gray value deviation of N pixels in the image to be detected in parallel.
[0019] In the above technical solution, the image processor is a dedicated processor suitable for image processing (matrix calculation) such as conversion and synthesis. The inherent characteristic of an image processor is its ability to support convenient parallel processing. The image color anomaly detection in this application is implemented using an image processor. Channel separation is not performed channel by channel, but rather multiple channels are separated simultaneously. When calculating the difference, the entire image to be detected is subtracted in parallel based on the corresponding pixel positions in the difference matrix. This parallel processing can effectively improve the computational efficiency of image detection. Furthermore, the K single-channel data obtained after channel separation of the image to be detected are implemented using a channel separation model. This channel separation model is based on deep neural network training, and this separation model based on deep neural network training can adapt to more complex scenarios.
[0020] Secondly, embodiments of this application provide a color anomaly detection device, comprising:
[0021] The separation unit is used to perform channel separation on the image to be detected to obtain K single-channel data, where each single-channel data is the grayscale value of the image to be detected under the corresponding primary color; and K is an integer not less than 2.
[0022] The determining unit is used to determine the difference matrix corresponding to the image to be detected based on the gray value deviation of each pixel in the image to be detected in at least two different single-channel data.
[0023] The determining unit is further configured to determine the degree of color difference based on the difference matrix, wherein the degree of color difference is used to characterize the degree of color anomaly in the image to be detected.
[0024] In one possible design, the determining unit is specifically used to group any two single-channel data from the K single-channel data into a group to obtain multiple channel groups.
[0025] The determining unit is specifically used to determine the grayscale value deviation of each pixel in any channel group under the channel group; and to obtain the grayscale value deviation matrix of the channel group based on the grayscale value deviation of each pixel.
[0026] The determining unit is specifically used to determine the difference matrix corresponding to the image to be detected based on the gray value deviation matrix of multiple channel groups.
[0027] In one possible design, the determining unit is specifically used to perform a difference calculation on the grayscale values corresponding to the pixel in the two single-channel data of the channel group to obtain the grayscale value deviation of the pixel.
[0028] The determining unit is specifically used to perform a summation operation on the gray value deviation matrix of the multiple channel groups to obtain the difference matrix corresponding to the image to be detected.
[0029] In one possible design, the determining unit is further specifically used to determine that the color anomaly exists in the image to be detected when the degree of color difference is less than a preset threshold.
[0030] In one possible design, the determining unit is specifically used to obtain the absolute values of all elements in the difference matrix.
[0031] The determining unit is specifically used to determine the degree of color difference based on a statistical measure of the absolute values of all the elements.
[0032] In one possible design, the image to be detected is an RGB format image converted from a YUV format image; the K single-channel data include at least two of the R single-channel data, G single-channel data, and B single-channel data in the RGB color space.
[0033] In one possible design, the separation unit is specifically used by the image processor to perform K-channel separation of the image to be detected in parallel using a channel separation model, thereby obtaining K single-channel data.
[0034] The determining unit is specifically used by the image processor to determine the gray value deviation matrix of each channel group in parallel; each pair of different single-channel data constitutes a channel group; the gray value deviation matrix of any channel group is obtained by the image processor to calculate the gray value deviation of N pixels in the image to be detected in parallel.
[0035] The beneficial effects of the image color anomaly detection device provided in the second aspect described above can be found in the beneficial effects of the various possible designs of the first aspect, and will not be repeated here.
[0036] Thirdly, embodiments of this application provide a computing device, including: a memory and a processor; the memory is used to store program instructions; the processor is used to invoke the program instructions in the memory to cause the computing device to execute any of the possible designs of the first aspect described above.
[0037] Fourthly, embodiments of this application provide a computationally readable storage medium storing a computer program executable by a computing device, which, when run on the computing device, performs any of the possible designs of the first aspect described above.
[0038] Furthermore, the technical effects of any of the implementation methods in the third to fourth aspects can be found in the technical effects of different implementation methods in the first aspect, and will not be repeated here. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 A system architecture diagram of an image color anomaly detection method provided in this application embodiment;
[0041] Figure 2 This is a schematic diagram of an image color anomaly detection method provided in an embodiment of this application;
[0042] Figure 3 A flowchart illustrating a method for determining a difference matrix provided in this application embodiment;
[0043] Figure 4 This is a schematic diagram of an image color anomaly detection subtraction method provided in an embodiment of this application;
[0044] Figure 5 A schematic diagram of an addition matrix calculation method provided in the application embodiment;
[0045] Figure 6 This application provides a schematic flowchart of a method for obtaining the degree of difference in an embodiment;
[0046] Figure 7 A schematic diagram of a difference matrix is provided for an embodiment of this application;
[0047] Figure 8 This application provides a schematic diagram of a channel separation model for image color anomaly detection.
[0048] Figure 9This application also provides an image color anomaly detection device in its embodiments;
[0049] Figure 10 A computing device is also provided as an embodiment of this application. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0051] Due to factors such as incorrect camera settings, abnormal images may appear in surveillance footage to varying degrees, significantly impacting the quality of the video. Therefore, detecting color anomalies in surveillance video can promptly identify these malfunctions, facilitating timely repairs by troubleshooting personnel.
[0052] Current technologies primarily employ a method of calculating the variance of color channels in blocks on the central processing unit (CPU) for color detection. The CPU, as the core of a computer system's computation and control, is the final execution unit for information processing and program execution. It can be figuratively understood as having 25% ALU (Arithmetic Logic Unit), 25% Control Unit, and 50% Cache. CPU operations involve logical computation, which is relatively slow because the calculation principle involves calculating the first problem, then the second, and so on, with the total time being the sum of the time taken for each problem. Current technologies for image color detection on the CPU divide the image into blocks, meaning the image is divided into different small parts. The pixel values in each small part are read and calculated, and the calculation proceeds from one small part to the next. This results in low computational efficiency and inaccurate results, leading to low robustness. Meanwhile, existing technologies employ image segmentation, which involves dividing the image to be detected into several small parts for processing. After processing, variance is calculated for each image block. In other words, existing technologies use variance calculation during segmentation, which involves squared algorithms and can lead to calculation errors due to the large amount of computation. Furthermore, this segmentation method may introduce noise at each step, resulting in a significant impact from noise.
[0053] This application provides an image color anomaly detection method to solve the above-mentioned problems. (Refer to...) Figure 1 , Figure 1 This application provides an image color anomaly detection system architecture diagram, which includes at least a monitor 101 and a server 102.
[0054] Monitor 101 is used to record surveillance videos in the city and send the images to be detected obtained from monitor 101 to server 102.
[0055] Server 102 processes the image to be detected obtained from monitor 101 and then performs the image color anomaly detection.
[0056] The connection between the monitor 101 and the server 102 can be made via a wireless network or other means, and this application does not impose any restrictions on this.
[0057] In this embodiment, the image to be detected will be transmitted from the monitor 101 to the server 102 for image color anomaly detection.
[0058] The specific process of the image color anomaly method is as follows: Figure 2 , Figure 2 This application provides a schematic diagram of an image color anomaly detection method flow, including:
[0059] S201. Perform channel separation on the image to be detected to obtain K single-channel data, where any single-channel data is the grayscale value of the image to be detected under the corresponding primary color; K is an integer not less than 2.
[0060] The image to be detected is formed using N primary colors, and the grayscale value of each single channel is separated and processed. Channel separation involves extracting the N primary colors from the image to be detected. After channel separation, K single-channel data are obtained. The image to be detected acquired from monitor 101 is subjected to channel separation processing, resulting in K single-channel data.
[0061] Currently, all colors in a color image can be represented by mixing the three primary colors "blue, green, and red," which is why a typical color image is called an RGB image. Channel separation involves separating the three primary colors from the RGB representation.
[0062] It should be understood that after channel separation processing, K single-channel data will be obtained, where K is an integer greater than 2. There is no specific limit to the number of K values. In the example shown in this application embodiment, K=3 corresponds to the three single-channel data of the three primary colors, namely R, G, and B single-channel data; or K=4 corresponds to the four single-channel data of the four primary colors; K can also be equal to other values, which are not specifically limited here.
[0063] S202. Based on the gray value deviation of each pixel in the image to be detected in at least two different single-channel data, determine the difference matrix corresponding to the image to be detected.
[0064] In a computer, a color image can be viewed as a matrix. Each element in the matrix is a pixel. In the previous step S201, K single-channel data were obtained. The grayscale value deviation between at least two of these single-channel data was used to determine the multiple single-channel difference matrices corresponding to the image to be detected.
[0065] like Figure 3 , Figure 3 A flowchart of a method for determining a difference matrix provided in this application embodiment is shown. The specific method includes:
[0066] S301. Combine any two single-channel data points from the K single-channel data points into a group to obtain multiple channel groups.
[0067] The process involves separating the image to be detected into K single-channel data points. Any two single-channel data points are grouped together to obtain multiple channel groups. Determining these multiple channel groups provides a foundation for subsequently determining the difference matrix, thus improving efficiency.
[0068] In one example, the image to be detected is separated into K single-channel data. When K=3, the corresponding single-channel data R, single-channel data G, and single-channel data B are separated. The single-channel data R and single-channel data G are grouped into group 1; the single-channel data G and single-channel data B are grouped into group 2; and the single-channel data B and single-channel data R are grouped into group 3.
[0069] S302. For each pixel in any channel group, determine the gray value deviation of the pixel in the channel group; based on the gray value deviation of each pixel, obtain the gray value deviation matrix of the channel group.
[0070] In a computer, a color image can be viewed as a matrix. Each element in the matrix is a pixel. After obtaining K single-channel data in the previous step S301, the data are grouped in pairs. Based on the grayscale value deviation between two single-channel data sets, multiple deviation matrices corresponding to the image to be detected are determined, and then the deviation matrix is finalized.
[0071] The specific method for determining the grayscale value deviation of a pixel in a channel group is to calculate the difference between the grayscale values of the pixel in the two single-channel data of the channel group to obtain the grayscale value deviation of the pixel.
[0072] In this process, when an anomaly occurs in the image to be detected, the color of the image to be detected after channel separation processing should theoretically be the same across all three single-channel data points. After obtaining K single-channel data points in the previous step S301, the single-channel data points are grouped into pairs, and the grayscale value deviations between the two single-channel data points in each pair are calculated to obtain the deviation matrix of the image to be detected. This difference calculation can effectively distinguish between images in abnormal states and enable rapid judgment.
[0073] In one example, three single-channel data points (R, G, B) are obtained. Any two single-channel data points are subtracted; for example, B and G single-channel data points are used. The difference between the B and G single-channel data points yields a deviation matrix. The difference operation involves subtracting the corresponding pixels in the matrix corresponding to the B single-channel data point from those in the matrix corresponding to the G single-channel data point. Figure 4 This is a schematic diagram of an image color anomaly detection subtraction method provided in an embodiment of this application, as shown below. Figure 4 As shown, the B single-channel data matrix has values of 0.2, 0.3, 0.1, and 0.5, and the G single-channel data matrix has values of 0.4, 0.4, 0.1, and 0.5. The corresponding difference positions are 0.4-0.2; 0.4-0.3; 0.1-0.1; and 0.5-0.5.
[0074] It should be understood that, for the deviation matrix obtained by subtracting any two single-channel data, the two single-channel data can be chosen arbitrarily. They can be the difference between single-channel B data and single-channel G data, or the difference between single-channel B data and single-channel R data, or the difference between single-channel G data and single-channel R data. No specific restrictions are imposed here.
[0075] S303. Determine the difference matrix corresponding to the image to be detected based on the gray value deviation matrix of multiple channel groups.
[0076] After obtaining multiple deviation matrices, these matrices are integrated into a single difference matrix. Specifically, this involves summing the elements of each matrix according to their positions. The difference matrix is obtained by summing the multiple matrices position by position based on the pixel values. The integration of multiple difference matrices is primarily for comparison with a subsequent preset threshold.
[0077] In one example, the pairwise subtraction of the three single-channel data yields three deviation matrices: deviation matrix 1 is obtained by subtracting the R single-channel data from the green G single-channel data; deviation matrix 2 is obtained by subtracting the G single-channel data from the blue B single-channel data; and deviation matrix 3 is obtained by subtracting the R single-channel data from the green B single-channel data. Combining deviation matrices 1, 2, and 3 yields the difference matrix 0. (The rest of the text appears to be unrelated and likely refers to a different example.) Figure 5 As shown, Figure 5 This is a schematic diagram illustrating a summation matrix calculation method provided in an embodiment of the application. For example... Figure 5 As shown, the pixel values in deviation matrix 1 are 0.1, 0.2, 0.3, and 0.4; the pixel values in deviation matrix 2 are -0.1, -0.3, 0.3, and -0.3; the pixel values in deviation matrix 3 are 0, 0.1, -0.6, and -0.1; and the pixel values in the difference matrix 0 obtained after summation are 0, 0, 0, and 0.
[0078] S203. Determine the degree of color difference based on the difference matrix, whereby the degree of color difference is used to indicate the extent of color anomalies in the image to be detected.
[0079] After determining the difference matrix, the degree of color difference is obtained based on it. The degree of color difference is a statistical measure of the difference matrix, and obtaining the degree of color difference provides a basis for subsequent calculations. Based on the degree of color difference, it can be determined whether the image to be detected has color anomalies. Image color anomalies are mostly based on the display of pure colors or a mixture of a limited number of colors, such as pure white, pure black, or monotonous colors.
[0080] After determining the degree of color difference based on the difference matrix, the method further includes: determining that there is a color anomaly in the image to be detected when the degree of color difference is less than a preset threshold.
[0081] Color anomaly refers to an abnormal color condition in the image to be detected. This may result in a black and white image or other color deviation phenomena.
[0082] In a black and white image, the R, G, and B channels are all identical, so the degree of difference can be used to determine whether the image to be detected is black and white. This is achieved by comparing the calculated degree of difference with a preset threshold, typically set to 1. When the degree of difference is less than the preset threshold, the image to be detected is determined to be black and white.
[0083] In one example, the R, G, and B single-channel data in a black and white image are all exactly the same. After subtraction, the difference matrix is obtained, and the degree of difference should also be 0. When the degree of difference = 0 is less than the preset threshold = 1, it can be confirmed that the image to be detected is a black and white image.
[0084] In images exhibiting color deviation, the differences in the R, G, and B channels may be very small. Therefore, the degree of difference can be used to determine whether the image to be detected is a color aberration. This is achieved by comparing the calculated degree of difference with a preset threshold to determine if the image to be detected is a color image.
[0085] In one example, taking a color image as an example, suppose the R, G, and B single-channel data of the color image are different and the differences are large. After subtraction, a difference matrix is obtained, and then the degree of difference is obtained. For example, if the degree of difference is 3, when the degree of difference = 3 is less than the preset threshold = 5, it can be confirmed that the image to be detected is a color abnormal image.
[0086] It should be understood that the preset threshold value is set specifically for different color conditions. No specific restrictions are imposed here.
[0087] like Figure 6 As shown, Figure 6 This application provides a flowchart illustrating a method for obtaining the degree of difference, whereby the specific method for determining the degree of color difference through a difference matrix mainly includes:
[0088] S601. Obtain the absolute value of all elements in the difference matrix.
[0089] The difference matrix may exhibit sign changes in its elements due to channel separation. Since absolute value processing is required for subsequent comparison with the threshold, this is necessary. In other words, after subtracting any two single-channel data points and performing cleaning and filtering operations to obtain the difference matrix, all elements in the difference matrix, i.e., the pixels, are then subjected to absolute value processing.
[0090] In one example, for ease of understanding, such as Figure 7 As shown, Figure 7 A schematic diagram of a difference matrix is provided for an embodiment of this application, such as... Figure 7 As shown in (a), the difference matrix has four elements: -0.2, -0.3, 0.8, and 0.9. After taking the absolute values of all elements in the difference matrix, as shown... Figure 7 As shown in (b), the four elements in the difference matrix become 0.2, 0.3, 0.8, and 0.9.
[0091] It should be understood that the above matrix is only an example to facilitate understanding of the assumptions of step S601, and no specific restrictions are placed on the specific element data in the matrix.
[0092] S602. Determine the degree of difference based on the statistics of the absolute values of all elements.
[0093] In step S601, after calculating the absolute value of the difference matrix, statistical processing is performed on the difference matrix to obtain the degree of difference. The purpose of calculating the statistics is to compare it with a subsequent preset threshold. The preset threshold is a number, and the difference matrix after absolute value processing is still a matrix. A matrix cannot be compared directly with a number, so statistical processing is necessary. The so-called degree of difference is the final result of the statistics of all elements in the difference matrix. A statistic is to transform the integers in a matrix into a number using certain mathematical methods. Here, the statistic can be the mean, that is, the average of the pixel data in the matrix after processing the absolute value of the difference matrix; it can also be the median, which is the middle number selected from the data in the series arranged from smallest to largest, and compared with the preset threshold; it can also be the mode, which is the most frequent number selected from a series of numbers, and compared with the preset threshold; or it can be other forms, which are not specifically limited here.
[0094] Following the above S601, Figure 7 (a) For example, the average value is used to process the difference matrix after the absolute value is processed. That is, the matrix with four elements is summed and then the average is calculated. That is, (0.2+0.3+0.8+0.9)÷4=0.55. The result 0.55 is the degree of color difference.
[0095] In one possible implementation, the image to be detected is an RGB format image converted from a YUV format image; the K single-channel data include at least two of the R single-channel data, G single-channel data, and B single-channel data in the RGB color space.
[0096] In a color image, all colors can be represented by mixing the three primary colors "blue, green, and red," which is why a typical color image is called an RGB image. Channel separation involves extracting the three primary colors from the RGB spectrum. The image to be detected, acquired from monitor 101, undergoes channel separation processing to obtain individual channel data. The image separation processing in this application is also based on this principle, resulting in R, G, and B single-channel data. Initially, the image to be detected is not in RGB format but in YUV format. In this embodiment, RGB image processing is more suitable, but the video image acquired by the camera is in YUV format, so format conversion is necessary when determining whether the image to be detected has color anomalies. After conversion, the size of the image to be detected is also adjusted to facilitate subsequent detection and calculation.
[0097] In one example, the acquired image to be detected is in YUV format. For subsequent channel separation processing, the YUV image is converted to RGB format, and the size of the converted RGB image is set to 32x32. Then, the converted RGB image undergoes channel separation processing to obtain R single-channel data, G single-channel data, and B single-channel data.
[0098] In one possible implementation, the method is applied to an image processor; performing channel separation on the image to be detected to obtain K single-channel data includes: the image processor performing K-channel separation on the image to be detected in parallel using a channel separation model to obtain K single-channel data; determining the difference matrix corresponding to the image to be detected based on the gray value deviation of each pixel in the image to be detected in at least two different single-channel data includes: the image processor determining the gray value deviation matrix of each channel group in parallel; each pair of different single-channel data constitutes a channel group; the gray value deviation matrix of any channel group is obtained by the image processor calculating the gray value deviation of N pixels in the image to be detected in parallel.
[0099] Image processors (IPCs) are dedicated processors used for image processing (matrix calculations) such as transformation and compositing. An inherent characteristic of IPCs is their ability to support convenient parallel processing. Due to these characteristics, during the detection of an image, channel separation is performed in parallel to obtain K single-channel data. When calculating the difference, the entire image under detection is subtracted in parallel based on the corresponding pixel positions in the matrix. This parallel processing effectively improves the computational efficiency of image detection.
[0100] In one example, when the image processor processes the image to be detected, three single-channel data are obtained simultaneously in the channel separation processing part, and in the difference calculation part, the pixels of the entire image to be detected are calculated simultaneously.
[0101] The image processor performs channel separation on the image to be detected by using a channel separation model to obtain individual channel data. Furthermore, the channel separation model is trained using a deep neural network, which makes the model more applicable.
[0102] like Figure 8 , Figure 8 This application provides a schematic diagram of a channel separation model for image color anomaly detection, as shown in the embodiment. Figure 8As shown, taking the difference between any two single-channel data as an example, the overall process of the image to be detected is as follows: the image to be detected is converted into RGB format (Image: 32x32) by the channel separation model to perform channel separation operation; the preprocessed image to be detected is input into the channel separation model to perform channel separation (Channelsplit) to obtain single-channel data (Channel1, Channel2, Channel3); the difference between any two single-channel data (Channel1, Channel3) is performed (Subtraction); the result of the difference is the difference matrix (Diff-score).
[0103] Next, the difference matrix Diff-score is processed by absolute value and average value to obtain the degree of color difference. The degree of color difference is compared with a preset threshold. When the degree of color difference is less than the preset threshold, the image to be detected is judged to be color abnormal.
[0104] Based on the same concept, such as Figure 9 As shown, Figure 9 An embodiment of this application also provides an image color anomaly detection device.
[0105] The separation unit 901 is used to perform channel separation on the image to be detected to obtain K single-channel data, where each single-channel data is the gray value of the image to be detected under the corresponding primary color; and K is an integer not less than 2.
[0106] The determining unit 902 is used to determine the difference matrix corresponding to the image to be detected based on the gray value deviation of each pixel in the image to be detected in at least two different single-channel data.
[0107] The determining unit 902 is further configured to determine the degree of color difference based on the difference matrix, wherein the degree of color difference is used to indicate the degree of color anomaly in the image to be detected.
[0108] In one optional implementation, the determining unit 902 is specifically used to group any two single-channel data from the K single-channel data into a group to obtain multiple channel groups.
[0109] The determining unit 902 is specifically used to determine the grayscale value deviation of each pixel in any channel group under the channel group; and to obtain the grayscale value deviation matrix of the channel group based on the grayscale value deviation of each pixel.
[0110] The determining unit 902 is specifically used to determine the difference matrix corresponding to the image to be detected based on the gray value deviation matrix of multiple channel groups.
[0111] In one optional embodiment, the determining unit 902 is specifically used to perform a difference calculation on the gray values corresponding to the pixel in the two single-channel data of the channel group to obtain the gray value deviation of the pixel.
[0112] The determining unit 902 is specifically used to perform a summation operation on the gray value deviation matrix of the multiple channel groups to obtain the difference matrix corresponding to the image to be detected.
[0113] In one optional embodiment, the determining unit 902 is further configured to determine that the color anomaly exists in the image to be detected when the degree of color difference is less than a preset threshold.
[0114] In one optional implementation, the determining unit 902 is specifically used to obtain the absolute values of all elements in the difference matrix.
[0115] The determining unit 902 is specifically used to determine the degree of color difference based on the statistical measure of the absolute values of all the elements.
[0116] In one optional implementation, the image to be detected is an RGB format image converted from a YUV format image; the K single-channel data include at least two of the R single-channel data, G single-channel data, and B single-channel data in the RGB color space.
[0117] In one optional implementation, the separation unit 901 is specifically used by the image processor to perform K-channel separation on the image to be detected in parallel using a channel separation model, thereby obtaining K single-channel data.
[0118] The determining unit 902 is specifically used by the image processor to determine the gray value deviation matrix of each channel group in parallel; each two different single-channel data constitute a channel group; the gray value deviation matrix of any channel group is obtained by the image processor to calculate the gray value deviation of N pixels in the image to be detected in parallel.
[0119] Based on the same technical concept, embodiments of the present invention also provide a computing device, such as... Figure 10 As shown, it includes at least one processor 1001 and a memory 1002 connected to at least one processor. In this embodiment of the invention, the specific connection medium between the processor 1001 and the memory 1002 is not limited. Figure 10 Taking the connection between processor 1001 and memory 1002 via a bus as an example. The bus can be divided into address bus, data bus, control bus, etc.
[0120] In this embodiment of the invention, the memory 1002 stores instructions that can be executed by at least one processor 1001. By executing the instructions stored in the memory 1002, at least one processor 1001 can perform the steps included in the aforementioned image color anomaly detection method.
[0121] The processor 1001 is the control center of the computing device, and can connect to various parts of the computing device using various interfaces and lines. It performs data processing by running or executing instructions stored in the memory 1002 and accessing data stored in the memory 1002. Optionally, the processor 1001 may include one or more processing units. The processor 1001 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles issuing instructions. It is understood that the modem processor may not be integrated into the processor 1001. In some embodiments, the processor 1001 and the memory 1002 may be implemented on the same chip; in some embodiments, they may be implemented on separate chips.
[0122] Processor 1001 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the image color anomaly detection processing method can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.
[0123] Memory 1002, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1002 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 1002 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In embodiments of the present invention, memory 1002 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.
[0124] Based on the same technical concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program executable by a computing device. When the computer program is run on the computer-readable storage medium, the computer-readable storage medium performs the steps of the above-described image color anomaly detection processing method.
[0125] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0126] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0128] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0129] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for detecting color anomalies in images, characterized in that, include: The image to be detected is subjected to channel separation to obtain K single-channel data, where each single-channel data is the grayscale value of the image to be detected under the corresponding primary color; K is an integer not less than 2. Based on the gray value deviation of each pixel in the image to be detected in at least two different single-channel data, the difference matrix corresponding to the image to be detected is determined. The degree of color difference is determined based on the difference matrix, and the degree of color difference is used to indicate the extent of color anomalies in the image to be detected; The step of determining the difference matrix corresponding to the image to be detected based on the grayscale value deviation of each pixel in the image to be detected in at least two different single-channel data includes: Group any two single-channel data points from the K single-channel data points to obtain multiple channel groups; For each pixel in any channel group, determine the grayscale value deviation of the pixel in the channel group; based on the grayscale value deviation of each pixel, obtain the grayscale value deviation matrix of the channel group. The difference matrix corresponding to the image to be detected is determined based on the gray value deviation matrix of multiple channel groups; Determining the grayscale value deviation of the pixel in the channel group includes: The grayscale value deviation of the pixel is obtained by calculating the difference between the grayscale values corresponding to the two single-channel data of the channel group. Based on the grayscale value deviation matrix of multiple channel groups, the difference matrix corresponding to the image to be detected is determined, including: The grayscale deviation matrices of the multiple channel groups are summed to obtain the difference matrix corresponding to the image to be detected.
2. The method as described in claim 1, characterized in that, After determining the degree of color difference based on the difference matrix, the method further includes: When the degree of color difference is less than a preset threshold, it is determined that there is a color anomaly in the image to be detected.
3. The method as described in claim 1, characterized in that, The degree of color difference is determined based on the difference matrix, including: Obtain the absolute value of all elements in the difference matrix; The degree of color difference is determined based on a statistical measure of the absolute values of all the elements.
4. The method as described in claim 1, characterized in that, include: The image to be detected is an RGB format image converted from a YUV format image; The K single-channel data include at least two of the R single-channel data, G single-channel data, and B single-channel data in the RGB color space.
5. The method according to any one of claims 1 to 4, characterized in that, The method is applied to an image processor; The image to be detected is subjected to channel separation to obtain K single-channel data, including: The image processor performs K-channel separation on the image to be detected in parallel using a channel separation model, thereby obtaining K single-channel data. Based on the grayscale value deviation of each pixel in the image to be detected in at least two different single-channel data, a difference matrix corresponding to the image to be detected is determined, including: The image processor determines the grayscale deviation matrix of each channel group in parallel; each pair of different single-channel data constitutes a channel group. The grayscale deviation matrix of any channel group is obtained by the image processor calculating the grayscale deviation of N pixels in the image to be detected in parallel.
6. An image color anomaly detection device, characterized in that, include: The separation unit is used to perform channel separation on the image to be detected to obtain K single-channel data, where each single-channel data is the grayscale value of the image to be detected under the corresponding primary color; and K is an integer not less than 2. The determining unit is used to determine the difference matrix corresponding to the image to be detected based on the gray value deviation of each pixel in the image to be detected in at least two different single-channel data. The determining unit is further configured to determine the degree of color difference based on the difference matrix, wherein the degree of color difference is used to characterize the degree of color anomaly in the image to be detected; The determining unit is specifically used to group any two single-channel data from the K single-channel data into a group to obtain multiple channel groups; and for each pixel in any channel group, determine the grayscale value deviation of the pixel in the channel group. Based on the gray value deviation of each pixel, the gray value deviation matrix of the channel group is obtained; based on the gray value deviation matrices of multiple channel groups, the difference matrix corresponding to the image to be detected is determined. The determining unit is specifically used to calculate the difference between the gray values corresponding to the pixel in the two single-channel data of the channel group to obtain the gray value deviation of the pixel; and to sum the gray value deviation matrices of the multiple channel groups to obtain the difference matrix corresponding to the image to be detected.
7. A computing device, characterized in that, It includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform any one of the methods described in 1 to 5.
8. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computing device, which, when run on the computing device, causes the computing device to perform the method as described in any one of claims 1 to 5.