Attention area marking method and device based on machine vision, equipment and medium
By extracting key features in the image and generating target significant maps, selecting gaze points to acquire sub-images, combining Gaussian pyramid and central-peripheral difference calculations, the efficiency and accuracy problems of machine vision system during detection of tiny objects in large backgrounds are solved, and the efficiency and accuracy of machine vision system is achieved. More efficient and accurate detection.
Patent Information
- Application Number
- CN202411071683.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-15
AI Technical Summary
Existing machine vision systems have low processing efficiency and poor detection accuracy when detecting small objects in large backgrounds, are susceptible to redundant data interference, and are difficult to meet real-time requirements.
By obtaining prior knowledge of the detection object, extracting the key feature set in the original image, generating a target significant map, selecting the gaze point and obtaining sub-images, combining Gaussian pyramid and central-peripheral difference calculations, the detection accuracy and efficiency are improved.
Improve detection accuracy and efficiency, reduce calculation amount, enhance robustness to image size, rotation and lighting changes, reduce false detection and missed detection rates, and improve user experience.
Smart Images

Figure CN118968025A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to a method, device, equipment and medium for marking an attention area based on machine vision. Background Art
[0002] In many industries such as metallurgy, machinery, textiles, food, packaging, pharmaceuticals, and agricultural, forestry, and animal husbandry product processing, the detection of tiny targets in a large background is a crucial task. Such detection targets usually account for less than 5% of the total detection area, such as cracks and scratches on the surface of cold-rolled strip steel, indentations and scars on the surface of rails, foreign matter in food, defects on the surface of leather, knots and insect eyes on large plates, and foreign fibers in cotton. The detection of these tiny targets is of great significance to ensure product quality and production safety.
[0003] Existing machine vision systems are limited by fixed pixels and preset speeds, and the image resolution and frequency of the collected images are consistent, resulting in excessive background data in the detection of large backgrounds and small targets, and low processing efficiency and detection efficiency. At the same time, the target is too small, which makes detailed information insufficient, difficult to judge, and prone to missed detection and false detection. Although the model based on the attention mechanism takes visual selective attention into account, it does not meet the real-time requirements or has not yet been put into practical application. Summary of the invention
[0004] In view of this, the embodiments of the present invention provide a method, device, equipment and medium for marking an attention area based on machine vision to solve the problems of low processing efficiency, poor detection accuracy and susceptibility to interference from redundant data when detecting small targets in a large background in existing machine vision systems.
[0005] In a first aspect, an embodiment of the present invention provides a method for marking an attention area based on machine vision, the method comprising:
[0006] Obtaining the original image within the preset range where the detection object is located;
[0007] Acquire prior knowledge of the detection object, and extract multiple key feature sets in the original image according to the prior knowledge;
[0008] Generate a plurality of corresponding feature maps based on the key feature set, and fuse the feature maps corresponding to each of the key feature sets to obtain a target saliency map;
[0009] Acquire a detection specification of the detection object, select a fixation point from the target saliency map according to the detection specification, and acquire a plurality of sub-images according to the fixation point;
[0010] It is determined whether the sub-image contains the detection object, and the sub-image containing the detection object is marked as a suspicious area.
[0011] In an optional implementation of the present application, the generating of the corresponding plurality of feature graphs based on the key feature set includes:
[0012] Constructing a Gaussian pyramid based on the key feature set;
[0013] The differences between the levels in the Gaussian pyramid are calculated to generate a plurality of feature maps corresponding to the key features.
[0014] In an optional implementation of the present application, selecting a fixation point from the target saliency map according to the detection specification includes:
[0015] Analyzing the target saliency map to obtain a candidate fixation point sequence, wherein the candidate fixation point sequence includes a plurality of candidate fixation points arranged in descending order of saliency strength;
[0016] Obtaining a preset number of candidate fixation points for the detection object in the detection specification;
[0017] In the candidate fixation point sequence, the preset number of candidate fixation points are selected from the starting position of the candidate fixation point sequence, and the selected preset number of candidate fixation points are used as fixation points.
[0018] In an optional implementation of the present application, analyzing the target saliency map to obtain a candidate fixation point sequence includes:
[0019] Analyzing the target saliency map to determine a plurality of salient regions;
[0020] Identifying extreme salient points in the salient area, and using the extreme salient points as candidate fixation points;
[0021] Detecting the saliency intensity of each of the candidate fixation points;
[0022] The candidate fixation points are sorted in descending order according to the saliency strength to obtain a candidate fixation point sequence.
[0023] In an optional implementation manner of the present application, acquiring a plurality of sub-images according to the gaze point includes:
[0024] Obtaining the detection window parameters set in the detection specification;
[0025] A plurality of detection windows are generated based on the detection window parameters, and a plurality of sub-images are acquired with the gaze point as the center using the detection windows.
[0026] In an optional implementation manner of the present application, the determining whether the sub-image contains the detection object includes:
[0027] Segmenting the sub-image to obtain a plurality of local regions;
[0028] Calculating the mean and variance of the local area;
[0029] Determine whether there is a detection object in the local area based on the mean and variance of the local area; if the sub-image contains at least one local area where the detection object exists, determine that the sub-image contains the detection object, or, if the sub-image does not contain a local area where the detection object exists, determine that the sub-image does not contain the detection object.
[0030] In an optional implementation manner of the present application, determining whether there is a detection object in the local area according to the mean and variance of the local area includes:
[0031] Obtaining the variance threshold and mean threshold set in the prior knowledge;
[0032] Comparing the mean of the local area with the mean threshold, and comparing the variance of the local area with the variance threshold;
[0033] If the mean of the local area is greater than or equal to the mean threshold and the variance is greater than or equal to the variance threshold, it is determined that the detection object exists in the local area; or, if the mean of the local area is less than the mean threshold or the variance is less than the variance threshold, it is determined that the detection object does not exist in the local area.
[0034] In a second aspect, an embodiment of the present invention provides a device for marking an attention area based on machine vision, the device comprising:
[0035] A first acquisition module is used to acquire an original image within a preset range where the detection object is located;
[0036] An extraction module, used to obtain prior knowledge of the detection object and extract a plurality of key feature sets in the original image according to the prior knowledge;
[0037] A fusion module, used to generate a plurality of corresponding feature maps based on the key feature set, and fuse the feature maps corresponding to each of the key feature sets to obtain a target saliency map;
[0038] A second acquisition module is used to acquire a detection specification of the detection object, select a fixation point from the target saliency map according to the detection specification, and acquire the original image according to the fixation point to obtain a plurality of sub-images;
[0039] The marking module is used to determine whether the sub-image contains the detection object, and mark the sub-image containing the detection object as a suspicious area.
[0040] In a third aspect, an embodiment of the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0041] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.
[0042] The method provided in the embodiment of the present application has the following beneficial effects:
[0043] The method provided in the embodiment of the present application uses the prior knowledge of the detection object to extract the key features in the original image, so as to more accurately capture its unique attributes and improve the detection accuracy. By constructing a Gaussian pyramid and calculating the hierarchical differences to generate a feature map, the method is robust to image size, rotation and illumination changes. Extracting the key features first and then constructing the pyramid reduces the amount of calculation and improves the processing speed compared to operating directly on the original image. The feature maps corresponding to each key feature are fused to obtain a more comprehensive and detailed target saliency map, which is helpful for subsequent detection or recognition tasks.
[0044] The method provided in the embodiment of the present application selects a fixation point from a target saliency map, and reuses multiple sub-images obtained from a camera based on these fixation points to obtain a more accurate image with higher image quality, thereby improving detection efficiency. The selection of fixation points is based on the target saliency map and the detection specification, ensuring that the sub-image contains key information of the detection object and improving detection accuracy. The method can flexibly adapt to different detection requirements and is implemented by obtaining parameters (such as the number of candidate fixation points, detection window parameters, etc.) in the detection specification. The process of reacquiring sub-images optimizes resource utilization and focuses on areas containing key information. The method can reduce processing time, improve detection accuracy, and enhance user experience, especially in scenarios requiring rapid response and accurate results. The candidate fixation point sequence is determined by analyzing the target saliency map to ensure that subsequent steps are concentrated in key information areas. The candidate fixation points with the highest saliency are selected as fixation points, further reducing the amount of data while retaining the most critical information. The sub-images are reacquired based on the detection window parameters set in the detection specification to ensure that the sub-images meet the detection requirements and further improve detection accuracy and efficiency.
[0045] The method provided in the embodiment of the present application achieves a detailed analysis of the image content by segmenting the sub-images and calculating the mean and variance of each local area, thereby improving the accuracy of judging whether the detected object exists. Using the mean and variance as the basis for judgment enhances the robustness of the algorithm to different conditions such as lighting and contrast. The sub-image segmentation strategy reduces unnecessary calculations and improves processing efficiency. The adjustable mean and variance threshold settings enable the process to adapt to the detection requirements of different scenarios and improve the versatility of the algorithm. Comprehensively considering the mean and variance of the local area reduces false detections and missed detections, and improves the user experience, especially in scenarios that require fast response and accurate results. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0047] Figure 1 is a flowchart of a method for marking an attention area based on machine vision according to some embodiments of the present invention;
[0048] Figure 2 is a schematic diagram of an image of cotton containing impurities after mean filtering preprocessing according to an embodiment of the present invention;
[0049] Figure 3 is a schematic diagram of an image of cotton containing impurities after median filtering preprocessing according to an embodiment of the present invention;
[0050] Figure 4 is a schematic diagram of a bottom-up and top-down combined gaze model according to an embodiment of the present invention;
[0051] Figure 5 is a schematic diagram of the effect of extracting cotton image edge and impurity features according to an embodiment of the present invention;
[0052] Figure 6 is a schematic diagram of a multi-layer pyramid structure and multi-scale analysis of cotton images according to an embodiment of the present invention;
[0053] Figure 7 is a schematic diagram of the composition and feature analysis of a saliency map of an image of cotton containing impurities according to an embodiment of the present invention;
[0054] Figure 8 is a schematic structural diagram of an irregular imaging machine vision system according to an embodiment of the present invention;
[0055] Fig. 9is a structural block diagram of a device for marking an attention area based on machine vision according to an embodiment of the present invention;
[0056] Fig.10 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0058] According to an embodiment of the present invention, a method, apparatus, device and medium for marking an attention area based on machine vision are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0059] In this embodiment, a method for marking an attention area based on machine vision is provided. Figure 1 is a flow chart of a method for marking an attention area based on machine vision according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:
[0060] Step S11, obtaining an original image within a preset range where the detection object is located.
[0061] In the embodiment of the present application, the machine vision system uses a high-precision main camera to capture clear images within a preset range of the detection object in the detection materials such as cold-rolled steel strip, rails, and food. The detection objects are impurities in the detection materials such as cold-rolled steel strip, rails, and food, such as cracks, scratches, indentations, scars, foreign matter, injuries, knots, worm eyes, foreign fibers, etc. The preset range is set according to specific needs to ensure that the impurities are fully presented. The original image contains impurities, background and noise, and the quality directly affects the detection accuracy. Therefore, it is necessary to fine-tune the parameters of the main camera and control the acquisition environment to obtain high-quality images to meet subsequent processing requirements.
[0062] In the embodiment of the present application, the pre-processed image is subjected to mean filtering and median filtering to reduce noise. Mean filtering smoothes the image background and requires edge processing strategy optimization. Median filtering eliminates impulse noise and keeps edges clear without being affected by extreme values. Figure 2 As shown in Figure 2, mean filtering makes the background of the cotton image containing impurities uniform without damaging the features. Figure 3 As shown in Figure 1, median filtering removes noise while keeping the image clear. Preprocessing significantly reduces noise and improves image quality, providing a reliable data basis for subsequent feature extraction, saliency map calculation, and target detection.
[0063] It should be noted that if Figure 4 As shown, the present invention uses a selective attention computational model to solve the problem of real-time machine vision detection of tiny targets in a large background. The model effectively improves detection efficiency and accuracy by simulating the selective attention mechanism of human vision and combining bottom-up and top-down processing methods. The following is a detailed explanation of each part of the model:
[0064] Bottom-up part: The function is to simulate the rough scanning of the scene by human eyes and build a saliency map to highlight the salient areas in the scene. The implementation method is to build a saliency map based on the Itti model. By calculating the data saliency of each part of the scene, a saliency map is generated to help quickly locate the area that may contain the detection object.
[0065] The top-down part: The function is to use prior knowledge and detection norms to guide the attention process and reduce the attention area irrelevant to the task. The implementation method is to adjust the bottom-up saliency map according to prior knowledge and detection norms to obtain a more accurate attention saliency map.
[0066] Decision-making part: The function is to use the attention saliency map and the original image to determine the real detection target through the detection threshold. The implementation method is to extract the most significant point from the attention saliency map as the candidate fixation point; re-acquire the sub-image with the candidate fixation point as the center; calculate the mean and variance of the sub-image, and determine whether there is a suspicious target through the evaluation function (such as the threshold of the mean and variance).
[0067] Step S12, obtaining prior knowledge of the detection object, and extracting multiple key feature sets in the original image based on the prior knowledge.
[0068] It should be noted that prior knowledge refers to the information about the detection object and its background that is known before the detection task begins. This information is used to effectively identify small targets from complex backgrounds. Prior knowledge can include the size range, color, brightness, shape and other characteristics of the detection object, as well as the difference from the background. For example, in cotton impurity detection, prior knowledge may include the possible color, size range, shape, etc. of the impurities. This prior knowledge can be obtained through expert experience, historical data analysis, domain knowledge base, etc.
[0069] In an embodiment of the present application, after obtaining prior knowledge, it is first necessary to select the features that are most helpful for the detection task based on this knowledge. For example, if the prior knowledge indicates that there is a significant difference in color between the impurity and the background, the color feature is one of the key features. In this embodiment, three features, brightness, color and direction, are mainly considered. The selection of these features is based on the understanding and analysis of the large background and small target detection task in the machine vision system. Secondly, these features need to be extracted from the original image. This usually involves the application of image processing and feature extraction algorithms. For example, brightness features can be extracted by converting color images into grayscale images; color features can be extracted by constructing color channels and opposing color features; and directional features can be extracted by applying algorithms such as Gabor filters. Finally, each extracted feature will form a feature set, which will be used in subsequent saliency map calculations and decision-making processes.
[0070] Specifically, the process of extracting brightness features includes: assuming that the input image is f(x,y), f_R, f_G, f_B are the red, green, and blue components respectively, then the brightness feature f_gray is:
[0071]
[0072] Among them, f_gray is the brightness feature; f_R is the red component; f_G is the green component; f_B is the blue component.
[0073] In addition, the color image can be converted into a grayscale image by taking the mean value. There are also other conversion methods, as shown below:
[0074] f_gray=0.3f_R+0.6f_G+0.1f_B
[0075] Among them, 0.3 is the proportion of the red component; 0.6 is the proportion of the green component; and 0.1 is the proportion of the blue component.
[0076] Specifically, this method is mainly based on the different sensitivities of human eyes to colors. People are more sensitive to green, so it accounts for a larger proportion. This transformation is helpful for human visual perception, but it will not change the quality of the image.
[0077] Specifically, the color features extracted f_R, f_G, f_B are the red, green, and blue components of the input image respectively. Based on the above three components, the grayscale information can be calculated:
[0078]
[0079] Among them, f_R, f_G, f_B are the red, green, and blue components of the input image respectively.
[0080] In order to eliminate the influence of hue, first adjust red, green and blue according to the above formula. The specific formula is as follows:
[0081] If f_gray ≠ 0
[0082] Among them, f_R' corresponds to the color characteristics of red; f_G' corresponds to the color characteristics of green; f_B' corresponds to the color characteristics of blue.
[0083] If f_gray=0, then let f_R'=0, f_G'=0, f_B'=0, and construct the color information of the following four channels, as shown below:
[0084]
[0085] Y=(f_R'+f_G')-2(|f_R'-f_G'|+f_B')
[0086] Among them, R, G, B, and Y represent the color characteristics of red, green, blue, and yellow, respectively. If R, G, B, and Y are less than 0, they are set to 0. The purpose is to reduce the correlation between each color and increase the difference.
[0087] Specifically, after obtaining the color information of the four channels, two opposing color features (red-green, blue-yellow) are constructed:
[0088]
[0089] Among them, rg is the red-green color feature; by is the blue-yellow color feature.
[0090] Specifically, in directional feature extraction, the Gabor filter achieves selective enhancement of information in different directions in the image through its specific kernel function design. The kernel function is essentially a complex sine function modulated by a Gaussian envelope, where the frequency of the sine wave controls the accuracy of image detail extraction, while the phase specifies the direction of feature extraction. By adjusting these parameters, Gabor filters with specific directions can be generated. When these filters are convolved with the brightness component of the image, the image edges and textures that match the filter direction are highlighted, while suppressing information in other directions.
[0091] Specifically, the Gabor filter in the spatial domain is the same as the Gaussian filter. It specifies the kernel function and then convolves with the image to obtain the filtering result:
[0092]
[0093] in:
[0094]
[0095] The above formula is a commonly used Gabor kernel function. The Gabor kernel function can be simply regarded as the result of a Gaussian envelope modulated by a complex sine function. It is actually a rotation formula. The frequency of the sine function can determine the different levels of refinement of the extracted image, and the phase can determine the direction of the extracted features.
[0096] Specifically, the filtering result can be divided into two parts, the real part h_real and the imaginary part h_i:
[0097]
[0098] Where ω represents the frequency of the sine wave, σ x ,σ y The standard deviation of the Gaussian envelope along the x-axis or y-axis. θ is a preset direction parameter. The Gabor filter in the specified direction can be obtained according to the formula of the filtering result.
[0099] After obtaining the Gabor filter of the specified direction, convolve it with the brightness information of the image to obtain the directional feature map of the specified direction. Let h(m,n) be a k×l Gabor filter, and the image brightness I(x,y) after being filtered by h(m,n) is ο(x,y), and the calculation formula is as follows:
[0100]
[0101] Among them, ο(x, y) is the image obtained after filtering; h(m, n) is the Gabor filter; I(x, y) is the brightness information of the image.
[0102] As an example, Figure 5 As shown, Gabor filtering takes σ x =σ y =2.33, a shows the three-dimensional views of multiple Gabor filters with different directional parameters, which intuitively reflects their directionality. b Through the directional test, it is verified that the Gabor filter enhances the edge when the edge direction is consistent with the filtering direction, and other directional information is suppressed. c shows the filtering effect of the Gabor filter on the same edge under different frequency parameters (w changes), reflecting the influence of frequency on the refinement of feature extraction. d is the result of a cotton image after Gabor filtering. It can be seen that there is a clear response in different directions around the impurities, highlighting the directional characteristics of the impurities, which is extremely beneficial for subsequent target detection and recognition.
[0103] Step S13, generating a plurality of corresponding feature maps based on the key feature set, and fusing the feature maps corresponding to each key feature set to obtain a target saliency map.
[0104] In the embodiment of the present application, the extracted key features (such as brightness, color, direction, etc.) are subjected to Gaussian pyramid multi-scale analysis, feature extraction, and central-peripheral difference calculation strategies to generate multiple feature saliency maps. Subsequently, these feature saliency maps are effectively fused using methods such as normalization and weighted summation, and finally a target saliency map that comprehensively reflects the saliency of the original image data is obtained. This process not only highlights the salient areas in the image that are related to the target detection task, but also reduces interference information that is not related to the task, providing strong support for subsequent target recognition and detection.
[0105] Specifically, for the original image f(x,y) with an input size of M×N, its brightness feature set is I(x,y), and its color feature set is The directional feature set is ο(x, y), where each feature consists of L layers of Gaussian pyramids, as shown below:
[0106]
[0107] Among them, each feature set calculates the feature map through the center-surround calculation strategy. The center-surround strategy simulates the receptive field of neuronal cells. This method is very sensitive to the local spatial information of the features and can obtain the local significance of the data.
[0108] The center-periphery calculation calculates the difference between different pyramid levels of the same feature. The central receptive field is represented by features with higher resolution (lower levels in the pyramid), such as c∈{2,3,4}, and the level interval is δ∈{2,3}. The peripheral receptive field is represented by features with lower resolution (higher levels in the pyramid). The peripheral selection level is: s=c+δ. Each feature requires 6 feature maps, and the level correspondence is: 2-4, 2-5, 3-5, 3-6, 4-6, 4-7. When the image between two levels is different in size, the interpolation method is used. Formula (1) represents the central-peripheral calculation formula of brightness feature:
[0109] I(c,s)=|I(c)ΘI(s)|
[0110] Among them, I(x,y) is the brightness feature set; c∈{2,3,4} is the feature with higher resolution; δ∈{2,3} is the level interval.
[0111] The center-periphery calculation method of color features needs to be adjusted. In order to highlight the differences between the color features, it is not calculated directly according to the formula of the filtering result, but calculated according to the following formula:
[0112] rg(c,s)=|(R(c)-G(c))Θ(G(s)-C(s))|
[0113] by(c,s)=|(B(c)-Y(c))Θ(Y(s)-B(s))|
[0114] The center-periphery calculation method of directional features is the same as the center-periphery calculation formula, except that when calculating directional features, usually not only one direction of features is calculated, but also multiple directions of features are calculated, such as 0, Therefore, the calculation formula is as follows:
[0115] Ο(c,s,θ)=|Ο(c,θ)ΘΟ(s,θ)|
[0116] Among them, through the above calculation formula, 6 images with features in multiple directions are obtained.
[0117] Specifically, in order to fuse the six images into a feature map, they need to be normalized. The formula is as follows, assuming the normalization range is [0,τ]:
[0118]
[0119] Among them, 6 feature maps represent the contrast distribution of a certain feature information of the image.
[0120] Specifically, each feature map only represents the contrast distribution at a certain resolution. Now we need the contrast of a certain feature of the entire image. Therefore, we need to fuse these 6 feature maps. The fusion method can be weighted summation, as shown below:
[0121]
[0122] According to the above formula, we can obtain the feature saliency maps ψ(I), Ψ(rg), Ψ(by), and Ψ(Ο). w(i) represents the weight factor, which are usually treated equally, and w(i)=1 is often taken.
[0123] Specifically, the ultimate goal is to construct a saliency map that reflects the saliency of the original image data. Therefore, in order to fuse the above feature saliency maps together, first normalize the feature saliency maps using the following formula:
[0124]
[0125] Among them, w(t) represents the weight factor of a certain feature, numPyr represents the number of pyramid layers, and numFeat represents the number of features.
[0126] Specifically, all feature maps are fused into a saliency map according to the following formula:
[0127]
[0128] Among them, w(i) represents the weight.
[0129] As an example, Figure 7 As shown in the figure, a salient map of an image of cotton with impurities is constructed, where e is the brightness feature salient map, f is the directional feature salient map, and g is the salient map. In the parameter setting, all weights are set to 1 and treated equally. From the salient map, it can be seen that the impurity position data is very significant, and these areas are the suspicious areas to be processed. However, in addition to the impurity area, there are obviously many other significant points. If prior knowledge and detection specifications are not added, a lot of useless information will be generated, causing frequent attention and affecting the efficiency of machine vision.
[0130] In the embodiment of the present application, generating a plurality of corresponding feature graphs based on the key feature set includes the following steps A1-A2:
[0131] Step A1: construct a Gaussian pyramid based on the key feature set.
[0132] In the embodiment of the present application, multi-resolution analysis is used to capture the scale information of key features (such as brightness, color, direction, etc.) in the image. In the specific implementation, firstly, for each key feature set, the image is gradually downsampled using a Gaussian window filter to construct a multi-layer pyramid structure, each layer representing the feature representation of the image at different resolutions. This process provides the necessary scale space foundation for the calculation of the subsequent feature saliency map, which helps to effectively extract and integrate feature information at multiple scales.
[0133] Specifically, the Gaussian pyramid is used to represent the spatial multi-resolution scale information of the image. Assuming the original I0(x, y) is an M×N image, the L-layer Gaussian pyramid algorithm is as follows:
[0134]
[0135] Among them, w(m,n) is a Gaussian window filter. The previous level image is reduced by 4 times compared with the next level image. The sampling method adopts alternate row and alternate column sampling. Combining these images together constitutes a Gaussian pyramid.
[0136] Specifically, a multi-layer pyramid structure, such as Figure 6 As shown in the figure, a is a pyramid diagram, and b is a 5-layer pyramid structure of a cotton image. The first layer is M×N, and the second layer is The third layer is …….An important advantage of multi-scale analysis is that features that are not easy to find at one scale may be easy to find at another scale.
[0137] Step A2, calculate the differences between the levels in the Gaussian pyramid and generate multiple feature maps corresponding to the key features.
[0138] In the embodiment of the present application, the differences between the various levels in the Gaussian pyramid are calculated to simulate the receptive field mechanism of neuronal cells, thereby generating multiple feature maps corresponding to key features such as brightness, color, and direction. This process utilizes the central-peripheral calculation strategy, and by comparing the feature information between different resolution levels (i.e., different pyramid layers), the saliency of the local area is highlighted, providing basic data for subsequent feature integration and saliency map construction.
[0139] Step S14, obtaining a detection specification of the detection object, selecting a fixation point from the target saliency map according to the detection specification, and obtaining a plurality of sub-images according to the fixation point.
[0140] In the embodiments of the present application, the detection specifications are first clarified, including the characteristics of the target size, color, brightness, shape, etc. Taking cotton impurity detection as an example, the specification defines the impurity area range (through the window size) and the color / grayscale threshold. These specifications are formulated based on prior knowledge and actual needs. Then, the saliency map is calculated by combining the bottom-up and top-down methods, and the highlighted area indicates high significance. According to the specification, the first k points with the highest significance values are selected as the fixation points, and k is preset by the specification. A detection window is generated with each fixation point as the center, and the detection object is re-photographed according to the detection window using the camera to obtain a sub-image of a fixed size. The sub-image size is determined by the target size range in the specification to ensure that the target is fully contained.
[0141] In the embodiment of the present application, selecting a fixation point from a target saliency map according to a detection specification includes the following steps B1-B3:
[0142] Step B1, analyzing the target saliency map to obtain a candidate fixation point sequence, wherein the candidate fixation point sequence includes a plurality of candidate fixation points arranged in descending order of saliency strength.
[0143] In an embodiment of the present application, a target saliency map obtained by a bottom-up saliency map calculation process is analyzed, wherein the target saliency map includes the data saliency of each part of the scene to be tested, that is, displays the visually prominent area. In the application of machine vision real-time detection of tiny targets, these visually significant areas contain targets or impurities that need to be detected. Therefore, analyzing the target saliency map can obtain a candidate fixation point sequence, which contains multiple candidate fixation points, and these candidate fixation points are arranged from high to low according to their saliency intensity.
[0144] In the embodiment of the present application, step B1 specifically includes the following steps B11-B14:
[0145] Step B11, analyzing the target salient map and determining multiple salient regions.
[0146] In an embodiment of the present application, first, a saliency map calculated from the bottom-up part is received as input. The saliency map has been processed through multiple steps such as multi-feature extraction, central-peripheral difference calculation, and feature map fusion, reflecting the saliency of each part of the data in the original image. Secondly, on the saliency map, by analyzing the distribution and change of pixel values, those parts that are significantly higher than the surrounding areas are identified. These parts are visually displayed as highlighted or prominent areas, corresponding to the areas in the original image that may contain the detection target. Thirdly, the identified highlighted areas are further determined as multiple specific salient areas. This usually involves image processing techniques such as threshold processing and connected area marking on the saliency map, so as to segment the highlighted parts on the saliency map into independent salient areas. Finally, the information of the determined multiple salient areas is output, including the position (such as the center point coordinates) and size (such as the bounding box size) of each salient area. This information will be used in subsequent steps to guide the machine vision system to further analyze and process the corresponding areas in the original image.
[0147] Step B12, identifying extreme salient points in the salient area, and using the extreme salient points as candidate fixation points.
[0148] In an embodiment of the present application, first, on the saliency map, extreme salient points are identified by local search or global comparison. Extreme salient points refer to points with the highest saliency in a certain neighborhood, and they represent the most prominent areas in the scene. These points can be isolated or form the center of a salient area. Secondly, the identified extreme salient points are used as candidate fixation points. Candidate fixation points are the focus of subsequent target detection and analysis, and they will guide the machine vision system to conduct in-depth analysis of the corresponding areas in the original image. In practical applications, in order to improve detection efficiency and reduce the amount of calculation, the number of candidate fixation points is usually limited according to the detection specification. For example, a maximum number of candidate points k is set according to prior knowledge, and only the top k points with the highest saliency are taken as candidate fixation points. Finally, for each candidate fixation point, its coordinate position in the saliency map (or original image) is recorded. These coordinate information will be used in the step of reacquiring a sub-image from the camera to analyze the sub-image.
[0149] Step B13, detecting the saliency intensity of each candidate fixation point.
[0150] In an embodiment of the present application, for each candidate fixation point, it is necessary to detect its significant intensity in the saliency map. The significant intensity is usually measured by the sum or average of the significant values in a certain area around the point. This area can be a window of fixed size centered on the candidate fixation point. On the saliency map, a window (e.g., 3x3, 5x5, etc.) is drawn with each candidate fixation point as the center, and then the sum or average of the significant values of all pixels in the window is calculated. This sum or average represents the significant intensity of the candidate fixation point. The significant intensity of each candidate fixation point is recorded for use in subsequent steps. These significant intensity values will be used to further screen the candidate fixation points and determine which points are most likely to be the real detection targets.
[0151] Step B14, sorting the candidate fixation points in descending order according to their saliency to obtain a candidate fixation point sequence.
[0152] In an embodiment of the present application, a sorting algorithm (such as quick sort, merge sort, etc.) is used to sort the candidate fixation points in descending order according to the significant intensity value. After sorting, the candidate point with the highest significant intensity will be at the front of the set, and the candidate point with lower significant intensity will be at the back. After the sorting is completed, a candidate fixation point sequence is obtained. The candidate points in this sequence are arranged in order from high to low in terms of significant intensity, which provides a more efficient processing order for subsequent target detection and recognition.
[0153] Step B2, obtaining a preset number of candidate fixation points for the detection object in the detection specification.
[0154] In an embodiment of the present application, it is necessary to obtain a preset value for the number of candidate fixation points from the detection specification. This preset value (denoted as k) is formulated based on prior knowledge and actual application requirements, and is used to guide how to extract a certain number of candidate fixation points from the saliency map for subsequent analysis. This preset number k represents the maximum number of targets expected to exist in the detection scene or the number of areas that need further detailed analysis. Specifically, for different detection tasks, the value of k may be different. For example, in the task of detecting impurities in cotton, a reasonable k value can be pre-set based on factors such as the planting density of cotton, the common distribution of impurities, and the resolution of the detection equipment to ensure that neither important targets are missed nor too many candidate points lead to excessive calculations and reduced detection efficiency.
[0155] Step B3: In the candidate fixation point sequence, a preset number of candidate fixation points are selected from the starting position of the candidate fixation point sequence, and the selected preset number of candidate fixation points are used as fixation points.
[0156] In the embodiment of the present application, a preset number of candidate fixation points are selected from a sequence of candidate fixation points arranged from high to low in terms of significance. Specifically, the first k candidate fixation points with the highest significance in the sequence are selected as fixation points. These selected fixation points will be used for tasks such as sub-image acquisition, feature extraction, and target recognition. Through these processing steps, it is possible to finally determine whether there are detection targets in the scene, as well as the specific locations and attributes of these targets.
[0157] In the embodiment of the present application, obtaining multiple sub-images according to the gaze point includes the following steps C1-C2:
[0158] Step C1, obtaining the detection window parameters set in the detection specification.
[0159] In an embodiment of the present application, it is necessary to read or obtain specific parameters about the detection window from the detection specification. These parameters define the size, shape and other properties of the detection window, which are used for subsequent analysis using sub-images obtained from the camera. For different detection tasks, the parameters of the detection window will be different. For example, in the task of detecting impurities in cotton, the size of the detection window needs to be set according to the expected size of the impurities to ensure that the impurity area can be completely included while avoiding the introduction of excessive background noise. In addition, the shape of the detection window is also adjusted according to the shape characteristics of the target to improve the accuracy of the detection.
[0160] Step C2, generating multiple detection windows based on the detection window parameters, and using the detection windows to obtain multiple sub-images with the gaze point as the center.
[0161] In the embodiment of the present application, an irregular imaging machine vision system is used to obtain multiple sub-images, such as Figure 8 As shown, the irregular imaging machine vision system includes: a PC, a transmission control mechanism, a main camera and a plurality of controlled slave cameras, and the PC is connected to the main camera, the slave camera and the transmission control mechanism respectively. The slave camera can move according to the gaze point. The distance between the main camera and the measured scene is greater than the distance between the slave camera and the measured scene.
[0162] The main camera is used to pre-attention the scene to be tested, take the original image within the preset range where the test object is located, and transmit the original image to the PC. The test object can be impurities in cold-rolled strip steel, rails, food and other test materials.
[0163] The PC is used to receive the original image, obtain the prior knowledge of the detection object, and extract multiple key feature sets in the original image based on the prior knowledge; generate multiple corresponding feature maps based on the key feature sets, and fuse the feature maps corresponding to each key feature set to obtain a target saliency map; obtain the detection specification of the detection object, and select the fixation point from the target saliency map according to the detection specification, and control the image acquisition from the camera according to the fixation point.
[0164] The slave camera is used to obtain multiple sub-images according to the gaze point and transmit the multiple sub-images to the PC. Specifically, the corresponding detection window is generated according to the preset detection window parameters (such as size and shape) with the screened gaze point (i.e., multiple candidate gaze points with the highest significance) as the center. The design of these detection windows takes into account the imaging characteristics of the main camera and the slave camera: the scene captured by the main camera is larger and the target is relatively small, so the detection window needs to adapt to this low resolution situation; while the local information captured by the slave camera has a higher resolution, which provides a basis for subsequent high-precision analysis.
[0165] The PC is used to determine whether the sub-image contains the detection object and mark the sub-image containing the detection object as a suspicious area.
[0166] It should be noted that the original image is a global image taken by the main camera, which includes the entire scene under test, but has a relatively low spatial resolution. The sub-image obtained based on the gaze point is a local image taken by the slave camera, which only contains a part of the scene under test, but has a higher spatial resolution and can provide more accurate and detailed information. This irregular imaging method not only reduces the amount of image data, but also improves the accuracy and computing speed of target detection.
[0167] Specifically, multiple salient regions can be obtained from the salient map. In each salient region, the most salient point is found as the coordinate of this region, so as to obtain a series of salient point coordinates. Here, the first k salient points are taken as candidate fixation points, which are set as:
[0168] Sal_Coor i (x,y),i=1,2,......k, where k is the number of significant points.
[0169] Design a window size a×b, take out a×b sub-image from the original image with Sal_Coor as the center. This sub-image is the object to be analyzed. The window size represents the size range of the detected object. Therefore, the sub-image taken out can be set as I_Sub i ,i=1,2,......k。
[0170] The irregular imaging machine vision system structure provided by the embodiment of the present application has significant advantages. First, it greatly reduces the amount of image data, because the system only needs to process and analyze the local image of the suspicious target area, without processing the entire global image. This not only saves storage space, but also improves processing efficiency. Secondly, since the local image has a higher spatial resolution, the accuracy of target detection is improved. Finally, this structure also makes the operation faster because the amount of image data that needs to be processed is reduced, thereby speeding up the response speed of the system.
[0171] Step S15, determining whether the sub-image contains the detection object, and marking the sub-image containing the detection object as a suspicious area.
[0172] Specifically, first, statistical features (mean, variance, etc.) are calculated for each sub-image, which reveal the difference between the sub-image and the background. Then, these feature values are input into the evaluation function, which is designed based on prior knowledge and detection specifications to evaluate the similarity between the sub-image and the detection object. In this case, the evaluation function compares the mean and variance of the sub-image with the preset threshold. If the feature value meets certain conditions (i.e., exceeds or falls below the threshold), it is considered that the sub-image may contain the detection object. Finally, these sub-images are marked as suspicious areas for subsequent detailed analysis or verification.
[0173] In the embodiment of the present application, determining whether a sub-image contains a detection object includes the following steps D1-D4:
[0174] Step D1, segmenting the sub-image to obtain multiple local areas.
[0175] In the embodiment of the present application, each sub-image extracted from the saliency map is further segmented into multiple smaller local areas, the purpose of which is to analyze the sub-image at a finer granularity level so as to more accurately determine whether the sub-image contains the detection object. The segmentation method may include grid-based uniform segmentation or image content-based adaptive segmentation.
[0176] Step D2, calculating the mean and variance of the local area.
[0177] In an embodiment of the present application, after obtaining multiple local areas of a sub-image, for each local area, its pixel value can be converted into a grayscale value (the sub-image is a color image collected from a camera), and then the mean and variance of the grayscale values of all pixels in the area are calculated. The mean reflects the average level of the pixel values in the local area, while the variance reflects the degree of discreteness of the pixel values in the local area. These two statistics can effectively capture the grayscale distribution characteristics of the local area and provide a basis for subsequent detection and judgment.
[0178] Specifically, the mean and variance of the image can reflect the image information globally.i Divide it into m×n areas, find the mean and variance of each small area, and then compare them to make a judgment.
[0179] Set I_Sub i The color information of the following four channels is constructed by I_Sub i _Ave, I_Sub i _Std, where i=1,2,......k.
[0180] I_Sub i Divided into m×n parts, I_Sub i _Sub j ,j=1,2,......m×n,I_Sub i _Sub j The mean and variance of I_Sub are i _Sub j _Ave, I_Sub i _Sub j _Std.
[0181] Step D3, determining whether there is a detection object in the local area according to the mean and variance of the local area.
[0182] Specifically, to determine whether a local area has impurities, the following evaluation function is used:
[0183]
[0184] Among them, Std_T represents the variance threshold, and Ave_T represents the mean threshold, which is the dividing line between the target and the background and is prior knowledge.
[0185] In the embodiment of the present application, step D3 specifically includes the following steps D31-D33:
[0186] Step D31, obtaining the variance threshold and mean threshold set in prior knowledge.
[0187] In the embodiment of the present application, the preset variance threshold and mean threshold are obtained from prior knowledge, wherein these thresholds are obtained based on the statistical analysis of a large amount of sample data and are used to distinguish the difference in statistical characteristics between the detected object and the background image. The variance threshold is mainly used to measure the degree of change of pixel values in a local area of the image, while the mean threshold is used to measure the average brightness or gray level of a local area of the image.
[0188] Step D32, comparing the mean of the local area with the mean threshold, and comparing the variance of the local area with the variance threshold.
[0189] In an embodiment of the present application, for each local area, its pixel value can be converted into a grayscale value (the sub-image is a color image captured from a camera), and then the mean and variance of the grayscale values of all pixels in the area are calculated. The mean index is used to evaluate the average brightness or grayscale level of the area, and the variance index reflects the degree of change of the pixel values in the area.
[0190] Specifically, by comparing the calculated mean with the preset mean threshold, it is possible to preliminarily determine whether the region may contain the detection object. Next, the variance of the local region is calculated, and the variance is compared with the preset variance threshold to further determine whether the region has sufficient texture or edge information, which are often important signs of the detection object. Through the comparative analysis of these two steps, the local region containing the detection object can be more effectively identified.
[0191] Step D33, if the mean of the local area is greater than or equal to the mean threshold and the variance is greater than or equal to the variance threshold, it is determined that there is a detection object in the local area, or, if the mean of the local area is less than the mean threshold or the variance is less than the variance threshold, it is determined that there is no detection object in the local area.
[0192] In the embodiment of the present application, if the mean of a local area is greater than or equal to the mean threshold, and the variance is greater than or equal to the variance threshold, it is considered that this area is likely to contain the detection object, because the average brightness or grayscale level and the degree of pixel value change of the area meet the preset conditions. On the contrary, if the mean of a local area is less than the mean threshold or the variance is less than the variance threshold, it is considered that this area does not contain the detection object, because the area does not meet the preset conditions in at least one statistical feature. Through such logical judgment, the local area containing the detection object can be accurately screened out from the image.
[0193] Step D4, if the sub-image contains at least one local area where the detection object exists, then determine that the sub-image contains the detection object, or, if the sub-image does not contain a local area where the detection object exists, then determine that the sub-image does not contain the detection object.
[0194] In the embodiment of the present application, after the detection and judgment of all local areas are completed, it is comprehensively determined whether the entire sub-image contains the detection object based on these judgment results. If at least one local area in the sub-image is judged to contain the detection object, it is considered that the sub-image contains the detection object and is marked as a suspicious area. On the contrary, if all local areas are judged to not contain the detection object, it is considered that the sub-image does not contain the detection object and is ignored. This step is a summary and decision-making process of the results of the previous steps.
[0195] In this embodiment, a device for marking an attention area based on machine vision is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0196] This embodiment provides a device for marking attention areas based on machine vision, such as Fig. 9 As shown, including:
[0197] A first acquisition module 91 is used to acquire an original image within a preset range where the detection object is located;
[0198] An extraction module 92, used to obtain prior knowledge of the detection object and extract multiple key feature sets in the original image based on the prior knowledge;
[0199] A fusion module 93 is used to generate a plurality of corresponding feature maps based on the key feature set, and fuse the feature maps corresponding to each key feature set to obtain a target saliency map;
[0200] A second acquisition module 94 is used to acquire a detection specification of the detection object, select a fixation point from the target saliency map according to the detection specification, and acquire a plurality of sub-images according to the fixation point;
[0201] The marking module 95 is used to determine whether the sub-image contains the detection object, and mark the sub-image containing the detection object as a suspicious area.
[0202] In an optional implementation of the present application, the fusion module 93 is used to construct a Gaussian pyramid based on the key feature set; calculate the differences between the levels in the Gaussian pyramid, and generate multiple feature maps corresponding to the key features.
[0203] In an optional embodiment of the present application, the second acquisition module 94 is used to analyze the target saliency map to obtain a candidate fixation point sequence, wherein the candidate fixation point sequence includes multiple candidate fixation points arranged in descending order of saliency intensity; obtain a preset number of candidate fixation points for the detection object in the detection specification; in the candidate fixation point sequence, select a preset number of candidate fixation points from the starting position of the candidate fixation point sequence, and use the selected preset number of candidate fixation points as fixation points.
[0204] In an optional embodiment of the present application, the second acquisition module 94 is used to analyze the target saliency map and determine multiple salient areas; identify extreme salient points in the salient areas and use the extreme salient points as candidate fixation points; detect the saliency intensity of each candidate fixation point; sort the candidate fixation points in descending order according to the saliency intensity to obtain a candidate fixation point sequence.
[0205] In an optional embodiment of the present application, the second acquisition module 94 is used to obtain the detection window parameters set in the detection specification; generate multiple detection windows based on the detection window parameters, and use the detection windows to obtain multiple sub-images centered on the gaze point.
[0206] In an optional embodiment of the present application, the marking module 95 is used to segment the sub-image to obtain multiple local areas; calculate the mean and variance of the local areas; determine whether there is a detection object in the local areas based on the mean and variance of the local areas; if the sub-image contains at least one local area where the detection object exists, it is determined that the sub-image contains the detection object, or, if the sub-image does not contain a local area where the detection object exists, it is determined that the sub-image does not contain the detection object.
[0207] In an optional embodiment of the present application, the marking module 95 is used to obtain the variance threshold and the mean threshold set in the prior knowledge; compare the mean of the local area with the mean threshold, and compare the variance of the local area with the variance threshold; if the mean of the local area is greater than or equal to the mean threshold and the variance is greater than or equal to the variance threshold, it is determined that there is a detection object in the local area, or, if the mean of the local area is less than the mean threshold or the variance is less than the variance threshold, it is determined that there is no detection object in the local area.
[0208] See also Fig.10 , Fig.10 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.10 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).
[0209] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0210] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0211] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0212] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.
[0213] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0214] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0215] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for marking attention areas based on machine vision, characterized in that: The method comprises: Obtaining the original image within the preset range where the detection object is located; Acquire prior knowledge of the detection object, and extract multiple key feature sets in the original image according to the prior knowledge; Generate a plurality of corresponding feature maps based on the key feature set, and fuse the feature maps corresponding to each of the key feature sets to obtain a target saliency map; Acquire a detection specification of the detection object, select a fixation point from the target saliency map according to the detection specification, and acquire a plurality of sub-images according to the fixation point; It is determined whether the sub-image contains the detection object, and the sub-image containing the detection object is marked as a suspicious area.
2. The method according to claim 1, characterized in that The generating a plurality of corresponding feature graphs based on the key feature set comprises: Constructing a Gaussian pyramid based on the key feature set; The differences between the levels in the Gaussian pyramid are calculated to generate a plurality of feature maps corresponding to the key features.
3. The method according to claim 1, characterized in that The selecting a fixation point from the target saliency map according to the detection specification comprises: Analyzing the target saliency map to obtain a candidate fixation point sequence, wherein the candidate fixation point sequence includes a plurality of candidate fixation points arranged in descending order of saliency strength; Obtaining a preset number of candidate fixation points for the detection object in the detection specification; In the candidate fixation point sequence, the preset number of candidate fixation points are selected from the starting position of the candidate fixation point sequence, and the selected preset number of candidate fixation points are used as fixation points.
4. The method according to claim 3, characterized in that The step of analyzing the target saliency map to obtain a candidate fixation point sequence includes: Analyzing the target saliency map to determine a plurality of salient regions; Identifying extreme salient points in the salient area, and using the extreme salient points as candidate fixation points; Detecting the saliency intensity of each of the candidate fixation points; The candidate fixation points are sorted in descending order according to the saliency strength to obtain a candidate fixation point sequence.
5. The method according to claim 1, characterized in that The acquiring a plurality of sub-images according to the gaze point comprises: Obtaining the detection window parameters set in the detection specification; A plurality of detection windows are generated based on the detection window parameters, and a plurality of sub-images are acquired with the gaze point as the center using the detection windows.
6. The method according to claim 1, characterized in that The determining whether the sub-image contains the detection object comprises: Segmenting the sub-image to obtain a plurality of local regions; Calculating the mean and variance of the local area; Determine whether there is a detection object in the local area according to the mean and variance of the local area; If the sub-image contains at least one local area where the detection object exists, it is determined that the sub-image contains the detection object, or, if the sub-image does not contain a local area where the detection object exists, it is determined that the sub-image does not contain the detection object.
7. The method according to claim 6, characterized in that Determining whether there is a detection object in the local area according to the mean and variance of the local area includes: Obtaining the variance threshold and mean threshold set in the prior knowledge; Comparing the mean of the local area with the mean threshold, and comparing the variance of the local area with the variance threshold; If the mean of the local area is greater than or equal to the mean threshold and the variance is greater than or equal to the variance threshold, it is determined that the detection object exists in the local area; or, if the mean of the local area is less than the mean threshold or the variance is less than the variance threshold, it is determined that the detection object does not exist in the local area.
8. A device for marking attention areas based on machine vision, characterized in that: The device comprises: A first acquisition module is used to acquire an original image within a preset range where the detection object is located; An extraction module, used to obtain prior knowledge of the detection object and extract a plurality of key feature sets in the original image according to the prior knowledge; A fusion module, used to generate a plurality of corresponding feature maps based on the key feature set, and fuse the feature maps corresponding to each of the key feature sets to obtain a target saliency map; A second acquisition module is used to acquire a detection specification of the detection object, select a fixation point from the target saliency map according to the detection specification, and acquire a plurality of sub-images according to the fixation point; The marking module is used to determine whether the sub-image contains the detection object, and mark the sub-image containing the detection object as a suspicious area.
9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.
Citation Information
Cited By
Rapid foreign fiber identification method and system based on artificial intelligence
CN120892972A
An artificial intelligence-based rapid foreign fiber identification method and system
CN120892972B
