Multi-mode target detection method and system based on machine vision

Abstract

The invention discloses a multi-mode target detection method and system based on machine vision, and relates to the technical field of machine vision, and the method comprises the steps: obtaining a real-time image through an image collection device, and carrying out the graying processing; carrying out binarization processing on the image, generating a region of interest through edge recognition, and obtaining an initial attention field; and then, a color block screening technology is adopted, virtual energy is generated, dynamic adaptation is performed on the initial attention field, and an optimized attention field is obtained. And in image recognition, extracting geometric information of the target by using the final attention field and performing target recognition. The method solves the problems of single mode, low precision, weak anti-interference capability, poor illumination adaptability and the like of the existing method, and has the advantages of high precision, multi-mode adaptability, strong anti-interference capability and illumination adaptability. The method has high flexibility and expandability, can be widely applied to the fields of industry, medical treatment, security and protection and the like, and improves the accuracy and efficiency of target detection.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a multi-modal target detection method and system based on machine vision. Background Technology

[0002] In the current field of machine vision, object detection and size measurement methods are widely used in various industries, such as industrial production lines, medical image analysis, intelligent security, and autonomous driving. However, existing object detection systems mostly employ high-performance industrial PCs or industrial computers, combined with complex vision libraries for image processing. These methods typically require precise camera intrinsic and extrinsic parameter calibration, have complex algorithmic flows, consume significant computational resources, and result in high overall system costs, making them difficult to deploy effectively in cost-sensitive or space-constrained embedded scenarios.

[0003] In existing technologies, the detection of different targets or tasks often requires redesigning and deploying software logic for each task, resulting in poor system flexibility and insufficient adaptability. Traditional methods have low distance calculation accuracy; many methods calculate distance based on only a single dimension of the target, ignoring the impact of target angular deformation on the calculation, leading to measurement errors. Regarding interference resistance, existing methods have weak ability to filter background noise, easily misidentifying non-target areas as detection objects, especially in environments with large lighting variations, where target recognition accuracy drops significantly. Furthermore, in traditional image processing methods, the binarization threshold is usually set to a fixed value, failing to automatically adapt to changes in ambient lighting. This can easily lead to target extraction failure under conditions of large fluctuations in light intensity. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a multi-mode target detection method based on machine vision to solve the problems of single mode, low distance calculation accuracy, weak anti-interference ability, and poor light adaptability of existing target detection methods.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a multi-mode target detection method based on machine vision, which includes acquiring a real-time image of the detection area using an image acquisition device and converting the image to grayscale.

[0008] After binarizing the grayscale image, the region of interest is generated based on edge recognition, thus obtaining the initial attention field;

[0009] By filtering color blocks in a grayscale image, virtual energy is generated in the initial attention field, and the initial attention field is dynamically adapted using the virtual energy to obtain the final attention field.

[0010] In the algorithm model used for image recognition, the final attention field is used to perform image recognition and extract the geometric information of the target to be tested, so as to obtain the recognition result of the target to be tested.

[0011] Based on the human-computer interaction interface, the measurement mode is switched; based on the preset geometric information type in each measurement mode, the detection result is output for each modality.

[0012] As a preferred embodiment of the multi-mode target detection method based on machine vision described in this invention, the binarization process includes converting the grayscale values ​​of the image into a black-and-white binary image.

[0013] In the black-and-white binary image, edge information is detected.

[0014] Based on the edge information, an approximation algorithm is used to connect the discontinuous edges in the image; contours with predetermined geometric rules are identified before and after the connection, and two regions of interest are generated based on the two identification results.

[0015] The region of interest is generated by dividing the region according to the geometric features on both sides of the contour, and by generating attention coefficients on both sides of each edge contour at the same scale through a pre-trained graph neural network.

[0016] For each consecutive contour, according to the approximation algorithm, the similarity before and after edge connection is used as the fusion weight for each consecutive contour;

[0017] The fusion weight at each contour is applied to the attention coefficients on both sides of the edge contour. The distribution of attention coefficients in the image without edge connection is used as the initial value. The distribution of attention coefficients after edge connection is weighted using the fusion weight, and then added to the initial value according to the pixel correspondence between the two images to obtain the attention coefficient distribution in the image.

[0018] After normalizing the distribution of attention coefficients in the figure, the initial attention field is obtained.

[0019] As a preferred embodiment of the multi-modal target detection method based on machine vision described in this invention, the graph neural network includes: making nodes represent elements that conform to geometric rules in the graph, edges represent the relationships between nodes, and enabling the graph neural network to propagate information between nodes through graph convolution operations, thereby completing the task of relationship inference, and generating attention coefficients based on the inference results.

[0020] As a preferred embodiment of the multi-mode target detection method based on machine vision described in this invention, the color block screening includes, in a grayscale image, taking consecutive pixels with the same grayscale as a color block;

[0021] A virtual energy is set for each color block, and the difference in grayscale value between adjacent color blocks is calculated based on the grayscale value of each color block.

[0022] The difference is used as an energy response between adjacent color blocks to represent the importance of the contours of adjacent locations for target detection.

[0023] As a preferred embodiment of the multi-modal target detection method based on machine vision described in this invention, the dynamic adaptation includes converting the difference between color blocks into a ratio of attention between adjacent color blocks through a preset linear transformation function.

[0024] The initial attention field is mapped onto a grayscale image according to the pixel correspondence, so that the color blocks are matched with the attention distribution in the initial attention field;

[0025] Considering that the recognition accuracy of the binary image used during the generation of attention distribution is lower than that of the grayscale image, setting the attention in the color block to be consistent ensures that the attention weight in the color block remains consistent throughout the adaptation process;

[0026] The proportional value is introduced between the color blocks to proportionally attenuate the attention distribution.

[0027] Specifically, during attenuation, attenuation occurs within color blocks in the direction of decreasing attention within the graphic; furthermore, attenuation occurs between different color blocks within large areas of equal attention, and also in adjacent areas where attention differs.

[0028] In a region with equal attention levels, the minimum attention value of the color block is used as the basis for attention decay between regions. Attention decay is applied to adjacent regions according to their original attention weight. After attention decay is completed in adjacent regions, attention decay is applied between different color blocks within the region based on the attention decayed in adjacent regions.

[0029] Following the attenuation method, the direction in which attention increases in the graphic between different color blocks is enhanced within the color blocks, proceeding synchronously with the attenuation process, thereby achieving synchronization of different attention cores;

[0030] After attenuation and enhancement are completed, the attention of all color blocks is normalized to obtain the final attention field;

[0031] In the final attention field, the portion with an attention weight less than a preset value is removed to obtain the observation pixel region of the target to be tested.

[0032] As a preferred embodiment of the multi-modal target detection method based on machine vision described in this invention, the observed pixel region of the target to be tested and the final attention field are input into an algorithm model for image recognition to perform image recognition.

[0033] The algorithm model for image recognition is a pre-trained neural network that extracts high-level features from images through multi-layer convolution operations and performs target classification and geometric information extraction based on these high-level features; the feature maps output by the convolutional layers are converted into target classification labels and geometric parameters through fully connected layers.

[0034] The training process of the neural network uses target-labeled data in the image for supervised learning to minimize image classification and regression errors, thereby ensuring the accuracy of the recognition results.

[0035] As a preferred embodiment of the machine vision-based multi-mode target detection method of the present invention, the measurement mode is the output selection of target geometric parameters and classification labels by pre-setting.

[0036] Secondly, the present invention provides a multi-mode target detection system based on machine vision, including an acquisition unit that uses an image acquisition device to acquire real-time images of the detection area and perform image grayscale conversion.

[0037] The analysis unit performs binarization on the grayscale image and generates a region of interest based on edge recognition, thereby obtaining an initial attention field.

[0038] The attention unit performs color block filtering in the grayscale image, generates virtual energy in the initial attention field, and dynamically adapts the initial attention field through the virtual energy to obtain the final attention field.

[0039] The recognition unit, in the algorithm model for image recognition, uses the final attention field to perform image recognition and extracts the geometric information of the target to be tested, thereby obtaining the recognition result of the target to be tested;

[0040] The interactive unit switches measurement modes according to the human-computer interaction interface; and outputs the detection results for each mode according to the preset geometric information type in each measurement mode.

[0041] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein the computer program, when executed by the processor, implements any step of the machine vision-based multi-modal target detection method as described in the first aspect of the present invention.

[0042] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the machine vision-based multi-modal target detection method as described in the first aspect of the present invention.

[0043] The beneficial effects of this invention are as follows: By combining image grayscale conversion, edge recognition, color block selection, and graph neural networks, this invention effectively solves the problems of low accuracy, poor flexibility, weak anti-interference ability, and poor lighting adaptability in existing target detection methods. First, through a multi-mode target detection method, the system can adapt to targets of different shapes and task requirements, avoiding the cumbersome steps of redeploying software logic and greatly improving the system's flexibility. Second, by considering target angle changes and multi-dimensional information, this invention significantly improves distance calculation accuracy, solving the error problem of traditional methods that estimate distance based on a single dimension. Furthermore, the system effectively suppresses background noise and enhances anti-interference ability through color block selection and virtual energy allocation, maintaining a high recognition accuracy even in complex lighting environments. Finally, dynamically adjusting the binarization threshold allows the system to automatically adapt to changes in ambient lighting, avoiding the poor lighting adaptability problem caused by fixed thresholds and improving the stability of target detection in environments with large fluctuations in light intensity. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a machine vision-based multimodal object detection method. Detailed Implementation

[0046] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0049] Reference Figure 1 This is one embodiment of the present invention, which provides a multi-modal target detection method based on machine vision, including the following steps:

[0050] S1: Using image acquisition equipment, the detection area is acquired in real time, and the image is converted to grayscale.

[0051] First, an image acquisition device (such as an industrial camera or webcam) is used to acquire real-time images of the detection area. The image acquisition device acquires images of the target area at a set resolution and frame rate and transmits them to the image processing unit. The acquired images are usually color images, but to simplify subsequent processing and improve computational efficiency, they need to be converted to grayscale. The grayscale process converts the color information of each pixel into grayscale values, typically based on a weighted average of red, green, and blue, resulting in a single-channel grayscale image. The grayscale image not only significantly reduces the amount of data but also simplifies subsequent calculations because the color information is no longer needed in image processing, while the grayscale image retains sufficient brightness and structural features. This process improves image processing efficiency and enhances the system's stability under different lighting conditions, avoiding interference from color changes in target recognition, thus making subsequent tasks such as target detection and edge recognition more efficient and accurate.

[0052] S2: After binarizing the grayscale image, the region of interest is generated based on edge recognition, thus obtaining the initial attention field.

[0053] Furthermore, the binarization process includes converting the grayscale values ​​of the image into a black-and-white binary image. In the black-and-white binary image, edge information is detected; wherein, an edge recognition algorithm, such as the Canny edge detection algorithm, is used. Based on the edge information, discontinuous edges in the image are connected using an approximation algorithm; contours with predetermined geometric rules are identified before and after the connection, and two regions of interest are generated based on the two identification results.

[0054] It's important to note that after image binarization, edge detection algorithms (such as Canny edge detection) identify edge information in the image. Since the edges of some targets may be discontinuous due to noise, lighting variations, or irregular target shapes, further processing is needed to transform these discontinuous edges into a complete target contour. For discontinuous edges, approximation algorithms determine whether to connect them by calculating the geometric similarity between adjacent edges. This similarity is measured by the relative position of pixels and the edge shape (e.g., whether they have straight lines or similar curvature). Specifically, when the distance between edge segments is small and their shapes or orientations are similar, the approximation algorithm connects these edges to form a complete edge contour. The approximation algorithm uses geometric shape similarity and positional similarity for this connection. For each pair of discontinuous edge segments in the image, their similarity is calculated. Similarity metrics can include: positional similarity: whether two edge segments are adjacent and sufficiently close, typically determined by a threshold; and shape similarity: whether the edge orientation, curvature, continuity, etc., match. If the similarity between edge segments exceeds a set threshold, they are considered to belong to the same target edge and are connected.

[0055] The "predetermined geometric contour" (such as rectangle, circle, triangle, etc.) ensures that the detected contour is a reasonable target area. This avoids misidentification of irregular shapes or messy edges.

[0056] The generation of the region of interest (ROI) involves dividing the region based on the geometric features on both sides of the contour, and generating attention coefficients on both sides of each edge contour at the same scale using a pre-trained graph neural network. For each continuous contour, the similarity before and after edge connection in the approximation algorithm is used as the fusion weight for each continuous contour. The fusion weight at each contour is applied to the attention coefficients on both sides of the edge contour. The distribution of attention coefficients in the image without edge connection is used as the initial value. After weighting the distribution of attention coefficients after edge connection using the fusion weight, the initial value is added according to the pixel correspondence between the two images to obtain the attention coefficient distribution in the image. After normalizing the attention coefficient distribution in the image, the initial attention field is obtained.

[0057] By generating attention coefficients on both sides of the edge contour at the same scale, the consistency and accuracy of the target region in the image are ensured. In image processing, the geometric features of the target (such as shape and size) may change due to different viewpoints or distances. Therefore, generating attention coefficients at the same scale can avoid calculation errors caused by scale differences, thus ensuring the correct alignment and attention allocation of the target region. Through edge recognition and approximation algorithms, the system can effectively connect discontinuous edges, generate fusion weights based on the similarity of adjacent edges, and apply them to the attention coefficients on both sides of the edge contour. Considering that the final image processing will be performed on a grayscale image, attention coefficients without contour completion are used as templates to preserve the original features to the greatest extent. In the initial image, unconnected edges generate initial attention coefficients. The connected edges are weighted by fusion weights, making the attention distribution of the target region more accurate. Finally, through normalization processing, an optimized attention field is obtained. Based on this, the system can efficiently and accurately extract the geometric information of the target and reduce background interference.

[0058] Furthermore, the graph neural network includes nodes representing geometrically regular elements in the graph, edges representing relationships between nodes, and graph convolution operations enabling the graph neural network to propagate information between nodes, thereby completing the task of relationship inference and generating attention coefficients based on the inference results. Specifically, nodes represent different shapes or geometric elements in the graph, such as different parts of a target, which may be target regions, contours, or specific shapes (such as rectangles, circles, etc.) in an image. Each node contains features related to that shape or geometric element, such as geometric information like position, size, and shape. For example, in a target contour, each important region or shape (such as corners, edge segments, etc.) can be a node. Edges represent the relationships between these nodes, describing the connections between adjacent nodes (shapes). For example, the distance, angle, and relative position between adjacent edge segments. The weights of the edges reflect the strength of the relationships between these adjacent nodes, which may be calculated based on shape similarity, geometric feature similarity, or spatial proximity. Through these edges, the graph neural network can propagate information between nodes, thereby completing the inference of relationships between different shapes and generating the weights of each node (shape) in the attention field. During training, the graph neural network uses labeled data in the image for supervised learning. Through graph convolution operations, the network can effectively propagate information between nodes, capture the geometric relationships between adjacent shapes, and generate attention coefficients for each node based on the inference results. These coefficients are used in subsequent target recognition to represent the degree of attention to the target region, thereby optimizing the accuracy and robustness of target recognition.

[0059] S3: By filtering color blocks in the grayscale image, virtual energy is generated in the initial attention field, and the initial attention field is dynamically adapted through the virtual energy to obtain the final attention field.

[0060] Furthermore, the color block selection includes, in the grayscale image, grouping consecutive pixels with consistent grayscale values ​​(positive and negative deviations within a preset range are considered consistent) into a single color block. A virtual energy is set for each color block, and based on the grayscale value of each color block, the difference in grayscale values ​​between adjacent color blocks is calculated. This difference is used as the energy response between adjacent color blocks to represent the importance of the contours of adjacent locations for target detection.

[0061] The dynamic adaptation includes converting the differences between color blocks into a proportional value of attention between adjacent color blocks using a preset linear transformation function. The initial attention field is then mapped onto a grayscale image according to pixel correspondence, matching the color blocks with the attention distribution in the initial attention field.

[0062] Considering that the recognition accuracy of the binary image used during attention distribution generation is lower than that of the grayscale image, setting the attention within each color block to be consistent ensures that the attention weights within the color blocks remain consistent throughout the adaptation process. A proportional value is introduced between the color blocks to attenuate the attention distribution proportionally. This differs from directly using grayscale as the attention source, as it addresses the issue of ineffective differentiation of importance in images with small grayscale differences. Normalizing through a proportional approach yields the attention value, effectively identifying "relative importance."

[0063] Specifically, during the processing of binary images, edge information is simplified to black and white, resulting in the loss of some details and grayscale variation information. Therefore, compared to grayscale images, binary images often have lower recognition accuracy, especially under conditions of lighting changes, noise interference, or unclear target details, in which case the binarized image may fail to accurately extract the complete contour and features of the target.

[0064] To overcome this problem, maintaining consistent attention weights within color patches ensures that even in low-precision binary images, consistent features within color patches (such as the relative stability of the target region) can still be effectively addressed. Through consistent attention allocation, even with some recognition errors or loss of detail, the system can still stably allocate attention within the target region, thus guaranteeing the accuracy and integrity of the target region in subsequent recognition.

[0065] Next, the introduction of a scaling factor and attenuation is to adjust the distribution of attention based on the relative importance between the target and background regions. In an image, the target typically has high grayscale contrast or edge intensity, while the background is relatively weak. By calculating a scaling factor based on the grayscale differences between color blocks, attention can be focused on the target region while attenuating attention to the background region. This attenuation process ensures that attention is more focused on the target while suppressing background interference. Especially when the target shape is irregular, the edges are blurred, or there are multiple targets, the attenuation process can effectively eliminate irrelevant areas and enhance the target recognition effect.

[0066] Specifically, during attenuation, attenuation occurs within color blocks in the direction of decreasing attention within the graphic; furthermore, attenuation occurs between different color blocks within large areas of equal attention, and also in adjacent areas where attention differs.

[0067] In a region with equal attention levels, the minimum attention value of the color block is used as the basis for attention decay between regions. Attention decay is applied to adjacent regions according to their original attention weight. After attention decay is completed in adjacent regions, attention decay is applied between different color blocks within the region based on the attention decayed in adjacent regions.

[0068] Following the attenuation method, the direction in which attention increases in the graphic is enhanced between different color blocks, and this enhancement occurs synchronously with the attenuation process, thereby achieving synchronization of different attention cores.

[0069] After attenuation and enhancement, the attention of all color blocks is normalized to obtain the final attention field. In the final attention field, the parts with attention weights less than a preset value are removed to obtain the observation pixel region of the target.

[0070] It's important to note that the attenuation and enhancement processes occur simultaneously, aiming to ensure that the attention across different target regions is adjusted according to their geometric features, thus maintaining overall scale consistency. Specifically, during attenuation, a proportional value is calculated based on the grayscale differences between color blocks, effectively distinguishing the geometric features of target regions in the image. This proportional value is then used to attenuate or enhance the attention of different color blocks accordingly. Attenuation is not only about reducing background interference—although it does have a background filtering effect to some extent—but its core objective is to highlight the geometric features of the target, especially in cases of irregular shapes, blurred edges, or multiple targets. The attenuation and enhancement processes accurately represent the geometric shape and regional features of each target, ensuring that the geometric information of the target region is fully expressed at the global scale. During the simultaneous attenuation and enhancement process, attention is balanced across different target regions to ensure that the geometric features of the target region are maximized, while avoiding excessive concentration or dispersion of attention in target regions, which could affect the overall target recognition performance. The attenuation and enhancement processes not only optimize the geometric features of the target region but also ensure consistent attention distribution across all regions of the image, maintaining a consistent salience and scale of the target in the image.

[0071] It's important to note that the enhancement process aims not merely to increase attention in the target region, but to maintain consistent attention across different target regions in the image. Especially when the target region is prominent in the image, the attenuation process might weaken areas of local prominence. To avoid this, the enhancement mechanism adjusts the attention distribution of adjacent regions, preventing excessive local attenuation and restoring the target region's attention to an appropriate level. The enhancement process ensures that the target region's attention is not lost due to local attenuation, particularly in complex scenes with multiple targets present, where the enhancement mechanism plays a balancing and stabilizing role during attenuation. This design enables the system to achieve global adaptability, ensuring that the geometric features of all target regions are accurately captured at a global scale during target detection, effectively preserving the salience of the target region while avoiding recognition errors caused by excessive local attenuation. Finally, the attention field, after attenuation and enhancement, is normalized to form a consistent attention distribution, thereby accurately extracting the observed pixel region of the target and ensuring the system's target recognition accuracy and robustness in complex backgrounds and multi-target environments.

[0072] S4: In the algorithm model for image recognition, the final attention field is used to perform image recognition and extract the geometric information of the target to be tested, so as to obtain the recognition result of the target to be tested.

[0073] The observed pixel region of the target and the final attention field are input into an image recognition algorithm model for image recognition. The image recognition algorithm model is a pre-trained neural network that extracts high-level features from the image through multi-layer convolutional operations and performs target classification and geometric information extraction based on these features. Fully connected layers convert the feature maps output from the convolutional layers into target classification labels and geometric parameters. The training process of the neural network uses target-labeled data from the image for supervised learning to minimize image classification and regression errors, ensuring the accuracy of the recognition results.

[0074] It's important to understand that by combining the final attention field, the system can effectively focus on the target region in the image, reducing interference from the background and thus enhancing target recognition. This method uses neural networks to extract high-level features from the image for target classification and geometric information extraction, ensuring that the system can efficiently and accurately complete target detection tasks in complex scenes.

[0075] In this embodiment, the neural network used is a convolutional neural network (CNN), which extracts spatial features of the image through multiple convolutional layers and performs target classification and geometric information regression through fully connected layers. This structure can extract deep image features from images and is suitable for target detection and shape analysis.

[0076] In other alternative embodiments, the neural network can also employ a Generative Adversarial Network (GAN) to enhance image quality or process low-quality images, especially when the target region is occluded or the background is complex; it can also employ a Transformer network, which utilizes a self-attention mechanism to capture long-range dependencies, suitable for processing complex images or multi-object detection tasks; furthermore, a Deep Residual Network (ResNet) can be used as an alternative to address the vanishing gradient problem in deep network training, improving the model's performance in more complex tasks. By selecting different neural network architectures, this invention can better adapt to different application requirements, improving the flexibility and accuracy of object detection.

[0077] S5: Switch the measurement mode according to the human-computer interaction interface; and output the detection results for each mode according to the preset geometric information type in each measurement mode.

[0078] To enhance the flexibility and adaptability of the target detection system, enabling it to automatically switch between different measurement modes based on varying application scenarios and user needs, a human-computer interaction interface allows users to select the appropriate measurement mode according to their actual needs. The system will then output the corresponding detection results based on the preset geometric information types. Multiple measurement modes are designed, allowing the system to provide accurate geometric information extraction under different conditions. These measurement modes are output selections based on pre-set target geometric parameters and classification labels. For example, Mode 1: Multi-shape recognition, enabling shape classification and size measurement.

[0079] Mode 2: Specifically find the smallest rectangle and measure its side length.

[0080] Mode 3: Detect line segments, measure their pixel length and convert them into physical length.

[0081] This embodiment also provides a multi-mode target detection system based on machine vision, including: an acquisition unit that uses an image acquisition device to acquire real-time images of the detection area and convert the images to grayscale.

[0082] The analysis unit performs binarization on the grayscale image and generates a region of interest based on edge recognition, thereby obtaining the initial attention field.

[0083] The attention unit performs color block filtering in the grayscale image, generates virtual energy in the initial attention field, and dynamically adapts the initial attention field through the virtual energy to obtain the final attention field.

[0084] The recognition unit, in the algorithm model for image recognition, uses the final attention field to perform image recognition and extracts the geometric information of the target to be tested, thereby obtaining the recognition result of the target to be tested.

[0085] The interactive unit switches measurement modes according to the human-computer interaction interface; and outputs the detection results for each mode according to the preset geometric information type in each measurement mode.

[0086] This embodiment also provides a computer device applicable to the multi-modal target detection method based on machine vision, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the multi-modal target detection method based on machine vision as proposed in the above embodiment.

[0087] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0088] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the multi-mode target detection method based on machine vision as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-modal target detection method based on machine vision, characterized in that: This includes using image acquisition equipment to acquire real-time images of the detection area and converting the images to grayscale; After binarizing the grayscale image, the region of interest is generated based on edge recognition, thus obtaining the initial attention field; By filtering color blocks in a grayscale image, virtual energy is generated in the initial attention field, and the initial attention field is dynamically adapted using the virtual energy to obtain the final attention field. In the algorithm model used for image recognition, the final attention field is used to perform image recognition and extract the geometric information of the target to be tested, so as to obtain the recognition result of the target to be tested. Based on the human-computer interaction interface, the measurement mode is switched; based on the preset geometric information type in each measurement mode, the detection result is output for each modality.

2. The multi-modal target detection method based on machine vision as described in claim 1, characterized in that: The binarization process includes converting the grayscale values ​​of the image into a black-and-white binary image; In the black and white binary image, edge information is detected; Based on the edge information, an approximation algorithm is used to connect the discontinuous edges in the image; contours with predetermined geometric rules are identified before and after the connection, and two regions of interest are generated based on the two identification results. The region of interest is generated by dividing the region according to the geometric features on both sides of the contour, and by generating attention coefficients on both sides of each edge contour at the same scale through a pre-trained graph neural network. For each consecutive contour, according to the approximation algorithm, the similarity before and after edge connection is used as the fusion weight for each consecutive contour; The fusion weight at each contour is applied to the attention coefficients on both sides of the edge contour. The distribution of attention coefficients in the image without edge connection is used as the initial value. The distribution of attention coefficients after edge connection is weighted using the fusion weight, and then added to the initial value according to the pixel correspondence between the two images to obtain the attention coefficient distribution in the image. After normalizing the distribution of attention coefficients in the figure, the initial attention field is obtained.

3. The multi-modal target detection method based on machine vision as described in claim 2, characterized in that: The graph neural network includes nodes representing elements that conform to geometric rules in the graph, edges representing relationships between nodes, and graph convolution operations enabling the graph neural network to propagate information between nodes, thereby completing the task of relationship inference, and generating attention coefficients based on the inference results.

4. The multi-modal target detection method based on machine vision as described in claim 3, characterized in that: The color block selection includes, in a grayscale image, selecting consecutive pixels with the same grayscale as a color block; A virtual energy is set for each color block, and the difference in grayscale value between adjacent color blocks is calculated based on the grayscale value of each color block. The difference is used as an energy response between adjacent color blocks to represent the importance of the contours of adjacent locations for target detection.

5. The multi-modal target detection method based on machine vision as described in claim 4, characterized in that: The dynamic adaptation includes converting the difference between color blocks into a ratio of attention levels between adjacent color blocks using a preset linear transformation function. The initial attention field is mapped onto a grayscale image according to the pixel correspondence, so that the color blocks are matched with the attention distribution in the initial attention field; Considering that the recognition accuracy of the binary image used during the generation of attention distribution is lower than that of the grayscale image, setting the attention in the color block to be consistent ensures that the attention weight in the color block remains consistent throughout the adaptation process; The proportional value is introduced between the color blocks to proportionally attenuate the attention distribution. Specifically, during attenuation, attenuation occurs within color blocks in the direction of decreasing attention within the graphic; furthermore, attenuation occurs between different color blocks within large areas of equal attention, and also in adjacent areas where attention differs. In a region with equal attention levels, the minimum attention value of the color block is used as the basis for attention decay between regions. Attention decay is applied to adjacent regions according to their original attention weight. After attention decay is completed in adjacent regions, attention decay is applied between different color blocks within the region based on the attention decayed in adjacent regions. Following the attenuation method, the direction in which attention increases in the graphic between different color blocks is enhanced within the color blocks, proceeding synchronously with the attenuation process, thereby achieving synchronization of different attention cores; After attenuation and enhancement are completed, the attention of all color blocks is normalized to obtain the final attention field; In the final attention field, the portion with an attention weight less than a preset value is removed to obtain the observation pixel region of the target to be tested.

6. The multi-modal target detection method based on machine vision as described in claim 5, characterized in that: The observed pixel region of the target to be tested and the final attention field are input into the algorithm model for image recognition to perform image recognition. The algorithm model for image recognition is a pre-trained neural network that extracts high-level features from images through multi-layer convolution operations and performs target classification and geometric information extraction based on these high-level features; the feature maps output by the convolutional layers are converted into target classification labels and geometric parameters through fully connected layers. The training process of the neural network uses target-labeled data in the image for supervised learning to minimize image classification and regression errors, thereby ensuring the accuracy of the recognition results.

7. The multi-modal target detection method based on machine vision as described in claim 6, characterized in that: The measurement mode is an output selection based on pre-set target geometric parameters and classification labels.

8. A machine vision-based multi-modal target detection system, based on the machine vision-based multi-modal target detection method according to any one of claims 1 to 7, characterized in that: It includes an acquisition unit that uses an image acquisition device to acquire real-time images of the detection area and convert the images to grayscale. The analysis unit performs binarization on the grayscale image and generates a region of interest based on edge recognition, thereby obtaining an initial attention field. The attention unit performs color block filtering in the grayscale image, generates virtual energy in the initial attention field, and dynamically adapts the initial attention field through the virtual energy to obtain the final attention field. The recognition unit, in the algorithm model for image recognition, uses the final attention field to perform image recognition and extracts the geometric information of the target to be tested, thereby obtaining the recognition result of the target to be tested; The interactive unit switches measurement modes according to the human-computer interaction interface; and outputs the detection results for each mode according to the preset geometric information type in each measurement mode.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the machine vision-based multi-mode target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the machine vision-based multi-mode target detection method according to any one of claims 1 to 7.

Images