Machine vision target positioning method and system based on adaptive adjustment of bounding box

Abstract

The application discloses a machine vision target positioning method and system based on boundary frame self-adaptive adjustment, relates to the technical field of target detection and positioning, and comprises the following steps: acquiring real-time images and depth images of a current monitoring environment; inputting the real-time images into a light-weight target detection model based on an improved YOLO, identifying targets in the images and extracting target boundary frames; aligning the real-time images and the depth images, determining the depth value of each pixel point according to the coordinates of each pixel point in the target boundary frame region; based on the depth value of each pixel point in the target boundary frame region, using an adaptive width and distance measurement algorithm to distinguish target regions and background regions in the target boundary frame region, extracting the overall depth value of the target regions, determining the target distance and the target position, re-identifying the boundaries around the targets in the target boundary frame region, determining the target width and height, and thus completing target positioning. The application can realize more accurate and reliable acquisition of target positioning information.

Description

Technical Field

[0001] This invention relates to the field of target detection and localization technology, and in particular to a machine vision target localization method and system based on adaptive bounding box adjustment. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Object detection and localization is one of the current research hotspots in the field of computer vision. With the systematic use of deep learning in image and object detection, deep learning-based object detection algorithms such as YOLOv4 and YOLOv5 are widely used in the field of object detection and localization, and can obtain object localization information including object distance, object width and height, and object position.

[0004] However, due to factors such as image background complexity, existing object detection and localization algorithms inevitably suffer from confusion between background and foreground characteristics. The presence of a background can lead to inaccurate target localization, and the acquired distance information may also involve background areas, resulting in inaccurate final target position and distance information. Furthermore, limited by the accuracy of object detection algorithms, the generated target boundary may not be perfectly seamless, with gaps containing non-target elements. These gaps can affect the subsequent acquisition of target width and height, leading to inaccurate target width and height calculations. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a machine vision target localization method and system based on adaptive bounding box adjustment. It utilizes an improved lightweight target detection model to track and monitor targets and extract target bounding boxes, effectively improving detection accuracy and efficiency. Furthermore, it employs adaptive width and distance measurement algorithms to optimize the extracted target bounding boxes, calculating more accurate and reliable target location information such as target position, target distance, target width, and height. This solves the problems of inaccurate target bounding box region identification and low accuracy due to background interference in target localization in existing technologies.

[0006] In a first aspect, the present invention provides a machine vision target localization method based on adaptive bounding box adjustment.

[0007] A machine vision target localization method based on adaptive bounding box adjustment includes:

[0008] Acquire real-time and depth images of the current monitoring environment;

[0009] Real-time images are input into a lightweight object detection model based on improved YOLO to identify objects in the image and extract object bounding boxes;

[0010] Align the real-time image with the depth image, map the target bounding box region in the real-time image to the depth image, and determine the depth value of each pixel in the target bounding box region based on the coordinates of each pixel in the target bounding box region; among them, pixels that do not have depth values ​​are labeled as NAN data type;

[0011] Based on the depth value of each pixel in the target bounding box region, an adaptive width and distance measurement algorithm is used to distinguish the target area and the background area in the target bounding box region, extract the overall depth value of the target area, determine the target distance and target position, and then identify the four boundaries of the target in the target bounding box region to determine the target width and height, thereby completing the target localization.

[0012] Secondly, the present invention provides a machine vision target localization system based on adaptive bounding box adjustment.

[0013] A machine vision target localization system based on adaptive bounding box adjustment, comprising:

[0014] The image acquisition module is used to acquire real-time and depth images of the current monitoring environment;

[0015] The target bounding box extraction module is used to input real-time images into a lightweight target detection model based on improved YOLO, identify targets in the image, and extract target bounding boxes;

[0016] The depth value mapping module is used to align the real-time image and the depth image, mapping the target bounding box region in the real-time image to the depth image, and determining the depth value of each pixel in the target bounding box region based on the coordinates of each pixel in the target bounding box region; among them, pixels that do not have a depth value are labeled as NAN data type;

[0017] The target localization module is used to distinguish between the target area and the background area in the target bounding box area based on the depth value of each pixel in the target bounding box area, using an adaptive width and distance measurement algorithm. It extracts the overall depth value of the target area, determines the target distance and target position, and then identifies the four boundaries of the target in the target bounding box area to determine the target width and height, thereby completing the target localization.

[0018] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-described machine vision target localization method based on adaptive bounding box adjustment when executing the executable instructions stored in the memory.

[0019] Fourthly, the present invention also provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-described machine vision target localization method based on adaptive bounding box adjustment.

[0020] Fifthly, the present invention also provides a computer program product comprising executable instructions stored in a computer-readable storage medium; wherein, when the processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, it implements the aforementioned machine vision target localization method based on adaptive bounding box adjustment.

[0021] The above one or more technical solutions have the following beneficial effects:

[0022] 1. This invention provides a machine vision target localization method and system based on adaptive bounding box adjustment. It utilizes an improved lightweight target detection model to track and monitor targets and extract target bounding boxes, effectively improving detection accuracy and efficiency. Furthermore, it employs adaptive width and distance measurement algorithms to optimize the extracted target bounding boxes, calculating and obtaining more accurate and reliable target localization information. This solves the problems of inaccurate target bounding box region identification and low accuracy caused by background interference in target recognition and localization in existing technologies.

[0023] 2. Considering that accurate and rapid extraction of target detection boxes is a crucial prerequisite for precise target localization, this invention proposes a lightweight target detection model based on an improved YOLO, using the existing YOLOv11 model as a foundation to reduce the computational and parameter load of deep learning models and improve detection accuracy. This is achieved by introducing a SaE attention module into the neck network to enhance the model's feature extraction capabilities, strengthen its ability to capture channel patterns and grasp global knowledge, extract deeper feature representations, and improve the final target detection box recognition accuracy. Furthermore, an improved TDDown module is proposed and introduced into the backbone and neck networks of the improved model to further lightweight the network model, reducing the number of parameters and computational load while maintaining channel information. These improvements effectively enhance the efficiency and accuracy of the final detection.

[0024] 3. This invention uses an adaptive width and distance measurement algorithm to optimize the extracted target bounding box. By analyzing the depth value of each pixel in the target bounding box, it determines whether there is background interference in the target detection box area, and further distinguishes the background from the foreground to ensure the accuracy of the final target depth distance calculation. By performing NAN judgment column by column or row by row, the target area is further refined to calculate a more accurate target height and width, thereby achieving more accurate and reliable target positioning information.

[0025] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0026] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0027] Figure 1 This is a flowchart of the machine vision target localization method based on adaptive bounding box adjustment according to an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the structure of the vision sensor in an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the lightweight target detection model based on improved YOLO in an embodiment of the present invention;

[0030] Figure 4 This is a schematic diagram of an adaptive width and distance measurement algorithm for detecting targets in an embodiment of the present invention;

[0031] Figure 5 This is a schematic diagram of a machine vision target localization system based on adaptive bounding box adjustment in an embodiment of the present invention.

[0032] Among them, 1. Vision sensor; 2. Photosensitive sensor; 3. Color camera; 4. Infrared camera; 4-1. Left infrared camera; 4-2. Right infrared camera; 5. Structured light projector. Detailed Implementation

[0033] It should be noted that the following detailed descriptions are exemplary and are intended only to describe specific embodiments and to provide further explanation of the invention, and are not intended to limit the scope of exemplary embodiments of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0034] Example 1

[0035] To address the problem of inaccurate target localization in existing technologies, this embodiment provides a machine vision target localization method based on adaptive bounding box adjustment. By combining image processing, deep learning, and target detection technologies, an improved target detection model and an adaptive width and distance measurement algorithm are proposed to achieve more accurate and reliable target localization and obtain more precise target localization information. The machine vision target localization method based on adaptive bounding box adjustment proposed in this embodiment, as follows... Figure 1 As shown, it includes the following steps:

[0036] Step S1: Obtain real-time and depth images of the current monitoring environment.

[0037] In this embodiment, a vision sensor is installed, positioned appropriately facing the monitoring direction and connected to an industrial control computer for real-time monitoring of the current environment. Figure 2 As shown, the visual sensor 1 includes an infrared camera 4 (including a left infrared camera 4-1 and a right infrared camera 4-2), a color camera 3, and a structured light projector 5. The structured light projector is used to provide illumination to make the images captured by the camera clearer; the color camera is used to acquire real-time RGB images; the infrared camera is used to acquire real-time infrared images, and can also perform binocular ranging through the left and right dual infrared cameras to obtain depth images of the current monitoring environment. Moreover, it can be combined with the structured light projector to obtain more accurate depth image information.

[0038] In the design of the aforementioned visual sensor, considering the impact of light intensity on image acquisition, this embodiment employs two types of cameras for image acquisition. Given that the RGB color images acquired by the color camera can provide more information for target detection under strong light conditions, and the infrared images acquired by the infrared camera can provide more information for target detection under weak light conditions, this embodiment also includes a photosensor 2. This photosensor is positioned on one side of the visual sensor to acquire the real-time light intensity of the current monitoring environment. The photosensor is communicatively connected to the visual sensor and the industrial control computer. The industrial control computer determines the appropriate image capture method based on the received light intensity to acquire a real-time image: when the light intensity is greater than a set value, the color camera is activated for real-time image capture; when the light intensity is less than the set value, the infrared camera is activated for real-time image capture.

[0039] Step S2: Input the real-time image into a lightweight object detection model based on improved YOLO to identify the target in the image and extract the target bounding box.

[0040] In this embodiment, an improved YOLO target detection model is used to monitor targets in an image. It learns image features through a multi-layer neural network, extracts target bounding boxes based on image features, and then combines the target bounding boxes with depth information acquired by a depth camera for subsequent precise target localization.

[0041] To reduce the computational load and parameter count of deep learning models and improve detection accuracy for better deployment on edge devices, this embodiment proposes a lightweight target detection model based on the existing YOLOv11 model. The model is improved to achieve this. Figure 3 As shown, specifically:

[0042] First, we need to clarify the structure of the existing YOLOv11 model, which mainly includes a backbone network, a neck network, and a head network. The image is input into the backbone network, and through downsampling feature extraction and feature fusion operations in the backbone network, depth features are extracted. The depth features are then fused and upsampled through the neck network, and finally the head network outputs the target detection box.

[0043] The YOLOv11 network model includes Input, Conv (Convolution), C3k2 (Fast Implementation of CSP Bottleneck with 2 convolutions, feature extraction), SPPF (Spatial Pyramid Pooling-Fast, multi-scale feature extraction), C2PSA (attention-based feature extraction and processing), Upsample, Concat, and Detect (object detection box output). Specifically, Conv is used for downsampling; C3k2 is used for feature extraction and fusion; SPPF extracts features at multiple scales through pooling operations to help the model better detect objects of different sizes; C2PSA introduces a self-attention mechanism to compensate for the lack of long-range dependencies in CNN-based models (i.e., basic convolutional models), enhancing the network's global modeling ability and extracting self-attention-weighted features; Concat merges features; and Detect outputs object detection boxes. Additionally, the diagram shows... k Represents the convolution kernel ( kernel ), s Indicates stride ( strideC3k=False indicates that the dynamic convolution kernel size control function in the C3k2 structure is not enabled, while C3k=True indicates that it is enabled; FC (Full Connected Layer) means fully connected layer, and C is concat, which means feature splicing and merging.

[0044] In this embodiment, to further enhance YOLO detection capabilities and reduce computational and parameter requirements, improvements were made to the existing YOLOv11 model:

[0045] (1) To improve the accuracy of detection, a SaE (Squeeze Aggregated Excitation) attention module based on the SENetV2 attention mechanism, namely SaEBlock, was introduced into the neck network. This SaE attention module was combined with the C3k2 module to propose the C3k2SaE module. In this module, the SENetV2 attention mechanism was placed after two convolutions of the Bottleneck layer of the C3k2 module to enhance the module’s feature extraction capability, make the module pay more attention to features rich in expressive power, reduce redundancy, and thus enhance the model’s detection performance.

[0046] Specifically, in the SaE attention module, the input features are compressed through two convolutional layers before being input into the SaE attention module. After adaptive two-dimensional average pooling (Adaptive AvgPool2d), the importance weights of each channel of the compressed features are extracted. The extracted importance weights are then aggregated through a fully connected layer (FC), and finally, a channel-weighted summation is performed based on the importance weights of each channel and the input features to obtain the deep feature representation. The calculation formula can be expressed as:

[0047] ;

[0048] in, This indicates an excitation operation. This indicates a squeeze operation. Indicates input features, This represents the activation function.

[0049] By introducing the aforementioned SaE attention module, the network's ability to capture channel patterns and grasp global knowledge can be effectively enhanced, thereby achieving better feature representation.

[0050] (2) To achieve lightweighting, this embodiment also introduces a TDDown (TrioDepth-Wise Convolution Down) module in the backbone and neck network. This module is an improvement on the SCDown module (a type of downsampling module). In this TDDown module, the input features are processed in parallel through two parallel Depth-Wise Convolution (DW Conv) operations. The two processed features are then fused and subjected to a final depth convolution for feature downsampling. This embodiment replaces the traditional Point-Wise Convolution (PW Conv) operation used in the original SCDown module with the above method to further lightweight the SCDown module, and then achieves further lightweight feature downsampling through the TDDown module.

[0051] By using the aforementioned depthwise convolution for downsampling, the spatial dimension of features can be reduced, and low-dimensional features can be extracted. This can help to effectively reduce the spatial dimension of the input tensor, thereby reducing the number of model parameters and computational cost, while preserving channel information.

[0052] Step S3: Align the real-time image and the depth image. Map the target bounding box region in the real-time image to the depth image. Determine the depth value of each pixel in the target bounding box region based on the coordinates of each pixel in the target bounding box region.

[0053] In the above steps, considering that real-time images can be acquired through a color camera or an infrared camera respectively, when acquiring RGB real-time images using a color camera, the RGB images are synchronized and aligned with the depth images acquired by the depth camera. This alignment process includes:

[0054] First, the RGB camera and depth camera are calibrated to obtain the intrinsic parameters of both cameras (including focal length, etc.). , And principal points, etc.) and extrinsic parameters (including rotation matrices) With translation vector This ensures that the spatial relationship between the two is known.

[0055] Then, the pixels in the depth image are converted into 3D spatial points, that is, based on the intrinsic parameters of the depth camera, combined with the coordinates of each depth pixel. and its corresponding depth value The 3D position coordinates in the camera coordinate system can be calculated. .

[0056] Then, using the extrinsic parameter matrix With translation vector The 3D points, after the above transformation, are converted from the depth camera coordinate system to the RGB camera coordinate system to ensure they are in the same spatial reference frame. Considering that the transformed 3D points need to be projected back into the RGB image coordinate system, the intrinsic parameters of the RGB camera are then used to reconstruct the coordinates of the 3D points. 2D pixel coordinates mapped to an RGB image Furthermore, since the RGB image and the depth image have different resolutions, the projected depth data may contain holes or pixel misalignments. Therefore, bilinear interpolation can be used to correct these issues and improve alignment quality.

[0057] The above method completes the mapping from the depth image to the RGB viewpoint, resulting in aligned RGB-D data. In fact, most current depth cameras typically have a built-in synchronization alignment mode, which can be directly invoked via an SDK (Software Development Kit).

[0058] When using an infrared camera to acquire real-time infrared images, since the depth camera is based on a binocular infrared camera, when using infrared images, it is only necessary to determine whether the depth image is from the left or right camera to complete the alignment, without having to perform the above image alignment processing.

[0059] After aligning the real-time image with the depth image, the corresponding region in the depth image is sliced ​​and analyzed based on the coordinates of each pixel in the target's bounding box in the image, thus obtaining the depth value of each pixel in the target's bounding box. However, considering the limitations of the depth camera's maximum and minimum detection distances (typically a minimum working distance of 200mm and a maximum working distance of 20000mm), unusable data is returned when the object is within or outside the minimum or maximum working distance. Therefore, this embodiment uses the NAN (Not a Number) data type from Python's NumPy library to mark pixels without depth values, thus indicating a default value.

[0060] Step S4: Based on the depth value of each pixel in the target bounding box region, an adaptive width and distance measurement algorithm is used to distinguish the target area and the background area in the target bounding box region, extract the overall depth value of the target area, determine the target distance and target position, and then identify the four boundaries of the target in the target bounding box region to determine the target width and height, thereby completing the target localization.

[0061] Step S4.1: Based on the depth value of each pixel in the target bounding box region, an adaptive width measurement algorithm is used to distinguish the target region and the background region in the target bounding box region, extract the overall depth value of the target region, and determine the target distance and target position.

[0062] Considering the background interference present in target detection algorithms during ranging, which may incorrectly treat background distance as target distance and lead to inaccurate target ranging, this embodiment proposes a more accurate vision-based ranging method, including:

[0063] Step S4.1.1: After obtaining the coordinates and depth values ​​of each pixel in the target bounding box, first determine whether there is background interference within the target bounding box. Specifically, ignore all NAN values ​​within the obtained target bounding box region, and then compare them with a preset difference threshold (usually the maximum width of the tested object category at various angles is selected as the difference threshold). When the difference between the minimum and maximum depth values ​​in the target bounding box region is greater than the difference threshold, it indicates that there is background in the image that is deeper than the detected object. In this case, it is determined that there is background interference in the current region, and the subsequent step S4.1.2 is executed; otherwise, it is considered that there is no background interference in the image, and the subsequent step S4.1.3 is executed.

[0064] Step S4.1.2: After confirming the presence of background interference, since the depth value of the background is usually much larger than the depth value of the target, the minimum and maximum depth values ​​of all pixels in the region can be extracted. The average of the minimum and maximum depth values ​​is then taken as the depth boundary between the background and the target. Depth values ​​less than or equal to the average are considered the depth values ​​of target pixels, while depth values ​​greater than the average are considered the depth values ​​of background pixels. This distinguishes the target area from the background area within the target bounding box. Then, the average of all target pixel depth values ​​less than or equal to the average is taken to obtain the overall average depth distance (i.e., average depth value) of the target area. This distance is the target distance, offering greater robustness and accuracy.

[0065] Preferably, an appropriate comparison threshold is selected based on the distance between the foreground and background in the actual environment. For example, when the distance between the background and foreground is small, a value as small as possible, such as 1 / 3 or 1 / 4 of their sum, should be used; conversely, a larger value, such as the average value, should be used. This method allows for a more accurate distinction between the background and foreground.

[0066] Step S4.1.3: When it is determined that there is no background interference in the image, the average value of the depth values ​​of all target pixels within the target detection box area is taken to obtain the overall average depth distance (i.e., average depth value) of the target area.

[0067] Step S4.1.4: Based on the determined target pixel coordinates, after transforming the camera pixel coordinates to world coordinates, the actual position of the target in world coordinates, i.e., the target position, is obtained. The coordinate transformation formula is:

[0068] ;

[0069] in, This represents the intrinsic parameter matrix of the camera. The principal point location is the origin of the depth image coordinate system (also known as the center of the pixel coordinate system). , Let these represent the rotation matrix and translation vector in the extrinsic parameter matrix, respectively. , These represent the coordinates of the pixels to be converted. , , This represents the transformed world coordinates.

[0070] Step S4.2: Based on the depth value of each pixel in the target bounding box region, an adaptive width measurement algorithm is used to identify the four boundaries of the target in the target bounding box region and determine the target width and height.

[0071] Considering that the target bounding box drawn by the target detection model may not be perfectly aligned, with gaps between the bounding boxes and non-target elements, these gaps may affect the subsequent acquisition and calculation of the target width. Furthermore, since NAN is a default and invalid value and cannot be used for calculation, its presence may lead to fatal errors when directly using left and right breakpoints for width measurement. Therefore, to achieve more accurate target width measurement, this embodiment proposes a method for determining the target width and height, such as... Figure 4 As shown, it is:

[0072] First, based on the depth value of each pixel in the target bounding box region, NAN (Navigational Anonymous) checks are performed column-by-column from left to right and row-by-row from top to bottom to determine the target's four boundaries and optimize the target bounding box. Specifically, the column-by-column or row-by-row NAN check is performed as follows: if all pixels in the current column or row are NAN, then that column or row is ignored, and the NAN check is performed on the next column or row, until the current column or row is not entirely NAN. This method further optimizes the previously imperfectly fitted bounding box, making it more accurately represent the left, right, top, and bottom boundaries of the target.

[0073] Secondly, based on the optimized target bounding box, the minimum values ​​of the columns of the left and right boundaries and the minimum values ​​of the rows of the top and bottom boundaries are calculated to determine the coordinates of the left and right endpoints and the top and bottom endpoints of the target, thus eliminating background interference. After converting the camera pixel coordinates to world coordinates, the world coordinates of the left endpoint of the target are obtained. world coordinates of the right endpoint ,calculate The actual width of the target can be obtained; similarly, the world coordinates of the upper and lower endpoints of the target can be obtained, and then the actual height of the target can be calculated.

[0074] Example 2

[0075] This embodiment provides a machine vision target localization system based on adaptive bounding box adjustment, including:

[0076] The image acquisition module is used to acquire real-time and depth images of the current monitoring environment;

[0077] The target bounding box extraction module is used to input real-time images into a lightweight target detection model based on improved YOLO, identify targets in the image, and extract target bounding boxes;

[0078] The depth value mapping module is used to align the real-time image and the depth image, mapping the target bounding box region in the real-time image to the depth image, and determining the depth value of each pixel in the target bounding box region based on the coordinates of each pixel in the target bounding box region; among them, pixels that do not have a depth value are labeled as NAN data type;

[0079] The target localization module is used to distinguish between the target area and the background area in the target bounding box area based on the depth value of each pixel in the target bounding box area, using an adaptive width and distance measurement algorithm. It extracts the overall depth value of the target area, determines the target distance and target position, and then identifies the four boundaries of the target in the target bounding box area to determine the target width and height, thereby completing the target localization.

[0080] like Figure 5 As shown, the system proposed in this embodiment specifically includes an industrial control computer, a visual sensor (including a color camera, an infrared camera, and a depth camera) for real-time monitoring of the presence of a target, a photosensitive sensor for detecting ambient light intensity, other peripheral expansion modules, and a human-computer interaction module for convenient user monitoring and customization. The image acquisition module includes a visual sensor and a photosensitive sensor. The visual sensor uses the color camera and infrared camera to capture images of the surrounding environment, acquires depth images through left and right infrared cameras and a structured light projector, and connects to the industrial control computer via communication signals to transmit image data.

[0081] In addition, the industrial control computer is equipped with a target bounding box extraction module, a depth value mapping module, and a target localization module. It stores a target detection model, which is trained using a training dataset to improve detection accuracy. The target detection model then performs inference and decision-making on real-time images to extract the target bounding box. Combined with the depth map acquired by the depth camera, adaptive width and distance measurement algorithms are applied to obtain accurate world coordinates (i.e., target position), nearest distance (i.e., target distance), and target width and height, among other target localization information. Furthermore, based on the acquired target localization information, the control module communicates with other peripheral modules via the communication module, enabling remote acquisition of target localization information and other extended functions.

[0082] Example 3

[0083] This embodiment provides an electronic device, including: a memory for storing executable instructions; and a processor for executing the executable instructions stored in the memory to implement the method provided in this embodiment.

[0084] Example 4

[0085] This embodiment also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, will cause the processor to execute the method described above in this embodiment.

[0086] Example 5

[0087] This embodiment provides a computer program product including executable instructions, which are computer instructions; the executable instructions are stored in a computer-readable storage medium. When the processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the electronic device performs the method described in this embodiment.

[0088] The steps and methods involved in Embodiments 2 to 5 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0089] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0090] The above description is only a preferred embodiment of the present invention. Although the specific implementation of the present invention has been described in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the present invention.

Claims

1. A machine vision target localization method based on adaptive bounding box adjustment, characterized in that, include: Acquire real-time and depth images of the current monitoring environment; Real-time images are input into a lightweight object detection model based on improved YOLO to identify targets in the images and extract target bounding boxes. The lightweight object detection model based on improved YOLO is based on the YOLO11 model and includes a backbone network, a neck network, and a head network. The neck network introduces a SaE attention module, which is combined with a C3k2 module to form a C3k2SaE module. The backbone network and the neck network introduce a TDDown module. In the SaE attention module, the input features are compressed by two convolutional layers. After compression, the importance weights of each channel of the compressed features are extracted by the adaptive two-dimensional average pooling operation in the SaE module. The extracted importance weights are then aggregated by a fully connected layer. Finally, the importance weights of each channel are weighted with the input features to obtain a deep feature representation. In the TDDown module, the input features are processed in parallel through two parallel deep convolution operations. After the two processed features are fused, they are then subjected to a deep convolution operation for feature downsampling to extract low-dimensional features. Align the real-time image with the depth image, map the target bounding box region in the real-time image to the depth image, and determine the depth value of each pixel in the target bounding box region based on the coordinates of each pixel in the target bounding box region; among them, pixels that do not have depth values ​​are labeled as NAN data type; Based on the depth value of each pixel in the target bounding box region, an adaptive width and distance measurement algorithm is used to distinguish the target area and the background area in the target bounding box region, extract the overall depth value of the target area, determine the target distance and target position, and then identify the four boundaries of the target in the target bounding box region to determine the target width and height, thereby completing the target localization.

2. The machine vision target localization method based on adaptive bounding box adjustment as described in claim 1, characterized in that, Acquire real-time images of the current monitoring environment, including: Real-time acquisition of the light intensity of the current monitoring environment; Based on light intensity, select the image capture method to obtain real-time images, including: When the light intensity exceeds the set value, the color camera is activated to capture images in real time. When the light intensity is less than the set value, the infrared camera is activated to capture images in real time.

3. The machine vision target localization method based on adaptive bounding box adjustment as described in claim 1, characterized in that, The determination of the target distance and target location includes: Based on the coordinates and depth values ​​of each pixel in the target bounding box, it is determined whether there is background interference within the target bounding box. Specifically, all NAN values ​​within the acquired target bounding box region are ignored, and the minimum and maximum depth values ​​of all pixels in the region are extracted. The difference between the minimum and maximum depth values ​​in the target bounding box region is compared with a preset difference threshold, and the presence of background interference is determined based on the comparison result. If background interference is detected, the depth value of each pixel in the target bounding box region is used as the average of the minimum and maximum depth values. The depth value of each pixel is then compared with this average value. Depth values ​​less than or equal to the average value are considered as target pixel depth values, while depth values ​​greater than the average value are considered as background pixel depth values. This distinguishes the target region from the background region within the target bounding box region. The average of all target pixel depth values ​​less than or equal to the average value is then taken to obtain the overall average depth distance of the target region, i.e., the target distance. If it is determined that there is no background interference, the average value of the depth values ​​of all target pixels within the target detection box area is taken to obtain the overall average depth distance of the target area, i.e., the target distance. Based on the determined target pixel coordinates, the actual position of the target is obtained after converting the camera pixel coordinates to world coordinates.

4. The machine vision target localization method based on adaptive bounding box adjustment as described in claim 1, characterized in that, The determination of the target width and height includes: Based on the depth value of each pixel in the target bounding box region, perform column-by-column NAN judgment from left to right and row-by-row NAN judgment from top to bottom to determine the four boundaries of the target and optimize the target bounding box. Based on the optimized target bounding box, the minimum values ​​of the columns of the left and right boundaries and the minimum values ​​of the rows of the top and bottom boundaries are calculated to determine the coordinates of the left and right endpoints and the top and bottom endpoints of the target. After converting the camera pixel coordinates to world coordinates, the world coordinate values ​​of the left and right endpoints and the top and bottom endpoints of the target are obtained, thereby determining the actual width and height of the target.

5. The machine vision target localization method based on adaptive bounding box adjustment as described in claim 4, characterized in that, The column-by-column or row-by-row NAN determination is as follows: if all pixels in the current column or row are NAN, then ignore that column or row and proceed to the next column or row for NAN determination, until the current column or row is not entirely NAN.

6. A machine vision target localization system based on adaptive bounding box adjustment, characterized in that, include: The image acquisition module is used to acquire real-time and depth images of the current monitoring environment; The target bounding box extraction module is used to input real-time images into a lightweight target detection model based on improved YOLO, identify targets in the image, and extract target bounding boxes. The lightweight target detection model based on improved YOLO is based on the YOLO11 model and includes a backbone network, a neck network, and a head network. The neck network introduces a SaE attention module, which is combined with the C3k2 module to form a C3k2SaE module. The backbone network and the neck network introduce a TDDown module. In the SaE attention module, the input features are compressed by two convolutional layers. After compression, the importance weights of each channel of the compressed features are extracted by the adaptive two-dimensional average pooling operation in the SaE module. The extracted importance weights are then aggregated by a fully connected layer. Finally, the importance weights of each channel are weighted with the input features to obtain a deep feature representation. In the TDDown module, the input features are processed in parallel through two parallel deep convolution operations. After the two processed features are fused, they are then subjected to a deep convolution operation for feature downsampling to extract low-dimensional features. The depth value mapping module is used to align the real-time image and the depth image, mapping the target bounding box region in the real-time image to the depth image, and determining the depth value of each pixel in the target bounding box region based on the coordinates of each pixel in the target bounding box region; among them, pixels that do not have a depth value are labeled as NAN data type; The target localization module is used to distinguish between the target area and the background area in the target bounding box area based on the depth value of each pixel in the target bounding box area, using an adaptive width and distance measurement algorithm. It extracts the overall depth value of the target area, determines the target distance and target position, and then identifies the four boundaries of the target in the target bounding box area to determine the target width and height, thereby completing the target localization.

7. An electronic device, characterized in that, include: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the machine vision target localization method based on bounding box adaptive adjustment as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The device stores executable instructions that, when executed by a processor, implement the machine vision target localization method based on adaptive bounding box adjustment as described in any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, it implements the machine vision target localization method based on bounding box adaptive adjustment as described in any one of claims 1-5.

Images