High-precision target identification and positioning method

Abstract

The invention relates to the technical field of computer vision and image processing, in particular to a high-precision target recognition and positioning method. The method comprises the following steps: carrying out image acquisition and preprocessing on a target area to obtain original image data; calibrating and correcting the original image data to obtain accurate image information; calculating depth information of the target by adopting a depth estimation method based on an SGM algorithm; parallax optimization is carried out on the depth information, and the accuracy and stability of the image are improved; and through a post-processing and output control module, depth information output in various formats is generated. The depth estimation method based on the SGM algorithm and the single-channel separation processing of the YUV image data are introduced, so that the target recognition and positioning precision and the processing speed are remarkably improved, and especially in potential application in the medical field, powerful technical support is provided for an intelligent microcirculation skin therapeutic instrument.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and specifically relates to a high-precision target recognition and positioning method based on the SGM algorithm, which aims to improve the accuracy and real-time performance of target detection and is used for automated processing of various application scenarios, especially potential applications in the medical field. Background Art

[0002] In the process of target recognition and positioning, accurately identifying the target type and its location is the core of many intelligent applications. At present, traditional target detection methods often rely on manual feature extraction and classification, which are limited by the ability to express features, and have problems such as insufficient accuracy and poor adaptability to complex environments. In addition, existing methods face the challenges of low efficiency and poor real-time performance when processing large-scale data. By introducing the SGM algorithm, the present invention shows significant advantages in depth estimation and can efficiently and accurately calculate the depth information of the target. Compared with traditional methods, the SGM algorithm has strong universality and can adapt to complex backgrounds and changing lighting conditions, thereby achieving accurate recognition and precise positioning of targets. This technology has broad application prospects in the medical field, especially in obtaining skin location information and modeling patients' affected areas, providing strong support. Summary of the invention

[0003] The present invention aims to solve the above problems and proposes a high-precision target recognition and positioning method based on the SGM algorithm.

[0004] The main objective of the present invention is to provide a high-precision target recognition and positioning method based on the SGM algorithm, which is applicable to an intelligent microcirculation skin treatment device. The present invention covers the following core modules: image acquisition and preprocessing module, image calibration and correction module, depth estimation module, parallax processing module and post-processing and output control module.

[0005] The main functions of the intelligent microcirculation skin treatment instrument include binocular depth estimation, skin classification, real-time 3D modeling, IMU calibration and posture solution. The instrument will be equipped with the D163A platform, collect skin data through dual cameras with MIPI-2lane interface, and transmit the algorithm results to the host computer software developed based on PYQT through the Gigabit Ethernet port UDP protocol, and display the 3D modeling results in real time. In addition, the project also plans to establish the skin pressure and vascular microcirculation dynamics equation to provide refined treatment strategies and explore corresponding massage treatment methods. The overall goal is to improve the accuracy and effectiveness of skin treatment and provide a scientific basis for doctors.

[0006] The image acquisition and preprocessing module will be used to process the RAW format original images taken by the binocular camera and build a diverse data set. This module uses the ISP (image signal processing) model to perform multiple processes to provide the required image data for subsequent stereo matching, thereby improving the efficiency of target detection.

[0007] The image calibration and correction module will be used to achieve accurate correction of image data after obtaining the intrinsic and extrinsic parameters of the camera through the Zhang Zhengyou calibration method. The operating mechanism of this module includes calculating the intrinsic and extrinsic parameters of the camera to reduce the impact of lens distortion and ensure the accuracy of image correction and epipolar correction.

[0008] The depth estimation module uses a cost calculation method based on Census-Transform to calculate the cost matrix of the image to capture the pixel differences between images; then, the cost is aggregated in 8 directions to obtain more accurate disparity information, and the WTA algorithm is used to calculate the disparity. This enables the system to process large-scale data quickly and efficiently, thereby calculating the depth information of each target and providing necessary support for the spatial positioning of the target.

[0009] The disparity processing module will be used to further improve the accuracy and stability of the disparity map. The module first performs left-right consistency detection to eliminate erroneous disparity caused by occlusion and noise. Afterwards, the isolated outliers are eliminated through the regional voting mechanism to improve the overall quality of the disparity map. At the same time, edge correction technology is used to enhance the clarity of the image edge, and the sub-pixel enhancement method is used to optimize the accuracy of the disparity. Finally, the disparity map is smoothed through median filtering to reduce the interference of noise on the final result, so as to ensure that the generated depth map has high reliability.

[0010] The post-processing and output control module is used to output the obtained depth map in a variety of formats including pseudo-color images. This module integrates a variety of output interfaces, so that the depth information can be easily integrated with other systems, while providing support for subsequent data analysis and applications. Through this module, it is ensured that the results of target recognition and positioning are not only highly accurate, but also provide the necessary depth information for practical applications, improving the efficiency and accuracy of target tracking and monitoring.

[0011] Preferably, the image acquisition and preprocessing module adopts the ISP model, which will be used to achieve image denoising, dynamic range optimization, color correction and white balance, image cropping and output in YUV format.

[0012] The image denoising aims to effectively remove noise while maintaining image edges.

[0013] The dynamic range optimization aims to effectively alleviate the brightness mismatch problem between the left and right images by enhancing the contrast and brightness of the image, so that the Census-Transform-based cost calculation method can enhance the distinction between different features in the same scene, thereby improving the matching accuracy and environmental adaptability.

[0014] The color correction and white balance are intended to reduce the impact of different ambient light sources on the image color and ensure that white and gray objects in the image present true colors to ensure the accuracy of the output grayscale.

[0015] The image cropping is intended to uniformly resize images of different resolutions and sizes to ensure that the binocular images input to the next module have a consistent size.

[0016] The output in YUV format is intended to provide an optimized input data format for the SGM algorithm. By converting the image to YUV format, the brightness information and color information can be effectively separated, so that the brightness channel (Y channel) is processed first when performing depth estimation, which significantly improves the calculation efficiency.

[0017] Preferably, the image calibration and correction module is used to implement image calibration and epipolar calibration.

[0018] The image calibration is used to obtain the internal and external parameters of the camera through the Zhang Zhengyou calibration method to achieve high-precision image correction. Its structure includes the acquisition of multiple calibration images, the extraction of feature points and the calculation of parameters. By shooting a calibration plate of known size, the module extracts corner point information and uses this information for parameter estimation to effectively eliminate lens distortion and geometric distortion, ensuring that the acquired image is accurate in geometric structure.

[0019] The epipolar calibration is used to improve the matching accuracy of the binocular images and the reliability of depth estimation. This module is executed after image calibration and ensures that the points between the binocular images can accurately correspond by calculating the epipolar lines. This module also involves the calculation of the relative position and posture of the cameras to ensure the geometric consistency of the stereo images.

[0020] Preferably, the depth estimation module is used for cost calculation, cost aggregation and disparity calculation.

[0021] The cost calculation is used to generate a cost matrix using the Census-Transform method to capture the pixel differences between binocular images. Census-Transform generates a bit string by comparing the grayscale value of pixels in a local neighborhood with the grayscale value of a central pixel, and uses it as a Census value.

[0022] The cost aggregation is used to optimize the cost matrix by an 8-direction aggregation method to ensure the accuracy and stability of the disparity information in different directions.

[0023] The disparity calculation is combined with the WTA algorithm to quickly calculate the disparity between images and generate an accurate depth map.

[0024] Preferably, the disparity processing module is not only used for disparity optimization, but also includes the function of disparity filling to improve the accuracy of the depth information of the image. The module can automatically analyze images from different perspectives, optimize disparity calculations, and ensure that the output depth map has higher detail performance. In addition, the disparity processing module also implements disparity smoothing based on image features to reduce interference caused by noise or lighting changes, thereby improving the overall quality of the disparity map. The module supports disparity filling of complex scenes, and completes the missing depth information through intelligent algorithms to ensure that the generated depth map can be used to reconstruct a three-dimensional model and provide a coherent and realistic effect.

[0025] In the above technical solution, the present invention provides a high-precision target recognition and positioning method, which has the following beneficial effects:

[0026] 1. The present invention introduces a depth estimation method based on the SGM algorithm and single-channel separation processing of YUV image data, which effectively improves the accuracy of target recognition and positioning and the algorithm processing speed, significantly enhances its adaptability in complex environments, and provides a solid foundation for the application of intelligent microcirculation skin therapeutic devices.

[0027] 2. The present invention significantly reduces the reliance on manual intervention and improves the efficiency of data processing through a fully automated image processing process, and can meet clinical needs more quickly and handle a large number of cases.

[0028] 3. The present invention combines multiple optimization strategies, such as cost calculation and disparity optimization, to effectively prevent overfitting, enhance the generalization ability of the method, and ensure that a high level of performance can be maintained in different scenarios.

[0029] 4. The multi-module structure designed in the present invention ensures the wide applicability and flexible scalability of the method, and can provide accurate depth information for various injuries (such as burns, trauma, etc.), thereby laying a scientific foundation for doctors to formulate personalized treatment plans and significantly improving clinical treatment effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1A schematic diagram of the module structure of a high-precision target recognition and positioning method of the present invention;

[0032] Figure 2 A data flow diagram of a high-precision target recognition and positioning method of the present invention;

[0033] Figure 3 A high-precision target recognition and positioning method according to the present invention is based on the schematic diagram of the SGM algorithm structure;

[0034] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0035] In order to clearly and completely describe the technical solutions in the embodiments of the present invention, detailed description is given below in conjunction with the drawings in the embodiments.

[0036] It should be noted that the described embodiments are only a part of the embodiments of the present invention, not all of them. Without doing creative work, all other embodiments that can be implemented by ordinary technicians in this field based on the content of the present invention are within the protection scope of the present invention.

[0037] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0038] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0039] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0040] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0041] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0042] See also Figure 1 The present invention relates to a high-precision target recognition and positioning method, including an image acquisition and preprocessing module, an image calibration and correction module, a depth estimation module, a parallax processing module, and a post-processing and output control module.

[0043] The input end of the image acquisition and preprocessing module is connected to the original image data captured by the sensor of the binocular camera. The image acquisition and preprocessing module performs AD acquisition on the left and right image signals from the binocular camera according to the set sampling rate and image capture parameters.

[0044] Among them, the image acquisition and preprocessing module also includes functions such as image denoising, dynamic range optimization, color correction and white balance processing. Specifically, the image acquisition and preprocessing module uses the hardware acceleration function of the D163A platform to select an efficient spatial filtering algorithm for denoising to reduce image noise and retain edge details. Dynamic range optimization dynamically adjusts the brightness and contrast of the image through hardware-supported HDR processing algorithms and gamma correction methods, implements local contrast enhancement to extract more details in specific areas, and ensures global consistency. At the same time, color correction and white balance calculate the color temperature and adjust the intensity of the RGB channels by analyzing the gray card or white area in the image to achieve accurate color reproduction.

[0045] Among them, the image acquisition and preprocessing module also includes functions such as image cropping and output in YUV format. Specifically, image cropping is achieved by defining a region of interest (ROI). During the cropping process, the algorithm will ensure that the image ratio and resolution match. After cropping, the processed image data will first be demosaiced to obtain the RGB value of each pixel, and then converted to YUV format to meet the processing requirements of subsequent modules. During this conversion process, the following calculations are performed for each pixel: Y=0.299R+0.587G+0.114B U=-0.147R-0.289G+0.436B V=0.615R-0.515G-0.100B

[0046] The conversion process uses optimized loops and parallel processing techniques to accelerate data conversion.

[0047] The dynamic input end of the image calibration and correction module is connected to the image acquisition and preprocessing module, and the static input end is connected to the configuration file of the image calibration and correction module. The configuration file needs to store the camera internal and external parameter matrix obtained by Zhang Zhengyou calibration method, and the key functions are image calibration and epipolar calibration.

[0048] Specifically, the image calibration and correction module first loads the camera's intrinsic and extrinsic parameter matrices according to the configuration file of the static input terminal. These parameters include the camera's intrinsic parameter matrix K, the camera's rotation matrix R, and the translation vector t. During the image calibration process, the module compares the coordinates of the feature points in the input image with the corresponding world coordinates and calculates the reprojection error to optimize the parameters. The reprojection error can be expressed by the following formula:

[0049] Among them, P image is the coordinate of the feature point actually detected, and The reprojected coordinates estimated based on the current parameters.

[0050] Epipolar calibration is based on the geometric relationship of the binocular camera and uses the fundamental matrix F to define the epipolar relationship between the left and right images. By calculating the fundamental matrix, the following epipolar constraints can be obtained: P' T F P =0

[0051] Among them, P' and P are the corresponding points in the left and right images respectively. On this basis, the module restricts the matching of feature points to the vicinity of their corresponding epipolar lines through epipolar constraints, thereby improving the accuracy of disparity calculation.

[0052] The input of the depth estimation module is connected to the image calibration and correction module. This module implements cost calculation, cost aggregation and disparity calculation. This module first separates the single channel data of the processed YUV format binocular image obtained at the input and stores them in arrays.

[0053] For the cost calculation, the present invention adopts the Census-Transform method, which is specifically implemented by using the local grayscale difference in the pixel neighborhood to convert the pixel grayscale into a bit string, by comparing the pixel grayscale value in the neighborhood window (this method uses a 5X5 window size) with the grayscale value of the central pixel of the window, mapping the Boolean value obtained by comparison into a bit string, and finally using the value of the bit string as the Census transformation value C of the central pixel. S , calculate the Hamming distance of the Census transformation values ​​of the two pixels corresponding to the left and right images as the cost value and store it in the cost matrix.

[0054] For the cost aggregation, the present invention adopts a one-dimensional path aggregation method, that is, the matching cost under all disparities of the pixel is one-dimensionally aggregated on all paths around the pixel to obtain the path cost value under the path, and then all the path cost values ​​are added to obtain the matching cost value of the pixel after aggregation. The cost of each pixel is aggregated along 8 directions according to the path cost calculation formula, so as to approximately solve the global energy function.

[0055] The path cost calculation formula is as follows

[0056] Among them, the first item is the matching cost value C, which belongs to the data item; the second item is the smoothing item, which means that all pixels in the neighborhood of the pixel are punished. The penalty for the case where the parallax change of adjacent pixels is very small (1 pixel) is relatively small; the penalty for the case where the parallax change of adjacent pixels is very large (greater than 1 pixel) is relatively large. The value accumulated to the path cost is the minimum value among the three cases of no penalty, small parallax change penalty and large parallax change penalty; the third item is to ensure that the new path cost value does not exceed a certain upper limit.

[0057] A winner-takes-all algorithm is used for the disparity calculation, and each pixel selects the disparity value corresponding to the minimum aggregation cost value as the final disparity.

[0058] The input end of the disparity processing module is connected to the depth estimation module. This module implements disparity optimization and disparity filling. The implementation of disparity optimization is characterized by sub-pixel fitting, consistency check, regional voting and edge correction. Among them, the sub-pixel fitting method improves the accuracy of the disparity value by fitting a univariate quadratic curve, thereby effectively capturing detail changes. Specifically, the whole pixel disparity value is first determined, and then the curve is fitted according to the cost value of its left and right neighboring pixels to calculate the extreme point, so as to obtain the precise disparity position of the sub-pixel. The consistency check verifies the reliability of the disparity matching by comparing the left and right disparity maps. The specific steps include calculating the disparity values ​​of the corresponding pixels of the left and right images. If the difference exceeds the set threshold, it is considered that the match is inconsistent, and then the disparity value is set to invalid. Regional voting collects valid disparity values ​​in the local neighborhood and selects the most appropriate disparity according to the voting results to ensure higher accuracy and consistency. Edge correction adjusts the disparity discontinuity of the edge area by introducing edge information.

[0059] The implementation of the disparity filling is characterized in that different processing strategies are used for occluded areas and mismatched areas to improve the filling effect. For occluded areas, the sub-minimum disparity of the background pixels is used for filling to ensure that the disparity value after filling matches the surrounding background and avoids the interference of foreground pixels. The specific steps include collecting disparity values ​​from the surrounding valid pixels and selecting the sub-minimum value for filling. For the mismatched area, a median filtering strategy is adopted to collect the disparity values ​​of the valid pixels in the neighborhood and calculate the median as the filling result. In addition, the angle sampling method is introduced in the filling process to locate the first valid pixel by emitting rays in multiple directions. Ultimately, the image after disparity filling shows more complete and coherent depth information.

[0060] The input of the post-processing and output control module is connected to the disparity processing module. The main function of this module is to process and format the optimized and padded depth map, and provide multiple output interfaces to meet different application requirements. Specifically, post-processing includes further color mapping of the disparity map to enhance the visual effect. In addition, the output control module can convert the processed data into different formats, such as YUV or RGB, to meet the requirements of various display devices. This module is also responsible for real-time transmission of depth map data through HDMI or other interfaces, and automatically adjusts the output parameters according to the characteristics of the display to ensure the optimization of the display effect.

[0061] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A high-precision target recognition and positioning method, characterized in that: include: Image acquisition and preprocessing module, image calibration and correction module, depth estimation module, parallax processing module and post-processing and output control module. The image acquisition and preprocessing module is used to process the RAW format original images taken by the binocular camera, and perform image denoising, dynamic range optimization, color correction and white balance and image cropping through the ISP model, and output YUV format image data to build a diverse data set and improve the efficiency of subsequent target detection. The image calibration and correction module is used to obtain the internal and external parameters of the camera through the Zhang Zhengyou calibration method, achieve accurate correction of image data, reduce the influence of lens distortion, and ensure the accuracy of the geometric structure of the image. The depth estimation module is used to adopt a cost calculation method based on Census-Transform to calculate the cost matrix of the image and perform 8-direction aggregation to obtain more accurate disparity information, and finally realize fast and accurate depth information calculation. The disparity processing module is used to further improve the accuracy and stability of the disparity map, and successively performs sub-pixel fitting, left-right consistency detection, regional voting mechanism and edge correction technology to optimize the overall quality of the disparity map. The post-processing and output control module is used to output the obtained depth map in multiple formats, integrate multiple output interfaces, support compatible depth maps and preset mapping tables, and output the color map in video stream format to the display, which can be used for visualization effect evaluation and support subsequent data analysis and application.

2. A high-precision target recognition and positioning method according to claim 1, characterized in that: The image acquisition and preprocessing module includes: a binocular camera module and image denoising, dynamic range optimization, color correction and white balance, image cropping and output in YUV format using an ISP model; The binocular camera module is used to obtain the binocular original image in RAW format. The image denoising function is used to effectively remove noise while maintaining the image edge. The dynamic range optimization function is used to enhance the contrast and brightness of the image, and effectively alleviate the brightness mismatch problem between the left and right images. The color correction and white balance function is used to reduce the impact of different ambient light sources on the image color, ensuring that white and gray objects in the image present true colors, so as to ensure the accuracy of the output grayscale. The image cropping function is used to ensure that the binocular images input to the next module have a consistent size. The output function in YUV format is used to convert the image into YUV format, which can effectively separate the brightness information and the color information.

3. A high-precision target recognition and positioning method according to claim 1, characterized in that: The image calibration and correction module includes: an image calibration submodule and an epipolar calibration submodule; The image calibration submodule is used to achieve high-precision image geometric correction, effectively eliminate lens distortion and geometric distortion, and improve image accuracy. The epipolar calibration submodule is used to improve the correspondence accuracy between image points and enhance the reliability of subsequent depth estimation.

4. A high-precision target recognition and positioning method according to claim 1, characterized in that: The depth estimation module includes: a cost calculation submodule, a cost aggregation submodule and a disparity calculation submodule; The cost calculation submodule is used to generate a cost matrix. The cost aggregation submodule is used to improve the stability of disparity information and ensure the consistency of depth estimation. The disparity calculation submodule is used to generate a depth map.

5. A high-precision target recognition and positioning method according to claim 1, characterized in that: The disparity processing module includes: a disparity optimization submodule and a disparity filling submodule; The disparity optimization submodule is used to improve the overall quality of the disparity map, reduce errors, and improve recognition accuracy. The parallax filling submodule is used to fill in the missing depth information, enhance the detail expression of the depth map, and ensure the authenticity of the three-dimensional reconstruction.

6. A high-precision target recognition and positioning method according to claim 1, characterized in that: The main function of the post-processing and output control module is to provide output interfaces in multiple output formats.

7. According to the output interface supporting multiple output formats as described in claim 6, the present invention provides an application example of the output interface, characterized in that: The video stream is integrated using pseudo color images, the image format is set to YUV 4:2:2, and it is sent to the display through the HDMI transmission protocol. The specific steps include: Step S71: For the depth map data, the depth values ​​are converted into corresponding color values ​​through a preset mapping algorithm to form a visual depth map. The mapping algorithm maps the farther depth values ​​to cold colors and the closer depth values ​​to warm colors based on the range of the depth values. Step S72: performing image processing on the generated color depth map, including scaling, cropping and format conversion, to ensure that the resolution of the output image matches the specification of the display. Step S73: Select the transmission protocol of the HDMI interface version 2.0, and transmit the modulated transmission signal to the display in real time through the interface. Step S74: By real-time monitoring of the EDID information of the display, the output frame rate is automatically adapted to ensure synchronous updating of the image frames.

8. A method for implementing a high-precision target recognition and positioning method comprises the following steps: Step S81: Start using the binocular camera module to capture the original image in RAW format and provide it to the ISP model. Step S82: image denoising, dynamic range optimization, color correction and white balance are implemented, and the preset image resolution in the configuration file of the image acquisition and preprocessing module is read to crop the image. Step S83: perform chromaticity separation on the image to obtain a YUV format image, and output the image data to the image calibration and correction module. Step S84: Obtain the intrinsic and extrinsic parameters of the camera through the Zhang Zhengyou calibration method, and write the intrinsic and extrinsic parameter matrix into the configuration file of the image calibration and correction module. Step S85: read the data in the configuration file, calibrate the image, use the internal and external parameters of the camera to establish an epipolar geometry model, calculate the epipolar position, and perform epipolar calibration. Step S86: Read the calibrated image data and separate the image Y channel data. Step S87: Using the Census Transform method, the pixel grayscale is converted into a bit string, and through dynamic programming, the cost is aggregated from 8 directions to obtain the optimal path cost value for each pixel. Step S88: perform parallax optimization, and process through left-right consistency detection, regional voting system, edge correction, sub-pixel enhancement and median filtering. Step S89: perform disparity filling (optional) and predict invalid pixels using surrounding valid values. Step S810-1: Output control, responsible for image file format conversion and providing multiple output ports. Step S810-2: compatible with the preset mapping table, output the pseudo color image in the video stream format, the display receives the video stream, and the visual effect evaluation can be performed. Step S810-1 and step S810-2 represent two parallel steps, and you can choose either one.

Images