Image acquisition processing and contrastive analysis method of private nursing instrument

Abstract

The invention belongs to the technical field of image processing, and particularly relates to an image acquisition processing and contrastive analysis method of a private nursing instrument, which comprises the following steps of: synchronously acquiring six-axis attitude data, laser ranging object distance data and light supplementing lamp illumination parameters, performing SIFT (scale invariant feature transform) feature matching and attitude threshold accurate detection jitter, and performing image processing; then eliminating jitter angle deviation through motion model construction and pixel reprojection, realizing scale normalization in combination with an object distance proportion and bilinear interpolation, correcting illumination deviation by means of brightness compensation and a gray world algorithm, and finally respectively correcting pixel position deviation, scale difference, gray mean value difference and color temperature deviation of the two images, so as to obtain a final image. The same scene data basis is provided for subsequent comparison; on the premise, on the basis of pixel-level and feature-level multi-dimensional comparison, the misjudgment problem caused by direct comparison of non-uniform reference images in a traditional algorithm is reduced, and the comparison misjudgment rate is further reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an image acquisition processing and comparative analysis method for a private care instrument. BACKGROUND

[0002] Gynecological diseases, including vulva diseases, vaginal diseases, uterine diseases, fallopian tube diseases, and ovarian diseases, etc., in order to treat vaginal diseases, it is usually necessary to clean and care the vagina and apply medicine, that is, to wash the vagina with a private care instrument, clean the dirt in the vagina, and then apply medicine. With the development of intelligent medical devices, the existing private care instruments begin to integrate image acquisition devices (such as miniature endoscopic cameras) to assist users in observing and self-checking the affected area through visualization. This function usually relies on image comparative analysis technology: the system will suggest the user to collect images of the same affected area before and after applying medicine, and then process the two images through an algorithm to evaluate the efficacy by comparing the changes in color, texture, and swelling degree, thereby facilitating self-checking by the patient. However, when this image comparative analysis technology is applied to the handheld care instrument scenario, there are certain technical defects. Due to the limitations of the use conditions and the user's own use ability, when the patient operates the handheld care instrument, it is inevitable to introduce shaking, angle deviation, and changes in shooting distance. In addition, the lighting conditions (such as brightness and color temperature) are also difficult to keep completely consistent every time. These factors will cause the images taken before and after to have great differences in scale, angle, brightness, and color, resulting in incorrect dynamic range and unclear details, which affects the subsequent image comparison and recognition. The traditional image comparison algorithm directly compares the images obtained under different bases at the pixel level or feature level, which will misjudge the differences caused by the above external factors as changes in the disease itself, thereby producing incorrect analysis results and misleading the user's judgment. SUMMARY

[0003] To solve the above problems in the prior art, the application provides an image acquisition processing and comparative analysis method for a private care instrument, which solves the problem of incorrect dynamic range and unclear details of the acquired images caused by image shaking when the patient operates the handheld care instrument to acquire images before and after, which affects the subsequent image comparison and recognition.

[0004] The purpose of the application can be achieved by the following technical scheme: an image acquisition processing and comparative analysis method for a private care instrument, comprising the following steps: S1: Collecting the first image before applying medicine and the second image after applying medicine of the affected area through the endoscopic camera integrated with the multi-spectrum light supplement lamp group, and synchronously acquiring the six-axis attitude data when the camera takes the two images, the path distance data of the laser ranging module, and the illumination parameters of the light supplement lamp group; S2: Extracting the key feature points of the lesion area of the two images based on the SIFT feature algorithm, calculating the Euclidean distance variation of the feature points in the pixel coordinate system, and conducting jitter detection combined with the jitter threshold of the six-axis attitude data; S3: Correcting the images determined to have jitter, which includes pixel correction, scale normalization, and illumination correction, and generating the corrected first image and the corrected second image after correction; S4: Comparing and analyzing the corrected first image and the corrected second image and generating an analysis result, and outputting the therapeutic effect evaluation result through the fuzzy comprehensive evaluation model combined with the comparison and analysis result; The result includes four levels: significant improvement, slight improvement, no change, and aggravation.

[0005] Preferably, in S3, the correction process includes: S31: Pixel-level correction: calculating the jitter amplitude vector based on the feature point variation, constructing the camera motion model through the six-axis attitude data, and performing pixel re-projection on the two images to obtain the second corrected image; S32: Scale normalization: calculating the object distance ratio of the two images combined with the path distance data of the laser ranging module, and normalizing the first image and the second corrected image to the same object distance scale through the bilinear interpolation algorithm to generate the corrected first image and the corrected second image; S33: Illumination equalization: extracting the illumination parameter difference of the first image and the second image, performing brightness compensation on the corrected first image and the corrected second image, and correcting the color temperature deviation through the gray world algorithm.

[0006] Preferably, in step S31, the pixel re-projection of the two images is performed using the inverse mapping algorithm, specifically including: Calculating the camera rotation matrix and translation vector based on the six-axis attitude data, establishing the perspective transformation relationship from the second image to the first image, and filling the pixels after transformation with bicubic interpolation.

[0007] Preferably, the comparison and analysis includes: Calculating the overall similarity of the first image and the second image using the structural similarity index; Extracting the lesion area through the U-Net segmentation network, calculating the RGB channel difference in the area, calculating the texture entropy value change based on the gray level co-occurrence matrix, and calculating the area change rate of the lesion area.

[0008] Preferably, in step S2, the SIFT feature algorithm is used to filter the initial feature points of the two photographed images respectively by a preset lesion feature library, and the feature points related to the lesion are retained.

[0009] Preferably, in step S2, a nearest neighbor matching algorithm (NN matching) is used to perform similarity matching on the SIFT feature descriptors of the first image and the second image.

[0010] Preferably, in step S33, the Retinex algorithm is used to separate the illumination component and the reflection component of the image by using a multi-scale Gaussian filter to perform brightness compensation on the corrected first image and the corrected second image, and the scale factors of the Retinex algorithm are 15, 80 and 250 respectively.

[0011] Preferably, in step S4, before the comparison and analysis of the two corrected images, the corrected first image and the corrected second image are also subjected to denoising processing by Gaussian filtering combined with median filtering.

[0012] The beneficial effects of the present application are: The present method has the technical advantages of multi-dimension and high adaptability in view of the problems such as shaking, angle deviation, change of shooting distance and inconsistent illumination conditions that are inevitable when the patient holds the private care instrument. The shaking is first detected accurately by SIFT feature matching and attitude threshold, then the shaking or angle deviation is eliminated by motion model construction and pixel re-projection, scale normalization is realized by combining object distance proportion and bilinear interpolation, and the illumination deviation is corrected by relying on brightness compensation and gray world algorithm, so that the pixel position deviation, scale difference, gray mean difference and color temperature deviation of the two images are finally corrected, thereby providing the same scene data basis for subsequent comparison. On this premise, multi-dimension comparison is performed based on pixel level (area, gray mean, gray variance) and feature level (SIFT matching rate, feature point distance), so as to reduce the misjudgment problem of traditional algorithm in direct comparison of non-uniform reference images, and further reduce the comparison misjudgment rate. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to facilitate the understanding of those skilled in the art, the present application will be further described below with reference to the accompanying drawings.

[0014] Figure 1 The method steps of the analysis method of the present application are shown in the figure. DETAILED DESCRIPTION

[0015] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the specific embodiments, structures, features and effects according to the present application are described in detail below with reference to the accompanying drawings and preferred embodiments.

[0016] Please refer to Figure 1 The embodiment provides an image acquisition processing and comparative analysis method of a private care instrument, comprising the following steps: S1: Collect the first image before applying medicine and the second image after applying medicine of the affected area through the endoscopic camera integrated with the multi-spectrum light supplement lamp group, and synchronously acquire the six-axis attitude data when the camera is photographed twice, the path distance data of the laser ranging module, and the illumination parameters of the light supplement lamp group; The endoscopic high-definition camera integrated at the front end of the probe of the private care instrument is used to collect the image of the affected area. The camera is equipped with a ring-shaped multi-spectrum light supplement lamp group with adjustable brightness. The light supplement lamp adopts 3 groups of color temperature adjustable LED, which ensures the imaging clarity under different skin colors and environmental light; When collecting the first image and the second image, the illumination parameters of the multi-spectrum light supplement lamp group are collected in real time through the color temperature sensor built in the care instrument, and the six-axis attitude data when the camera is photographed twice, the path distance data of the laser ranging module, and the illumination parameters of the light supplement lamp group are synchronously stored for subsequent calling; The first photographing is within 2 hours before applying medicine treatment, and the second photographing is 24±2 hours after treatment. When photographing, the user is guided by the display screen of the care instrument to maintain the initial distance (8-12 mm) between the probe and the affected area, and is prompted to remain still for more than 3 seconds. The six-axis attitude data includes three-axis acceleration and three-axis angular velocity. The photographing distance is recorded through the laser ranging sensor, the camera moving track is generated combined with the displacement sensor, the path distance data of the camera is generated through the camera moving track, the real-time brightness and color temperature parameters of the light supplement lamp are synchronously collected, and all data and images are stored through timestamp binding; S2: Extract the key feature points of the lesion area of the two photographed images based on the SIFT feature algorithm, including: screening the initial feature points of the two photographed images through the preset lesion feature library, and retaining the feature points related to the lesion. The Euclidean distance change of the feature points in the pixel coordinate system is calculated, and the jitter detection is performed combined with the jitter threshold of the six-axis attitude data. Specifically, it includes: First, the lesion area of the first image and the second image is preprocessed: the adaptive threshold segmentation algorithm (Otsu threshold method based on image gray histogram) is used to preliminarily segment the lesion area and normal skin area, and then the Canny edge detection algorithm (the threshold range is set to 50-150) is used to optimize the lesion area boundary, excluding the interference area such as hair and stains, and finally determining the core lesion area to be analyzed; SIFT feature extraction is performed on the positioned lesion area, including the steps of: ① Construct a Gaussian scale space, the σ value range is set to 1.6-6.4, the octave layer number is set to 4-5, and the extreme points in the image, i.e. potential feature points, are detected; ②Precise positioning of extreme points, eliminating low-contrast and edge response false feature points; ③Assign a main direction to each feature point, based on the gradient direction histogram, with an angle accuracy of 10°; ④Generate a 128-dimensional feature descriptor, divide a 16x16 pixel neighborhood centered on the feature point, calculate 8 direction gradients for each 4x4 sub-block to form a 128-dimensional vector, ensuring that the feature point has scale invariance and rotation invariance; Use the nearest neighbor matching algorithm (NN matching) to perform similarity matching of the SIFT feature descriptors of the first image and the second image: calculate the descriptor Euclidean distance of the feature point pair to be matched, set a distance threshold, and filter out feature point pairs with high matching degree, i.e. effective feature point pairs, using the set distance threshold; Shake detection logic: based on the six-axis attitude data, set a shake threshold, according to the lens focal length and object distance, through the pinhole camera model, convert the maximum allowed offset of the six-axis attitude into a pixel offset threshold, the pinhole camera model is used to represent the relationship between pixel size and object distance; For each effective feature point pair, calculate its coordinate difference Euclidean distance in the pixel coordinate system (x-axis horizontal, y-axis vertical); If the proportion of feature point pairs that exceed the pixel offset threshold is ≥30% (which can be adjusted according to the actual scene, taking into account sensitivity and false detection rate), it is determined that the image has shake or angular offset and needs to be corrected; S3: Correct the image determined to have shake, the correction process includes pixel correction, scale normalization and illumination correction, generate a corrected first image and a corrected second image after correction, specifically, the correction process includes: S31: Pixel-level correction: calculate the shake amplitude vector based on the feature point change, construct the camera motion model through the six-axis attitude data, perform pixel re-projection on the two captured images to obtain a second corrected image, specifically: Use the inverse mapping algorithm to perform pixel re-projection on the two captured images, specifically including: Calculate the camera rotation matrix and translation vector based on the six-axis attitude data, establish the perspective transformation relationship from the second image to the first image, and use bicubic interpolation to fill in the transformed pixel points S32: Scale normalization: combine the path distance data of the laser ranging module to calculate the object distance ratio of the two captured images, normalize the first image and the second corrected image to the same object distance scale through the bilinear interpolation algorithm to generate a corrected first image and a corrected second image; S33: Light equalization: extract the difference of light parameters of the first image and the second image, perform brightness compensation on the corrected first image and the corrected second image, and correct the color temperature deviation through the gray world algorithm. Specifically, the Retinex algorithm can be used to separate the light component and the reflection component of the image using multi-scale Gaussian filtering to perform brightness compensation on the corrected first image and the corrected second image, and the scale factor of the Retinex algorithm is 15, 80, and 250 respectively.

[0017] S4: Perform denoising processing on the corrected first image and the corrected second image through Gaussian filtering combined with median filtering, and then perform comparative analysis on the corrected first image and the corrected second image, and generate an analysis result; The comparative analysis includes overall comparison and detail analysis, specifically: The structural similarity index is used to calculate the overall similarity of the first image and the second image; The lesion area is extracted through the U-Net segmentation network, the RGB channel difference in the area is calculated, the texture entropy value change is calculated based on the gray level co-occurrence matrix, and the area change rate of the lesion area is calculated. Combined with the comparative analysis result, the fuzzy comprehensive evaluation model outputs the efficacy evaluation result, which includes four levels of significant improvement, slight improvement, no change and aggravation.

[0018] The fuzzy comprehensive evaluation model is obtained by training a learning model using pre-acquired training data, which includes judging the diagnosis and treatment evaluation effect from the first image to the second image in each type of comparative analysis result based on the comparative analysis result and the judgment of the doctors and experts.

[0019] This method aims to address the problems of inevitable shaking, angle deviation, shooting distance change and inconsistent lighting conditions when the patient holds the private care instrument. It has the advantages of multi-dimensional and high adaptability: by synchronously collecting six-axis attitude data, laser ranging distance data and light supplement lighting parameters, SIFT feature matching and attitude threshold are used to accurately detect shaking, a motion model is constructed to eliminate shaking or angle deviation, a scale normalization is achieved by combining distance proportion and bilinear interpolation, and a light compensation and gray world algorithm is used to correct light deviation. Finally, the pixel position deviation, scale difference, gray mean difference and color temperature deviation of the two images are corrected to provide the same scene data basis for subsequent comparison. On this premise, based on multi-dimensional comparison at the pixel level (area, gray mean, gray variance) and feature level (SIFT matching rate, feature point distance), the misjudgment problem of traditional algorithms directly comparing non-uniform reference images is reduced, and the comparison misjudgment rate is further reduced. Through active compensation of handheld operation errors, the user threshold is greatly reduced, the home scene is perfectly adapted, and objective, accurate and practical quantitative support is provided for the efficacy tracking of private care.

[0020] The above merely describes preferred embodiments of the present application, and is not intended to limit the present application in any form. Although the present application has been described above with preferred embodiments, it is not intended to limit the present application, and any person skilled in the art can make some changes or modifications to the above disclosed technical content to obtain equivalent embodiments with equivalent changes, as long as the changes or modifications do not deviate from the technical solutions of the present application. Any modification, equivalent change and modification made to the above embodiments according to the technical essence of the present application still falls within the scope of the technical solutions of the present application.

Claims

1. A method for image acquisition, processing, and comparative analysis of a private care device, characterized in that: Includes the following steps: S1: The endoscope camera with integrated multispectral supplementary lighting group acquires the first image of the affected area before medication and the second image after medication, and simultaneously acquires the six-axis attitude data of the camera during the two shots, the path distance data of the laser ranging module, and the illumination parameters of the supplementary lighting group. S2: Extract key feature points of the lesion area from two images captured using the SIFT feature algorithm, calculate the change in Euclidean distance of the feature points in the pixel coordinate system, and perform jitter detection by combining the jitter threshold of the six-axis pose data. S3: The image determined to have jitter is corrected. The correction process includes pixel correction, scale normalization, and illumination correction. After the correction process, a first-correction image and a second-correction image are generated respectively. S4: Compare and analyze the first and second corrected images and generate analysis results. Combine the comparative analysis results and output the efficacy evaluation results through the fuzzy comprehensive evaluation model. The results are categorized into four levels: significant improvement, slight improvement, no change, and worsening.

2. The image acquisition, processing, and comparative analysis method for a private care device according to claim 1, characterized in that: In S3, the correction process includes: S31: Pixel-level correction: Calculate the jitter amplitude vector based on the change of feature points, construct a camera motion model through six-axis attitude data, and perform pixel reprojection on the two captured images to obtain a secondary corrected image; S32: Scale normalization: Combine the path distance data of the laser ranging module to calculate the object distance ratio of the two shots, and normalize the first image and the second corrected image to the same object distance scale through bilinear interpolation algorithm to generate the corrected first image and the corrected second image. S33: Illumination equalization: Extract the difference in illumination parameters between the first image and the second image, perform brightness compensation on the corrected first image and the corrected second image, and correct color temperature deviation through the gray world algorithm.

3. The image acquisition, processing, and comparative analysis method for a private care device according to claim 1, characterized in that: Step S31 includes using a reverse mapping algorithm to reproject pixels from the two captured images, specifically including: Based on the six-axis attitude data, the camera rotation matrix and translation vector are calculated, the perspective transformation relationship from the second image to the first image is established, and bicubic interpolation is used to fill the transformed pixels.

4. The image acquisition, processing, and comparative analysis method for a private care device according to claim 1, characterized in that: The comparative analysis includes: The structural similarity index is used to calculate the overall similarity between the first image and the second image; The U-Net segmentation network is used to extract the lesion region, calculate the RGB channel difference within the region, calculate the texture entropy change based on the gray-level co-occurrence matrix, and calculate the area change rate of the lesion region.

5. The image acquisition, processing, and comparative analysis method for a private care device according to claim 1, characterized in that: In step S2, the SIFT feature algorithm filters the initial feature points of the two captured images using a preset lesion feature library, retaining feature points related to the lesion.

6. The image acquisition, processing, and comparative analysis method for a private care device according to claim 5, characterized in that: Step S2 includes using the nearest neighbor matching algorithm (NN matching) to perform similarity matching on the SIFT feature descriptors of the first image and the second image.

7. The image acquisition, processing, and comparative analysis method for a private care device according to claim 2, characterized in that: In step S33, the Retinex algorithm is used to separate the illumination component and reflection component of the image using multi-scale Gaussian filtering, and brightness compensation is performed on the first image and the second image to be corrected. The scale factors of the Retinex algorithm are 15, 80 and 250 respectively.

8. The image acquisition, processing, and comparative analysis method for a private care device according to claim 1, characterized in that: In step S4, before comparing and analyzing the two corrected images, the first corrected image and the second corrected image are denoised by combining Gaussian filtering with median filtering.

Images