Method and device for identifying real size information of stoma based on machine vision

Abstract

The invention provides a method and device for identifying real size information of a stoma based on machine vision, and belongs to the technical field of medical assistance. The method comprises the steps of obtaining calibration parameters, receiving equipment information which is input by a user and used for shooting, calling the calibration parameters according to the equipment information, obtaining an image which is input by the user and comprises a reference substance and a stoma, and inputting the image into a machine vision fusion deep learning model which is trained by using labeling information in advance, the method comprises the following steps: dividing a reference object area and a stoma area, extracting edge contours of the reference object area and the stoma area, performing size transformation according to calibration parameters, real size information of the reference object area and the edge contours of the reference object area and the stoma area, and calculating to obtain real size information of the stoma area. According to the method, the defects of manual trimming are overcome, full-closed-loop automatic operation from image acquisition, data processing to machining is achieved, and a complete technical chain of intelligent measurement, accurate cutting and efficient adaptation is formed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical assistance, and relates to a method and device for identifying real stoma size information based on machine vision. BACKGROUND

[0002] By the end of 2024, according to incomplete statistics, the cumulative number of stoma patients in China has exceeded 1 million, and the number of patients is still increasing year by year. As a key appliance for storing human excreta, the adaptability of the stoma bag plays a decisive role in the daily care of stoma patients. However, in the existing stoma care, more reliance is placed on manual measurement and manual trimming, and due to the lack of precise measurement technology support, the stoma bag aperture after trimming often deviates from the actual stoma size. This size mismatch will directly cause excretion leakage after application, and further induce inflammation of the tissue around the stoma. In addition, in the process of frequent replacement of the stoma bag, the subjectivity of manual experience judgment is easy to cause measurement error accumulation, further increasing the risk of care, and the increased risk of care will also have a negative psychological impact on the patient to some extent, which is easy to affect the patient's attitude towards treatment. Therefore, providing a method for accurately and simply identifying the real size of the stoma is a problem to be solved. SUMMARY

[0003] The purpose of the present application is to solve the above-mentioned problems existing in the prior art, and a method for identifying real stoma size information based on machine vision is proposed.

[0004] The purpose of the present application can be achieved by the following technical scheme: a method for identifying real stoma size information based on machine vision, comprising:

[0005] Calibrating at least two and more devices with shooting function in advance to obtain calibration parameters;

[0006] Receiving device information input by a user for shooting, and calling the calibration parameters corresponding to the device according to the device information;

[0007] Obtaining an image input by the user containing a reference object and a stoma;

[0008] Inputting the image into a model of machine vision fusion deep learning trained in advance using labeling information, segmenting a reference object region and a stoma region, and then extracting an edge contour of the reference object region and an edge contour of the stoma region;

[0009] According to the calibration parameters, the real size information of the reference object region, the edge contour of the reference object region and the edge contour of the stoma region, performing size transformation, and calculating to obtain the real size information of the stoma region.

[0010] As an optional embodiment of the present application, at least two and more devices with shooting functions are calibrated in advance to obtain calibration parameters, including:

[0011] A calibration board is set up;

[0012] The calibration board is shot by the devices in a preset number and in multiple directions, and the shot images are converted to a three-dimensional space for analysis by an image processing tool;

[0013] The devices are calibrated according to the three-dimensional space positions of the devices and the three-dimensional space position of the calibration board, and an intrinsic matrix of the devices is calculated and obtained.

[0014] As an optional embodiment of the present application, the model of machine vision fusion deep learning trained by using the labeling information, includes:

[0015] Training data are obtained, wherein the training data include labels, reference object regions and stoma regions;

[0016] The labels are used to indicate the reference object regions and the stoma regions, and the model is trained by using the training data;

[0017] The model is used to identify a region to be segmented, and the region to be segmented includes reference object regions and stoma regions;

[0018] When the model is trained, the training completion degree of the deep learning model is determined according to a preset training index, and it is determined whether the training is completed according to the training completion degree.

[0019] As an optional embodiment of the present application, the training data are obtained, including:

[0020] An image set is obtained, and the images in the image set contain reference objects and stomas at the same time;

[0021] The reference objects and the stomas in the images are labeled point by point to obtain reference object regions and stoma regions;

[0022] The reference object regions and the stoma regions are represented by corresponding labels;

[0023] The reference object regions and their labels and the stoma regions and their labels in the same image are taken as a group of the training data.

[0024] As an optional embodiment of the present application, the training completion degree of the model is determined according to a preset training index, including:

[0025] The preset training index includes identification loss, identification accuracy and comprehensive performance;

[0026] determine the accuracy and generalization ability of the model according to the identification loss;

[0027] determine the stability of the model according to the identification accuracy;

[0028] comprehensively score the training ability of the model based on the identification loss and the identification accuracy, and the training is completed if the comprehensive score reaches a preset score standard.

[0029] As an optional embodiment of the present application, the accuracy and generalization ability of the deep learning model are determined according to the identification loss, including:

[0030] The identification loss includes a training loss and a verification loss;

[0031] The accuracy of the model in identifying the region to be segmented is determined according to the training loss;

[0032] The generalization ability of the model in identifying the region to be segmented is determined according to the verification loss.

[0033] As an optional embodiment of the present application, the stability of the model is determined according to the identification accuracy, including:

[0034] The identification accuracy includes precision and recall rate;

[0035] The precision is obtained by statistically comparing the positive samples obtained by the model in predicting the region to be segmented with the real position results corresponding to the region to be segmented;

[0036] The recall rate is obtained by statistically comparing the real positive samples predicted by the model in predicting the region to be segmented with the real position results corresponding to the region to be segmented;

[0037] The stability is determined according to the precision and the recall rate.

[0038] As an optional embodiment of the present application, the real size information of the stoma region is calculated and obtained according to the calibration parameter, the real size information of the reference region, the edge contour of the reference region and the edge contour of the stoma region, including:

[0039] The focal length of the device used for shooting is obtained according to the intrinsic matrix, and the real distance between the device and the stoma region is calculated and obtained according to the real size information of the reference region and the pixel size information corresponding to the edge contour of the reference region, wherein the real distance is based on the parallel relationship between the plane of the reference object and the imaging plane of the device used for shooting.

[0040] According to the real distance, the focal length and the pixel size information corresponding to the edge contour of the stoma region, real size information of the stoma region is calculated.

[0041] The application also provides a device for identifying real size information of a stoma based on deep learning, comprising:

[0042] A calibration module is configured to calibrate at least two or more devices with shooting functions in advance to obtain calibration parameters.

[0043] A parameter calling module is configured to receive device information for shooting input by a user and call calibration parameters of a corresponding device according to the device information.

[0044] A received image module is configured to acquire an image containing a reference object and a stoma input by a user.

[0045] An analyzed image module is configured to input the image into a model of machine vision fusion deep learning trained in advance using labeling information, segment a reference object region and a stoma region, and then extract an edge contour of the reference object region and an edge contour of the stoma region.

[0046] A calculated size module is configured to perform size conversion according to the calibration parameters, real size information of the reference object region, the edge contour of the reference object region and the edge contour of the stoma region to calculate real size information of the stoma region.

[0047] The application also provides an electronic device, comprising:

[0048] A processor;

[0049] A memory for storing executable instructions of the processor;

[0050] When the processor is configured to execute the executable instructions, the method for identifying real size information of a stoma based on machine vision is realized.

[0051] Compared with the prior art, the application accurately segments a stoma region through a model of machine vision fusion deep learning, makes up for the deficiency of manual pruning, and can embed calibration parameters of a calibrated shooting device and the model in an APP, and simultaneously connect a matched CNC small stoma cutting device. The application realizes full closed-loop automatic operation from image acquisition, data processing to mechanical processing, forms a complete technical chain of "intelligent measurement-precise cutting-high efficient adaptation", and provides a modern solution with innovation and practicability for the field of stoma nursing. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a method flowchart for identifying real size information of a stoma based on machine vision according to an embodiment of the application.

[0053] Figure 2 is a calibration board fixed image of an embodiment of the present application;

[0054] Figure 3 is a calibration board image calibration image of an embodiment of the present application;

[0055] Figure 4 is a calibration board image space pose image for a shooting device of an embodiment of the present application;

[0056] Figure 5 is a total average error image of an embodiment of the present application calibration;

[0057] Figure 6 is an original image of an embodiment of the present application containing a reference object and a stoma;

[0058] Figure 7 is a labeling image of an embodiment of the present application reference object area and stoma area;

[0059] Figure 8 is a device block diagram of an embodiment of the present application for identifying real size information of a stoma based on machine vision. DETAILED DESCRIPTION

[0060] The following is a specific embodiment of the present application and further describes the technical solutions of the present application in conjunction with the accompanying drawings, but the present application is not limited to these embodiments.

[0061] Embodiment 1

[0062] This embodiment proposes a method for identifying real size information of a stoma based on machine vision. A user can take a stoma image by holding a device with shooting function in a scene that makes the body feel relaxed and pleasant, upload it to an APP, wait for the APP to analyze the real size information of the stoma, and send it to a connected matching CNC small stoma cutting device to cut the stoma bag material. As shown in the figure, the method comprises: Figure 1

[0063] S1, calibrate at least two and more devices with shooting function in advance to obtain calibration parameters;

[0064] S2, receive user input device information for shooting, and retrieve calibration parameters of the corresponding device according to the device information;

[0065] S3, obtain an image input by a user containing a reference object and a stoma;

[0066] S4, input the image into a model trained using labeling information in advance, which is a fusion of machine vision and deep learning, to segment the reference object area and the stoma area, and then extract the edge contour of the reference object area and the edge contour of the stoma area;​

[0067] S5, according to the calibration parameter, the real size information of the reference region, the edge contour of the reference region and the edge contour of the stoma region, performing size transformation to obtain the real size information of the stoma region.

[0068] In this embodiment, the reference object needs to be placed near the stoma region before shooting. The patient himself or his accompanying person can use a device with shooting function to shoot. However, shooting needs to consider many factors, such as the placement position of the reference object needs to be determined according to the skin wrinkle and other conditions to make the reference object and the stoma basically at the same level. Moreover, the imaging plane of the device camera should be parallel to the plane of the reference object. However, even if the above conditions are met, the shooting effect of the image is different due to the different shooting habits of each person. On the one hand, although each person tries to make the imaging plane parallel to the plane of the reference object before shooting, there will still be errors due to subjective judgment. On the other hand, the real distance between the device and the reference object is inconsistent when each user shoots, for example, some users are used to shooting at a relatively close distance between the device and the reference object, and some users are used to shooting at a relatively far distance between the device and the reference object. In short, the shooting habits of users will also affect the image effect. Therefore, the device needs to be calibrated in advance, and the purpose is to obtain its fixed original factory shooting parameters. According to the original factory shooting parameters, combined with the pixel size of the edge contour of the reference region and the edge contour of the stoma region in the image uploaded by the user, the real distance between the device and the stoma is calculated to improve the accuracy of identifying and segmenting the real size information of the stoma region. At least two devices with shooting function sold on the market can be obtained in advance, which can be the same type but different models of mobile phones, or different types of devices. The device is calibrated to obtain the corresponding original factory shooting parameters as the calibration parameter. In this way, when the user uses the APP for the first time, the device information such as the device model is input first, and the corresponding calibration parameter can be retrieved according to the device model to analyze the image uploaded by the subsequent user. In fact, in the actual application process, the device includes a handheld mobile device, such as a mobile phone, a camera, a sports camera, etc., and can also include a temporary fixed device, such as a tool with a fixing function, such as a tripod, to fix the device for shooting. In summary, the devices with shooting function on the market are calibrated, so that the corresponding calibration parameters can be retrieved when different device information is input by different users.

[0069] After that, the user uses the device to take an image, and it should be noted that the reference object needs to be placed on the skin surface near the stoma when taking the image. The selected skin surface should be as smooth as possible, and the reference object should be made of a flexible material to alleviate the impact of skin undulations, ensure that the reference object is in the same imaging plane as the stoma, and reduce the impact of the patient's uneven skin causing the image not to be in the same plane. In this embodiment, the reference object is circular. Compared with other shapes of reference objects, such as equilateral triangles, regular polygons, and hexagons, the arbitrary edge point spacing of the circular reference object profile is the diameter, which has a clear uniqueness. The pixel spacing of the profile of other shapes of reference objects is not a fixed value, resulting in a higher complexity of obtaining the pixel size than the circular reference object.

[0070] After the user takes the image, the image is uploaded to the APP. The APP in this embodiment has a pre-trained machine vision fusion deep learning model. The image is input into the model, and the model identifies whether there is a reference object area and a stoma area. The reference object area and the stoma area are labeled and segmented out. Further, the edge profile of the reference object area and the edge profile of the stoma area are extracted using morphological operations or Canny operators. According to the edge profile, the width and height of the reference object area and the stoma area in the digital image are determined, i.e., the pixel size in pixels. Combined with the known physical real size of the reference object, the physical real size information of the stoma area is obtained through size conversion calculation.

[0071] The physical real size information is then sent to a matching CNC (Computer Numerical Control) small cutting device. According to the physical real size information of the stoma area, a suitable stoma bag is cut, improving the matching and fitting of the stoma bag aperture and the actual stoma size. It should be noted that the CNC small cutting device is an existing product, and matching means pairing with the APP. For users, it provides convenience. They only need to prepare the reference object, the APP, and the matching CNC small cutting device to complete the pre-work of stoma care. After cutting the stoma bag, the user performs subsequent care, and the user can be the patient himself or a caregiver. To some extent, it improves the patient's willingness to actively treat.

[0072] Preferably, at least two and more devices with shooting functions are calibrated in advance to obtain calibration parameters, including:

[0073] A calibration board is set up.

[0074] The device is used to take a preset number of images of the calibration board from multiple directions, and the images are converted to a three-dimensional space for analysis by an image processing tool.

[0075] The device is calibrated based on its three-dimensional spatial position and the calibration plate's three-dimensional spatial position, and the intrinsic parameter matrix corresponding to the device is calculated.

[0076] In this embodiment, the calibration plate is set by marking 49 identical dots (7x7) within a square frame. The calibration plate is printed on A4 paper without scaling and then fixed flat to the wall. Figure 2 As shown. The calibration board can also be set with 81 identical dots in a 9x9 grid within a box; it can also be in other patterns, such as a checkerboard. Regardless of the size and pattern of the calibration board, after setting it up, print the calibration board on an A4 sheet of paper without scaling and fix it flat to the wall.

[0077] This embodiment uses an iPhone 13 as an example, taking approximately 20 images of the calibration board from different angles. To improve calibration accuracy, the number of images can be increased; the preset number depends on the actual situation. The captured calibration board images are imported into an image processing tool. In this embodiment, the image processing tool used is the Matlab Image Processing Toolbox. During the import process, relevant parameters of the calibration board image need to be set. These parameters include the center distance, number of rows, and number of columns of the black circular area of ​​the calibration board. After completing the parameter configuration, the calibration operation is performed, obtaining the following calibration result: the calibrated calibration board image, as shown below. Figure 3 As shown; the spatial pose of the calibration board image relative to the iPhone 13 camera, such as Figure 4 As shown; the overall average error is used to evaluate calibration accuracy, such as... Figure 5 The overall mean error shown is OverallMean Error = 0.58 pixels.

[0078] analyze Figure 4 During shooting, the calibration board was fixed to the wall. When converting to a spatial coordinate system for analysis, the spatial position of the iPhone 13 was fixed, according to... Figure 3 The calibrated images are transformed into images of different poses for the iPhone 13 with a fixed spatial position. That is, during shooting, the position of the iPhone 13 is dynamic due to shooting from different angles and distances, while the position of the calibration board image fixed on the wall is fixed. However, in the spatial coordinate system, the position of the iPhone 13 is set to be fixed, while the position of the calibration board image is dynamically changing. Then, based on the spatial coordinate system, the 3D spatial position of the iPhone 13 and the 3D spatial position of each calibration board image are obtained, and camera calibration is performed to obtain the intrinsic parameter matrix K of the iPhone 13. The intrinsic parameter matrix K is a 3*3 matrix, and its first row elements are (F... x ,0,C x The second row contains elements (0, F). y Cy ), the third row element is (0, 0, 1). Wherein F x is the focal length of the horizontal direction of the shooting device, in pixels; F y is the focal length of the vertical direction of the shooting device, in pixels; wherein C x is the x-coordinate of the center point of all calibration board images, wherein C y is the y-coordinate of the center point of all calibration board images. The obtained intrinsic matrix of iPhone 13 is shown in Figure 5 , and the focal length f thereof can be determined as: F x = 3.0888 * 10 3 pixel, F y = 3.0893 * 10 3 pixel. Similarly, by using other models of shooting devices, the above calibration process is performed to obtain the original calibration parameters focal length f of the device. Since there are various models of devices, it is necessary to calibrate more than one device. It should be noted that the device focal length f is a fixed parameter provided by the manufacturer, and the calibration (especially with distortion correction) is to obtain the actual intrinsic matrix of the device, which may contain individual differences, and thus the specific focal length f is determined through the above calibration process. After the calibration is completed, when the user inputs the shooting device information as iPhone 13, the intrinsic matrix including the focal length f can be called out.

[0079] Preferably, the model of machine vision fusion deep learning trained by using the labeling information in advance comprises:

[0080] acquire training data, wherein the training data comprises labels, reference object regions and stoma regions;

[0081] the labels are used to indicate the reference object regions and the stoma regions, and the model is trained by using the training data; wherein the model is used to identify a region to be segmented, and the region to be segmented comprises reference object regions and stoma regions;

[0082] when training the model, a training completion degree of the model is determined according to a preset training index, and it is determined whether the training is completed according to the training completion degree.

[0083] The training data contains the stoma region, the reference region and their corresponding labels. The training data is divided into a training set and a validation set, and the ratio is determined by the training personnel. In this embodiment, the model used is the Yolov8s-seg model. The training data in the training set is used to train the Yolov8s-seg model, so that the model automatically learns the rules of labeling the reference and the stoma from the training data. Therefore, when the model is applied, it can predict and identify the stoma and the reference, and label the stoma region and the reference region when receiving the image uploaded by the user. The training data in the validation set is used to verify the learning ability of the model. During the whole training process of the model, the number of iterations and related parameters of the model are adjusted before each batch of training, and whether the model after this training reaches the best performance is determined according to the preset training index. If it reaches the best performance, the training is ended and the optimal training weight file is obtained. If it does not reach the best performance, the training is continued using the training data.

[0084] Preferably, obtaining the training data comprises:

[0085] obtaining an image set, wherein each image in the image set contains a reference and a stoma;

[0086] labeling the reference and the stoma in the image point by point to obtain a reference region and a stoma region;

[0087] using corresponding labels to represent the reference region and the stoma region;

[0088] using the reference region and its label and the stoma region and its label in the same image as a group of training data.

[0089] The image set is a set of tens of millions of original images containing stomas and references. These original images are input into a labeling software. For each original image, as shown in Figure 6 , the circular reference and the stoma are labeled point by point, and the circular reference region and the stoma region are accurately outlined with multiple points. The processed original image is as shown in Figure 7 . Finally, 0 is used to represent the circular reference region, 1 is used to represent the stoma region, and each outlined region is converted into a.txt text format, and the corresponding label is used as a group of training data for model training.

[0090] Preferably, the training completion degree of the machine learning model is determined according to a preset training index, comprising:

[0091] The preset training index includes identification loss, identification accuracy and comprehensive performance.

[0092] The accuracy and generalization ability of the model are determined according to the identification loss.

[0093] determine the stability of the model recognition according to the recognition accuracy;

[0094] Based on the recognition loss and the recognition accuracy, the training ability of the model is comprehensively scored, and if the comprehensive score reaches a preset score standard, the training is complete.

[0095] The recognition loss reflects the prediction accuracy of the model on the training set, and reflects the learning performance of the model through the validation set. In simple terms, whether the model is learning the rules of the training data, the learning ability of the model is verified through the validation set to ensure that there is no overfitting, etc.

[0096] The recognition accuracy can reflect the stability of the model recognition as a whole. In simple terms, whether the model recognition is accurate and whether there is a missed recognition situation.

[0097] The recognition loss and the recognition accuracy are combined to comprehensively score the deep learning model. In simple terms, how powerful the learning ability of the model is, and the training state of the model is determined by the comprehensive score.

[0098] When the model is preliminarily trained by the recognition loss, the training state of the model based on the training in place can be determined by mAP(mean average precision). mAP is the core score of target detection in the model, which can be understood as the comprehensive performance of precision and recall rate under different difficulties. For example, mAP50 is the comprehensive score under the condition that the IoU(target box overlap degree) is 50%, and mAP50-95 is the average score under different standards from IoU 50% to 95%.

[0099] The comprehensive performance is converted into a curve for analysis. If the curve shows an upward and then stable trend, it means that the overall target detection ability of the model has been "fully trained", and the recognition performance tends to be stable whether it is a relaxed standard(IoU is 50%) or a strict standard(IoU is 95%), which represents a very good training state.

[0100] Preferably, the accuracy and generalization ability of the model recognition are determined according to the recognition loss, comprising:

[0101] The recognition loss includes training loss and validation loss;

[0102] The accuracy of the model in recognizing the region to be segmented and its edge contour is determined according to the training loss;

[0103] The generalization ability of the model in recognizing the region to be segmented and its edge contour is determined according to the validation loss.

[0104] The recognition loss can be understood as the difference between the model prediction result and the true result. The recognition loss is divided into training loss and validation loss. When the training loss continues to decrease, it indicates that the model is learning the law of the annotation information in the training data and is more and more "understanding" the training data, and the prediction is more and more accurate. For example, train / box_los decreases from more than 1 to about 0.2, which indicates that the model is more and more able to predict the position of the reference object region and the stoma region.

[0105] The validation loss is the performance of the model on the validation set. The validation set is new data that the model has not seen before. When the performance of the recognition data in the validation set is stable, it indicates that the model has good generalization ability for recognizing the reference object region and the stoma region, and has learned the general law of the annotation information, rather than learning through overfitting, i.e. memorizing the training set. When the performance of the model on the validation set is stable, it means that the model has been trained.

[0106] Preferably, the stability of the deep learning model is determined according to the recognition accuracy, comprising:

[0107] The recognition accuracy includes precision and recall rate;

[0108] The precision is obtained by statistically comparing the positive samples obtained by the model predicting the region to be segmented with the true position result corresponding to the region to be segmented;

[0109] The recall rate is obtained by statistically comparing the true positive samples predicted by the model with the true position result corresponding to the region to be segmented;

[0110] The stability is determined according to the precision and the recall rate.

[0111] The positive samples predicted by the model include true positives and false positives, and the false positives are also called false positives. Specifically, the model predicts 100 stoma regions, of which 90 are true stoma regions, so the 90 true positives are true positives, and the remaining 10 are deviated from the true stoma regions, so the 10 false positives are false positives. The precision of the model prediction positive sample is 90%.

[0112] The true positive samples predicted by the model also include true positives and false positives, and the false positives are also called false positives. Specifically, for example, there are 100 true positive samples of stoma regions, and the model finds 80 stoma regions from them, so the recall rate is 80%.

[0113] When the precision and recall rate rapidly rise to be stable, it indicates that the model has just started to "make rapid progress", and soon learns how to accurately find the target and recognize the wrong target, until the learning ability reaches a bottleneck and is basically stable.

[0114] Preferably, according to the calibration parameter, the real size information of the reference object region, the edge profile of the reference object region and the edge profile of the stoma region, a size transformation is performed to obtain the real size information of the stoma region, including:

[0115] According to the intrinsic matrix, the focal length of the device used by the user for shooting is obtained, and according to the real size information of the reference object region and the pixel size information corresponding to the edge profile of the reference object region, the real distance between the device and the stoma region is calculated and obtained, wherein the real distance is based on the parallel relationship between the plane of the reference object and the imaging plane of the device used for shooting.

[0116] According to the real distance, the focal length and the pixel size information corresponding to the edge profile of the stoma region, the real size information of the stoma region is calculated and obtained.

[0117] In this embodiment, the real size information of the circular reference object is a fixed value, and the diameter is 20 mm. After the focal length f is obtained, the edge profile of the circular reference object region is extracted according to the morphological operation or Canny operator, and the pixel coordinate value thereof can be obtained, and the corresponding pixel size is calculated. According to the principle of spatial triangle similarity, the following conversion relationship is established: f / D=R pixel / R real , wherein f is the focal length; D is the real distance from the device to the stoma plane; R pixel is the pixel diameter of the circular reference object, which is obtained according to the pixel size; and R real is the diameter of the circular reference object, which is 20 mm. According to the above formula, the real distance D from the device to the stoma plane can be derived.

[0118] Again, according to the principle of spatial triangle similarity, the following conversion relationship is established: f / D=W pixel / W real , wherein W pixel is the pixel size calculated according to the pixel coordinate value of the edge profile of the stoma region extracted by the morphological operation or Canny operator. After the real distance D from the device to the stoma plane is calculated, the real size information of the stoma region is derived according to the above formula.

[0119] In this step, the edge detection is enhanced by morphological operation or Canny operator, and the physical distance from the camera of the device to the stoma plane is measured by combining the principle of triangle similarity, and finally the high-precision conversion from the pixel coordinate of the edge profile of the stoma to the physical coordinate in millimeter level is realized.

[0120] The model accurately identifies and segments the reference object area and the stoma area, and obtains the edge profile of the reference object area and the edge profile of the stoma area according to morphological operation or Canny operator, accurately calculates the physical real size information of the stoma area, makes up for the deficiency of manual pruning, and can also embed the calibration parameters of the pre-calibrated device and the model in the APP, and connect the matching CNC small stoma cutting device. For the user, only the device needs to be used for shooting, ensuring that the reference object and the stoma are located on the same horizontal plane and the imaging plane of the device is parallel to the reference object plane and shooting, uploading the device information and the photographed image to the APP, waiting for the cutting to be completed, and the user only needs to click the four function buttons on the APP page in turn, so that the stoma measurement and cutting process can be completed within 3 to 5 minutes. For the APP, the image uploaded by the user is taken as the input of the model, the model performs region segmentation, and then the edge profile is extracted through morphological operation or Canny operator to obtain the pixel coordinates. The APP transmits the pixel coordinates of the stoma area to the Arduino single-chip microcomputer, the single-chip microcomputer drives the CNC Shield control panel, accurately controls the three-axis linkage, so as to realize the millimeter-level point-to-point motion of the cutting tool, and finally completes the personalized customization of the stoma bag bottom plate. For the CNC small stoma cutting device, after receiving the pixel coordinates, the cutting instruction is executed until the cutting is completed. Realize the full closed-loop automatic operation from image acquisition, data processing to mechanical processing, form the complete technical chain of "intelligent measurement-accurate cutting-high efficiency adaptation", realize the millimeter-level accurate matching of the stoma shape, and significantly shorten the cutting time to 20% of the traditional method, so as to greatly reduce the leakage risk and the incidence of skin complications, and provide a modern solution with innovation and practicality for the field of stoma care.

[0121] Embodiment 2

[0122] Based on the same principle as the foregoing method, an apparatus 100 for identifying the real size information of the stoma based on machine vision is also proposed, as shown in Figure 8 , which comprises:

[0123] The calibration module 110 is used for pre-calibrating at least two and more devices with shooting function to obtain calibration parameters;

[0124] The parameter calling module 120 is used for receiving the device information input by the user for shooting, and calling the calibration parameters of the corresponding device according to the device information;

[0125] The image receiving module 130 is used for acquiring the image input by the user containing the reference object and the stoma;

[0126] The image analysis module 140 is configured to input the image into a model of machine vision fusion deep learning which is trained using the labeling information in advance, segment a reference object region and a stoma region, and then extract an edge contour of the reference object region and an edge contour of the stoma region.

[0127] The size calculation module 150 is configured to perform size transformation according to the calibration parameter, the real size information of the reference object region, the edge contour of the reference object region and the edge contour of the stoma region, and calculate the real size information of the stoma region.

[0128] Embodiment 3

[0129] Further, an electronic device is provided, comprising:

[0130] a processor;

[0131] a memory for storing processor-executable instructions;

[0132] The processor is configured to implement the method for identifying real size information of a stoma based on machine vision according to the executable instructions.

[0133] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications will also change accordingly.

[0134] In addition, it should be noted that the descriptions such as "first", "second", "one" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the technical features indicated or the number of the technical features indicated. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically limited. The terms "connection", "fixation" and the like should be understood broadly, for example, "fixation" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be direct connection, or indirect connection through an intermediate medium; can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise specifically limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0135] In addition, the technical solutions among various embodiments of the present application can be combined with each other, but it must be based on that a person skilled in the art can realize, when the combination of the technical solutions appears contradictory or cannot be realized, it should be considered that the combination of the technical solutions does not exist, and is not within the protection scope required by the present application.

[0136] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art can make various modifications or supplements to the described specific embodiments or replace them with similar ways, but will not deviate from the spirit of the present application or exceed the scope defined by the appended claims.

Claims

1. A method for identifying stoma true size information based on machine vision, characterized in that, The method comprises the following steps: Calibrating at least two or more devices with shooting function in advance to obtain calibration parameters; Receiving device information input by a user for shooting, and calling calibration parameters of corresponding devices according to the device information; Obtaining an image input by the user, the image containing a reference object and a stoma; Inputting the image into a model of machine vision fusion deep learning trained in advance using annotation information, segmenting a reference object region and a stoma region, and then extracting an edge contour of the reference object region and an edge contour of the stoma region; According to the calibration parameters, real size information of the reference object region, the edge contour of the reference object region and the edge contour of the stoma region, performing size transformation to calculate real size information of the stoma region.

2. The method of identifying stoma true size information based on machine vision according to claim 1, wherein, The method of calibrating at least two or more devices with shooting function in advance to obtain calibration parameters comprises the following steps: Setting a calibration board; Shooting the calibration board in multiple directions by using the devices in a preset number, and converting the shot images into a three-dimensional space for analysis by using an image processing tool; Calibrating according to the three-dimensional space position of the devices and the three-dimensional space position of the calibration board to calculate an intrinsic matrix of the devices.

3. The method of identifying stoma true size information based on machine vision according to claim 1, wherein, The model of machine vision fusion deep learning trained in advance using annotation information comprises the following steps: Obtaining training data, wherein the training data comprises labels, a reference object region and a stoma region; The labels are used to indicate the reference object region and the stoma region, and the model is trained using the training data; The model is used to identify a region to be segmented, and the region to be segmented comprises the reference object region and the stoma region; When training the model, the training completion degree of the model is determined according to a preset training index, and it is determined whether the training is completed according to the training completion degree.

4. The method of identifying stoma true size information based on machine vision according to claim 3, characterized in that, The method of obtaining training data comprises the following steps: Obtaining an image set, wherein the images in the image set contain a reference object and a stoma; Annotating the reference object and the stoma in the images point by point to obtain a reference object region and a stoma region; Using corresponding labels to represent the reference object region and the stoma region; Taking the reference object region and the label thereof, and the stoma region and the label thereof in the same image as a group of training data.

5. The method of identifying stoma true size information based on machine vision according to claim 3, characterized in that, The method of determining the training completion degree of the model according to a preset training index comprises the following steps: The preset training index comprises identification loss, identification accuracy and comprehensive performance; The accuracy and generalization ability of the model are determined according to the identification loss; The stability of the model is determined according to the identification accuracy; The training ability of the model is comprehensively scored based on the identification loss and the identification accuracy, and if the comprehensive score reaches a preset score standard, the training is completed.

6. The method of identifying stoma true size information based on machine vision according to claim 5, characterized in that, The method of determining the accuracy and generalization ability of the model according to the identification loss comprises the following steps: The identification loss comprises training loss and verification loss; The accuracy of the model in identifying the region to be segmented is determined according to the training loss; The generalization ability of the model in identifying the region to be segmented is determined according to the verification loss.

7. The method of identifying stoma true size information based on machine vision according to claim 5, characterized in that, The method of determining the stability of the model according to the identification accuracy comprises the following steps: The recognition accuracy includes precision and recall rate; The precision is obtained by statistically comparing the positive sample obtained by the model predicting the region to be segmented with the real position result corresponding to the region to be segmented; The recall rate is obtained by statistically comparing the real positive sample of the model predicting the region to be segmented with the real position result corresponding to the region to be segmented; The stability capability is determined according to the precision and the recall rate.

8. The method of identifying stoma true size information based on machine vision according to claim 2, characterized in that, According to the calibration parameter, the real size information of the reference object region, the edge contour of the reference object region and the edge contour of the stoma region, size transformation is performed to obtain the real size information of the stoma region, including: According to the intrinsic matrix, the focal length of the device used for shooting is obtained, and according to the real size information of the reference object region and the pixel size information corresponding to the edge contour of the reference object region, the real distance between the device and the stoma region is calculated and obtained, wherein the real distance is based on the parallel relationship between the plane of the reference object and the imaging plane of the device used for shooting; According to the real distance, the focal length and the pixel size information corresponding to the edge contour of the stoma region, the real size information of the stoma region is calculated and obtained.

9. An apparatus for identifying stoma true size information based on machine vision, characterized in that, It includes: A calibration module is used to calibrate at least two and more devices with shooting function in advance to obtain calibration parameters; A parameter calling module is used to receive device information input by a user for shooting, and call calibration parameters of the corresponding device according to the device information; An image receiving module is used to obtain an image input by a user containing a reference object and a stoma; An image analyzing module is used to input the image into a model of machine vision fusion deep learning trained in advance using labeling information, segment the edge contour of the reference object region and the edge contour of the stoma region, and then extract the edge contour of the reference object region and the edge contour of the stoma region; A size calculating module is used to perform size transformation according to the calibration parameter, the real size information of the reference object region, the edge contour of the reference object region and the edge contour of the stoma region to obtain the real size information of the stoma region.

10. An electronic device, comprising: It includes: A processor; A memory for storing processor executable instructions; When the processor is configured to execute the executable instructions, the method for identifying real size information of a stoma based on machine vision according to any one of claims 1-8 is realized.

Images