A bone disc replacement method and system based on image recognition

Abstract

The present invention discloses a bone dish replacement method and system based on image recognition, belonging to the field of image processing technology, and the method comprises: obtaining a desktop image; filtering the desktop image based on adaptive smoothing filtering; segmenting the bone dish image from the filtered desktop image based on a neural network; introducing a Canny operator to detect the bone dish outline in the bone dish image and extract the bone dish area; calculating the contour integrity of the bone dish outline; binarizing the bone dish area; calculating the total number of white pixels in the bone dish area and the proportion of white pixels at the edge; calculating the bone dish filling degree according to the contour integrity of the bone dish outline and the total number of white pixels in the bone dish area and the proportion of white pixels at the edge; when the bone dish filling degree is greater than the preset filling degree, replacing the corresponding bone dish. It can provide real-time and continuous monitoring, realize timely replacement of bone dishes, improve service efficiency, and keep the table clean.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a bone disc replacement method and system based on image recognition. Background Art

[0002] As people's living standards improve, the frequency of dining in restaurants increases. Currently, people mainly place the leftovers from dining in bone dishes. As dining progresses, the leftover food in the bone dishes will accumulate more and more, affecting the dining experience.

[0003] At present, in order to solve the problem that the residual food stored in the bone dishes will accumulate and affect the dining experience, the main method adopted is manual monitoring by waiters. The bone dishes are replaced uniformly for diners in the middle of the meal. This is time-consuming and labor-intensive, and it is unable to provide real-time and continuous manual monitoring. It increases the workload of waiters during busy periods, making it difficult to respond to all table needs in time and it is difficult to find the right time to replace the bone dishes on each table in a timely manner. Summary of the invention

[0004] In order to solve the technical problems that the current method of manual monitoring by waiters requires uniformly replacing bone plates for diners in the middle of a meal, which is time-consuming and labor-intensive, and cannot provide real-time and continuous manual monitoring, increases the workload of waiters during busy hours, makes it difficult to respond to all table needs in a timely manner, and is difficult to find the right time to replace the bone plates on each table in a timely manner, the present invention provides a bone plate replacement method and system based on image recognition.

[0005] First aspect

[0006] The present invention provides a bone disc replacement method based on image recognition, comprising:

[0007] S1: Get desktop image;

[0008] S2: performing filtering processing on the desktop image based on adaptive smoothing filtering;

[0009] S3: Based on the neural network, the bone disc image is segmented from the filtered desktop image;

[0010] S4: introducing the Canny operator to detect the bone disc contour in the bone disc image and extract the bone disc area;

[0011] S5: calculating the contour completeness of the bone disc contour;

[0012] S6: performing binarization processing on the bone disc region;

[0013] S7: Calculate the total number of white pixels in the bone disc area and the proportion of white pixels at the edge;

[0014] S8: Calculating the bone disc filling degree according to the outline completeness of the bone disc outline, the total number of white pixels in the bone disc area, and the proportion of white pixels at the edge;

[0015] S9: When the filling degree of the bone dish is greater than a preset filling degree, the corresponding bone dish is replaced.

[0016] Second aspect

[0017] The present invention provides a bone disc replacement system based on image recognition, comprising a processor and a memory for storing processor executable instructions; the processor is configured to call the instructions stored in the memory to execute the bone disc replacement method based on image recognition in the first aspect.

[0018] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0019] In the present invention, image recognition is used to automatically identify whether the residual food in the bone dish is full enough. When the filling degree of the residual food reaches a preset condition, the bone dish is replaced. There is no need for the waiter to monitor the status of the bone dish all the time during the meal, which saves time and effort, can provide real-time and continuous monitoring, and respond to all table needs in time, so as to realize timely replacement of bone dishes, improve service efficiency, keep the table clean, and enhance the dining experience of diners. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The preferred implementation modes will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present invention.

[0021] Figure 1 It is a flow chart of a bone disc replacement method based on image recognition provided by the present invention.

[0022] Figure 2 It is a schematic diagram of a bone disc image provided by the present invention.

[0023] Figure 3 It is a structural schematic diagram of a bone disc replacement system based on image recognition provided by the present invention. DETAILED DESCRIPTION

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation methods of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0025] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0026] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] In this document, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection. It can be a mechanical connection or an electrical connection. It can be directly connected or indirectly connected through an intermediate medium, and it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0028] In addition, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0029] Example 1

[0030] In one embodiment, the reference specification Figure 1 , showing a schematic flow chart of a bone disc replacement method based on image recognition provided by the present invention.

[0031] The present invention provides a bone disc replacement method based on image recognition, comprising:

[0032] S1: Get the desktop image.

[0033] Specifically, a camera can be installed on the top of the restaurant to capture images of the tabletop.

[0034] S2: Filter the desktop image based on adaptive smoothing filtering.

[0035] Among them, the adaptive smoothing filter is a new filtering method provided by the present invention, which can dynamically adjust the filtering degree according to the local characteristics of the image, so as to achieve smoothing the image while retaining the edge information of the image.

[0036] In a possible implementation, S2 specifically includes sub-steps S201 to S203:

[0037] S201: Calculate the gradient component of the desktop image:

[0038]

[0039] Among them, G x represents the horizontal gradient component, G y represents the vertical gradient component, (x, y) represents the pixel coordinates, and f represents the desktop image.

[0040] It should be noted that the gradient component refers to the rate of change of the grayscale of an image at a certain point, that is, the spatial variation of the grayscale. The gradient component is often used to represent the strength and direction of the edge or texture at each position in the image.

[0041] S202: Calculate the filter weight value of each pixel:

[0042]

[0043] Among them, w represents the filter weight value, exp represents the exponential function with e as the base, and h represents the edge retention amplitude coefficient.

[0044] Among them, those skilled in the art can set the size of the edge retention amplitude coefficient h according to actual conditions, and the present invention does not limit it.

[0045] The introduction of the edge preservation amplitude coefficient h can adjust the trade-off between smoothing and edge preservation, which makes the filtering process more flexible and can adapt to different scenarios and requirements.

[0046] In the present invention, the filter weight value is calculated by gradient information, so that the degree of filtering can be adaptively adjusted in different areas. In areas with larger gradients, the weight is smaller to achieve retention near the edge; in areas with smaller gradients, the weight is larger to achieve smoothing processing, thereby enhancing the adaptability of the filter.

[0047] S203: Filter the desktop image according to the filter weight value of each pixel point:

[0048]

[0049] Where t represents the number of iterations, f t+1 represents the desktop image after t+1 filtering, f t Represents the desktop image after t times of filtering.

[0050] In the present invention, multiple iterations of filtering processes are performed to help remove noise in the image, thereby improving the image quality.

[0051] S3: Based on the neural network, the bone disc image is segmented from the filtered desktop image.

[0052] In a possible implementation, S3 specifically includes sub-steps S301 to S304:

[0053] S301: Convert the filtered original desktop image into an RGB image, an HSV image and a Lab image respectively.

[0054] It should be noted that RGB, HSV (Hue, Saturation, Value) and Lab (CIELAB) are three commonly used color spaces, which have different applications and advantages in image processing and computer vision. In the RGB color space, the color of an image consists of three color channels: red (R), green (G) and blue (B). The color of each pixel can be represented by a triplet (R, G, B), where the value range of each component is usually 0 to 255. The HSV color space represents color as three components: hue (Hue), saturation (Saturation) and lightness (Value). The Lab color space is a color representation method based on human visual perception, including three components: brightness (L, from black to white), a-axis (from green to red) and b-axis (from blue to yellow). In the Lab color space, the perceived distance of colors is more uniform, so it is more suitable for some applications that need to consider differences in color perception.

[0055] S302: Obtain semantic segmentation results of RGB images, HSV images, and Lab images through the DeepLabV3+ network.

[0056] Among them, DeepLabV3+ (DeepLab version 3plus) is a deep learning model for image semantic segmentation. It is the latest version of the DeepLab series. DeepLabV3+ uses dilated convolution, also known as expanded convolution, to increase the receptive field of the convolutional neural network. This helps the model better capture the global information in the image, especially for semantic segmentation tasks, where global information is very important for correctly classifying each pixel. By adopting dilated convolution kernels of different sizes, DeepLabV3+ builds a pyramid structure that can capture multi-scale features at the same time. This helps the model to more accurately segment objects and structures of different sizes. By separating the convolution operation, the number of parameters and the amount of calculation can be reduced while maintaining good performance. DeepLabV3+ introduces a decoder module for better processing of the details and edges of the segmentation results. This helps to improve the accuracy of segmentation, especially for the recognition of object edges. At the same time, DeepLabV3+ uses spatial pyramid pooling to capture the multi-scale features of the image, allowing the model to better adapt to objects of different sizes and shapes.

[0057] Semantic segmentation is a task in the field of computer vision, whose goal is to assign a label to each pixel in an image to indicate which semantic category the pixel belongs to in the image. The result of semantic segmentation is a label map with the same size as the input image, where each pixel corresponds to a semantic category. Through the semantic segmentation results, training is performed on a large-scale labeled dataset to learn pixel-level semantic information.

[0058] S303: By wavelet transform, the semantic segmentation results of the RGB image, the HSV image and the Lab image are fused to obtain a fused segmentation image.

[0059] Among them, wavelet transform is a mathematical tool used to decompose signals or images into components of different frequencies. It has the ability to provide local information in both the time domain and the frequency domain, so it is widely used in image processing. Wavelet transform has the characteristics of multi-scale analysis and can capture both details and overall features in an image.

[0060] In a possible implementation, sub-step S303 specifically includes S3031 to S3034:

[0061] S3031: Decompose low-frequency components and high-frequency components from the semantic segmentation results of the RGB image, the HSV image, and the Lab image through wavelet transform.

[0062] In the present invention, wavelet transform extracts information of an image at different frequencies through multi-scale decomposition, which helps to capture details and overall features in the image.

[0063] S3032: For the low-frequency components of the semantic segmentation results of the RGB image, HSV image, and Lab image, a weighted average fusion rule is used for fusion:

[0064] F mix,l (x,y)=β1F RGB,l (x,y)+β2F HSV,l (x,y)+β3F Lab,l (x,y)

[0065] Among them, F mix,l Indicates the fusion of low-frequency components, F RGB,l represents the low-frequency component in the semantic segmentation result of the RGB image, β1 represents the weight coefficient of the RGB image, and F HSV,l represents the low-frequency component in the semantic segmentation result of the HSV image, β2 represents the weight coefficient of the HSV image, and F Lab,l It represents the low-frequency component in the semantic segmentation result of the Lab image, and β3 represents the weight coefficient of the Lab image.

[0066] Among them, those skilled in the art can set the weight coefficient β1 of the RGB image, the weight coefficient β2 of the HSV image, and the weight coefficient β3 of the Lab image according to actual conditions, and the present invention does not limit this.

[0067] In the present invention, the low-frequency component adopts the weighted average fusion rule, taking into account the weights of RGB, HSV, and Lab images. This weighted average takes into account the contribution of each color channel, making the low-frequency component more representative. At the same time, the low-frequency component usually contains the global information and overall structure of the image. Through weighted average fusion, the global information is more comprehensively considered. This is helpful for a better understanding of the overall semantic structure of the image for semantic segmentation tasks.

[0068] S3033: For the high-frequency components of the semantic segmentation results of the RGB image, HSV image, and Lab image, the maximum fusion rule is used for fusion:

[0069] F mix,h (x,y)=max[F RGB,h (x,y),F HSV,h (x,y),F Lab,h (x,y)]

[0070] Among them, max[] means taking the maximum value, F RGB,h Represents the high-frequency component in the semantic segmentation result of the RGB image, F HSV,h Represents the high-frequency component in the semantic segmentation result of the HSV image, F Lab,h Represents the high-frequency component in the semantic segmentation result of the Lab image.

[0071] In the present invention, high-frequency components usually contain details and significant features in the image. By adopting the maximum value fusion rule, the maximum value of the high-frequency components in each channel is selected, which emphasizes the features with the most significant information in each channel, and helps to highlight the important details and edge information in the image. By selecting the maximum value of the high-frequency component, the contrast of the image can be effectively improved. This is because selecting the maximum value at each pixel position can make the significant details more prominent and enhance the visual effect of the image.

[0072] S3034: Perform inverse wavelet transform on the fused frequency components to obtain a fused segmented image.

[0073] Among them, the inverse wavelet transform (IWT) is the inverse process of wavelet transform. In wavelet analysis, a signal or image can be decomposed into multiple frequency bands through wavelet transform, and the inverse wavelet transform is to recombine these frequency bands to restore the original signal or image.

[0074] S304: Input the fused segmented image and the filtered original desktop image into a U-Net neural network to segment the bone disc image.

[0075] S4: Introduce the Canny operator to detect the bone disc outline in the bone disc image and extract the bone disc area.

[0076] Among them, the Canny edge detection operator is an edge detection method commonly used in image processing. The specific content can be found in the relevant prior art, and the present invention will not be repeated.

[0077] In a possible implementation, the present invention proposes a new Canny edge detection technology. S4 specifically includes sub-steps S401 to S407:

[0078] S401: Generate a grayscale histogram of the bone disc image.

[0079] In the present invention, by generating a grayscale histogram of the bone disc image, the grayscale distribution of the image can be visualized and analyzed, which helps to understand the overall brightness and contrast characteristics of the bone disc image and provide a reference for subsequent processing steps.

[0080] S402: Calculate the gradient component of the bone disc image:

[0081]

[0082] Among them, G x represents the horizontal gradient component, G y represents the vertical gradient component, (x, y) represents the pixel coordinates, and f represents the desktop image.

[0083] In the present invention, calculating the gradient component of the bone disc image helps to capture the edge information in the image. The gradient represents the change in the pixel value of the image, so by calculating the gradient component, the edge position in the image can be found, which is an important prerequisite for subsequent edge detection and region segmentation.

[0084] S403: Calculate the gradient strength and gradient direction of the pixel:

[0085]

[0086] Among them, G represents the gradient strength and θ represents the gradient direction.

[0087] The gradient strength is the modulus or size of the gradient, which is used to measure the magnitude of the change in pixel values ​​in an image.

[0088] The gradient direction refers to the direction of the gradient vector, that is, the direction in which the pixel value in the image changes fastest.

[0089] S404: When the bone disc image has multiple gradient information, retain the maximum pixel points and suppress the non-maximum pixel points.

[0090] In the present invention, the local maximum value is retained in the gradient direction, and the non-maximum value is suppressed, which helps to refine the edge and improve the accuracy of detection.

[0091] S405: Set a high threshold and a low threshold. When the gradient strength of a pixel is greater than the high threshold, the pixel is determined to be a strong edge pixel. When the gradient strength of a pixel is between the low threshold and the high threshold, the pixel is determined to be a weak edge pixel. When the gradient strength of a pixel is less than the low threshold, the pixel is suppressed.

[0092] In a possible implementation manner, the present invention proposes a new method for determining the high threshold and the low threshold, specifically:

[0093] Determine the current segmentation threshold k, 0≤k≤255.

[0094] Count the proportion r of foreground pixels with grayscale values ​​in the range [0, k] in the bone disc image under the current segmentation threshold k f , the average grayscale u of the foreground pixels f And the ratio of background pixels with grayscale values ​​in the range of (k, 255] to the bone disc image is r a , the average grayscale u of background pixels a .

[0095] Calculate the inter-class variance under the segmentation of the current segmentation threshold k:

[0096] σ 2 =r f r a (u f -u a ) 2

[0097] Among them, σ 2 represents the between-class variance.

[0098] The segmentation threshold corresponding to the maximum inter-class variance is taken as the optimal segmentation threshold k m .

[0099] In the present invention, selecting a threshold with the largest inter-class variance helps to improve the accuracy of segmentation.

[0100] According to the optimal segmentation threshold k m Determine the high and low thresholds:

[0101] H=k m

[0102]

[0103] Among them, H represents the high threshold and L represents the low threshold.

[0104] It should be noted that by setting the high threshold as the optimal segmentation threshold, it is ensured to be sensitive enough to the difference between the foreground and the background. This setting enables the high threshold to better distinguish the target object (bone disc area) from the background. Setting the low threshold to 1 / 3 of the optimal segmentation threshold helps to deal with noise or subtle changes in the image. A lower low threshold can increase the sensitivity to areas with small grayscale changes, thereby enhancing the robustness of the overall segmentation.

[0105] In the present invention, the segmentation threshold is determined by a dynamic adjustment method. This allows the optimal segmentation threshold to be adaptively selected in different images or scenes, rather than using a fixed threshold. This is particularly important for image segmentation under different lighting conditions or shooting environments.

[0106] S406: Connect the strong edge pixels to obtain the bone disc outline.

[0107] In the present invention, by setting a high threshold and a low threshold, the gradient image can be divided into strong edges and weak edges. The strong edge pixels are connected to form an edge contour, which helps to extract the edge information in the bone dish image.

[0108] S407: Extract the bone disc area according to the bone disc contour.

[0109] In the present invention, the introduction of the Canny operator helps to improve the accuracy, robustness and reliability of the bone disc contour in the bone disc image, and provides high-quality input for subsequent bone disc region extraction and analysis.

[0110] S5: Calculate the contour completeness of the bone disc contour.

[0111] In a possible implementation, S5 specifically includes: calculating the contour completeness of the bone disc contour according to the following formula:

[0112]

[0113] Among them, O represents the contour completeness, S a Indicates the actual area of ​​the bone disc, S m Represents the area of ​​the minimum circumscribed circle of the bone disc area.

[0114] Among them, the contour completeness is an indicator used to quantify the integrity of the bone disc shape. By calculating the ratio of the actual area of ​​the bone disc contour to the area of ​​its minimum circumscribed circle, a value between 0 and 1 can be obtained to measure the compactness and completeness of the bone disc contour.

[0115] Reference Manual Attached Figure 2, showing a schematic diagram of a bone disc image provided by the present invention.

[0116] It is understandable that if Figure 2 As shown, when residual food overflows the bone dish, the integrity of the bone dish's outline will be destroyed, which can be used to assess whether the bone dish needs to be replaced.

[0117] S6: Binarization processing is performed on the bone disc area.

[0118] Among them, binarization is the process of converting a grayscale image into an image containing only two grayscale values ​​(usually 0 and 255). The main purpose of this process is to simplify the image and highlight the target area in the image. In a binary image, the target area is usually represented as white (255) and the background is represented as black (0).

[0119] In the present invention, the binarization process can strengthen the outline of the bone disc area and make it clearer. This helps the subsequent image analysis and processing, especially the algorithm that is sensitive to the outline information. At the same time, converting the image into a binary form can greatly simplify the analysis of the bone disc area. In a binary image, each pixel has only two values, which reduces the complexity of the data and makes the algorithm easier to implement and understand.

[0120] S7: Calculate the total number of white pixels in the bone disc area and the proportion of white pixels at the edge.

[0121] It should be noted that, in the application process of the present invention, the color of the bone dish is white, and most restaurants also use white bone dishes. Therefore, the total number of white pixels and the proportion of edge white pixels can both evaluate the amount of residual food in the bone dish.

[0122] In a possible implementation, S7 specifically includes sub-steps S701 to S703:

[0123] S701: Extract white pixels in the bone disc area and count the total number of white pixels S w .

[0124] S702: Determine whether the white pixel is connected to the bone disc contour through other white pixels. If so, determine it as an edge white pixel and keep it. Otherwise, remove it.

[0125] It should be noted that if Figure 2 As shown, the reason why non-edge white pixels that are not connected to the bone dish outline through other white pixels need to be removed is that it is very likely that some residual food itself is also white. Including it in the calculation range will reduce the accuracy of subsequent bone dish filling calculations.

[0126] S703: Calculate the ratio of edge white pixels in the bone disc area:

[0127]

[0128] Among them, R represents the proportion of white pixels on the edge, S we represents the total number of white pixels at the edge, and S represents the total number of pixels in the bone disc area.

[0129] S8: Calculate the bone disc filling degree according to the outline completeness of the bone disc outline, the total number of white pixels in the bone disc area, and the proportion of white pixels at the edge.

[0130] In a possible implementation, S8 specifically includes: calculating the bone dish filling degree according to the following formula:

[0131] σ=λ1O+λ2S w +λ3R

[0132] Among them, σ represents the bone disc filling degree, O represents the contour completeness, λ1 represents the weight coefficient of contour completeness, S w represents the total number of white pixels, λ2 represents the weight coefficient of the total number of white pixels, R represents the proportion of edge white pixels, and λ3 represents the weight coefficient of the proportion of edge white pixels.

[0133] Among them, those skilled in the art can set the weight coefficient λ1 of contour completeness, the weight coefficient λ2 of the total number of white pixels, and the weight coefficient λ3 of the proportion of edge white pixels according to actual conditions, and the present invention does not limit this.

[0134] In the present invention, by considering multiple factors such as contour completeness, total number of white pixels and proportion of edge white pixels, the system can have a more comprehensive understanding of the state of the bone disc, which helps to improve the accuracy of the system's judgment on the state of the bone disc.

[0135] S9: When the filling degree of the bone dish is greater than the preset filling degree, the corresponding bone dish is replaced.

[0136] Among them, those skilled in the art can set the size of the preset filling degree according to actual conditions, and the present invention does not limit this.

[0137] Specifically, an automated device may be used to automatically replace the bone dish, or a reminder may be broadcast to the waiter when the filling degree of the bone dish is greater than a preset filling degree, reminding the waiter to replace it.

[0138] In the present invention, when the filling degree of residual food reaches a preset condition, the bone dish is replaced. There is no need for the waiter to monitor the status of the bone dish all the time during the meal, which saves time and effort, can provide real-time and continuous monitoring, and respond to all table needs in time, so as to achieve timely replacement of bone dishes, improve service efficiency, keep the table clean, and enhance the dining experience of diners.

[0139] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0140] In the present invention, image recognition is used to automatically identify whether the residual food in the bone dish is full enough. When the filling degree of the residual food reaches a preset condition, the bone dish is replaced. There is no need for the waiter to monitor the status of the bone dish all the time during the meal, which saves time and effort, can provide real-time and continuous monitoring, and respond to all table needs in time, so as to realize timely replacement of bone dishes, improve service efficiency, keep the table clean, and enhance the dining experience of diners.

[0141] Example 2

[0142] In one embodiment, the reference specification Figure 3 , showing a structural schematic diagram of a bone disc replacement system based on image recognition provided by the present invention.

[0143] The present invention provides a bone disc replacement system 20 based on image recognition, comprising a processor 201 and a memory 202 for storing executable instructions of the processor 201. The processor 201 is configured to call the instructions stored in the memory 202 to execute the bone disc replacement method based on image recognition in Example 1.

[0144] The bone disc replacement system based on image recognition provided by the present invention can realize the steps and effects of the bone disc replacement method based on image recognition in the above-mentioned embodiment 1. To avoid repetition, the present invention will not go into details.

[0145] Compared with the prior art, the present invention has at least the following beneficial technical effects:

[0146] In the present invention, image recognition is used to automatically identify whether the residual food in the bone dish is full enough. When the filling degree of the residual food reaches a preset condition, the bone dish is replaced. There is no need for the waiter to monitor the status of the bone dish all the time during the meal, which saves time and effort, can provide real-time and continuous monitoring, and respond to all table needs in time, so as to realize timely replacement of bone dishes, improve service efficiency, keep the table clean, and enhance the dining experience of diners.

[0147] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A bone dish replacement method based on image recognition, characterized in that: include: S1: Get desktop image; S2: performing filtering processing on the desktop image based on adaptive smoothing filtering; S3: Based on the neural network, the bone disc image is segmented from the filtered desktop image; S4: introducing the Canny operator to detect the bone disc contour in the bone disc image and extract the bone disc area; S5: calculating the contour completeness of the bone disc contour; S6: performing binarization processing on the bone disc region; S7: Calculate the total number of white pixels in the bone disc area and the proportion of white pixels at the edge; S8: Calculating the bone disc filling degree according to the outline completeness of the bone disc outline, the total number of white pixels in the bone disc area, and the proportion of white pixels at the edge; S9: When the filling degree of the bone dish is greater than a preset filling degree, the corresponding bone dish is replaced.

2. The bone disc replacement method based on image recognition according to claim 1 is characterized in that: The S2 specifically includes: S201: Calculate the gradient component of the desktop image: Among them, G x represents the horizontal gradient component, G y represents the vertical gradient component, (x, y) represents the pixel coordinates, and f represents the desktop image; S202: Calculate the filter weight value of each pixel: Among them, w represents the filter weight value, exp represents the exponential function with e as the base, and h represents the edge retention amplitude coefficient; S203: Filter the desktop image according to the filter weight value of each pixel point: Where t represents the number of iterations, f t+1 represents the desktop image after t+1 filtering, f t Represents the desktop image after t times of filtering.

3. The bone dish replacement method based on image recognition according to claim 1, characterized in that: The S3 specifically includes: S301: Convert the filtered original desktop image into an RGB image, an HSV image and a Lab image respectively; S302: Obtain semantic segmentation results of RGB images, HSV images, and Lab images through the DeepLabV3+ network; S303: fusing the semantic segmentation results of the RGB image, the HSV image, and the Lab image through wavelet transform to obtain a fused segmentation image; S304: Input the fused segmented image and the filtered original desktop image into a U-Net neural network to segment a bone disc image.

4. The bone dish replacement method based on image recognition according to claim 3 is characterized in that: The S303 specifically includes: S3031: Decomposing low-frequency components and high-frequency components from the semantic segmentation results of the RGB image, the HSV image, and the Lab image through wavelet transform; S3032: For the low-frequency components of the semantic segmentation results of the RGB image, HSV image, and Lab image, a weighted average fusion rule is used for fusion: F mix,l (x,y)=β1F RGB,l (x,y)+β2F HSV,l (x,y)+β3F Lab,l (x,y) Among them, F mix,l Indicates the fusion of low-frequency components, F RGB,l represents the low-frequency component in the semantic segmentation result of the RGB image, β1 represents the weight coefficient of the RGB image, and F HSV,l represents the low-frequency component in the semantic segmentation result of the HSV image, β2 represents the weight coefficient of the HSV image, and F Lab,l represents the low-frequency component in the semantic segmentation result of the Lab image, and β3 represents the weight coefficient of the Lab image; S3033: For the high-frequency components of the semantic segmentation results of the RGB image, HSV image, and Lab image, the maximum fusion rule is used for fusion: F mix,h (x,y)=max[F RGB,h (x,y),F HSV,h (x,y),F Lab,h (x,y)] Among them, max[] means taking the maximum value, F RGB,h Represents the high-frequency component in the semantic segmentation result of the RGB image, F HSV,h Represents the high-frequency component in the semantic segmentation result of the HSV image, F Lab,h Represents the high-frequency component in the semantic segmentation result of the Lab image; S3034: Perform inverse wavelet transform on the fused frequency components to obtain a fused segmented image.

5. The bone dish replacement method based on image recognition according to claim 1, characterized in that: The S4 specifically includes: S401: Generate a grayscale histogram of the bone disc image; S402: Calculate the gradient component of the bone disc image: Among them, G x represents the horizontal gradient component, G y represents the vertical gradient component, (x, y) represents the pixel coordinates, and f represents the desktop image; S403: Calculate the gradient strength and gradient direction of the pixel: Among them, G represents the gradient strength and θ represents the gradient direction; S404: when the bone disc image has multiple gradient information, retaining the maximum pixel points and suppressing the non-maximum pixel points; S405: Setting a high threshold and a low threshold, when the gradient strength of a pixel point is greater than the high threshold, determining the pixel point as a strong edge pixel point; when the gradient strength of a pixel point is between the low threshold and the high threshold, determining the pixel point as a weak edge pixel point; when the gradient strength of a pixel point is less than the low threshold, suppressing the pixel point; S406: Connect the strong edge pixel points to obtain the bone disc contour; S407: Extracting the bone disc area according to the bone disc contour.

6. The bone dish replacement method based on image recognition according to claim 5 is characterized in that: The high threshold and the low threshold are determined in the following manner: Determine the current segmentation threshold k, 0≤k≤255; Count the proportion r of foreground pixels with gray values ​​in the range [0, k] in the bone disc image under the current segmentation threshold k f , the average grayscale u of the foreground pixels f And the ratio r of background pixels whose grayscale values ​​are in the range of (k, 255] to the bone disc image a , the average grayscale u of background pixels a ; Calculate the inter-class variance under the segmentation of the current segmentation threshold k: σ 2 =r f r a (u f -u a ) 2 Among them, σ 2 represents the between-class variance; The segmentation threshold corresponding to the maximum inter-class variance is taken as the optimal segmentation threshold k m ; According to the optimal segmentation threshold k m Determine the high threshold and the low threshold: H=k m Among them, H represents the high threshold and L represents the low threshold.

7. The bone dish replacement method based on image recognition according to claim 1, characterized in that: The S5 is specifically: The contour integrity of the bone dish contour is calculated according to the following formula: Among them, O represents the contour completeness, S a Indicates the actual area of ​​the bone disc, S m Represents the area of ​​the minimum circumscribed circle of the bone disc area.

8. The bone dish replacement method based on image recognition according to claim 1, characterized in that: The S7 specifically includes: S701: Extract white pixels in the bone disc area and count the total number of white pixels S w ; S702: Determine whether the white pixel is connected to the bone disc contour through other white pixel points; if so, determine it as an edge white pixel and retain it; otherwise, remove it; S703: Calculate the percentage of edge white pixels in the bone disc area: Among them, R represents the proportion of white pixels on the edge, S we represents the total number of white pixels at the edge, and S represents the total number of pixels in the bone disc area.

9. The bone dish replacement method based on image recognition according to claim 1, characterized in that: The S8 is specifically: The bone dish filling degree was calculated according to the following formula: σ=λ1O+λ2S w +λ3R Among them, σ represents the bone disc filling degree, O represents the contour completeness, λ1 represents the weight coefficient of contour completeness, S w represents the total number of white pixels, λ2 represents the weight coefficient of the total number of white pixels, R represents the proportion of edge white pixels, and λ3 represents the weight coefficient of the proportion of edge white pixels.

10. A bone disc replacement system based on image recognition, characterized in that: It comprises a processor and a memory for storing processor executable instructions; the processor is configured to call the instructions stored in the memory to execute the bone disc replacement method based on image recognition as described in any one of claims 1 to 9.

Images