Oral cavity medicine applying device based on image recognition

Abstract

The invention discloses an oral cavity medicine applying device based on image recognition. The oral cavity medicine applying device comprises a medicine applying device and a display device which are in wireless data connection. The medicine applying device comprises a handle with a first wireless transmission module and a probe which is installed on the handle and provided with a light source and a camera. The display device comprises a second wireless transmission module, a processor and a display screen. Wherein the probe further comprises a drug delivery part used for applying a drug to a target affected part, the first wireless transmission module is used for transmitting an image acquired by the camera to the second wireless transmission module in a wireless mode, and the processor is used for recognizing and processing the image acquired by the second wireless transmission module. The target affected part is recognized according to the processed image, and the medicine applying prompt information is given out, the target affected part is automatically recognized, the medicine applying prompt information is sent out, a user can apply medicine by himself / herself conveniently, and the user experience feeling is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and particularly to an oral medicine application device based on image recognition. Background Art

[0002] Oral ulcer is an oral disease with a wide coverage of disease types and a high incidence rate among current oral diseases. Its incidence has no obvious relationship with seasons, and it may occur in children, young adults, and the elderly. In the prior art, there are usually two ways to treat oral ulcers: oral medicine application and taking oral medications. Among them, for oral medicine application, it is usually necessary to use a cotton swab or an external object to press the tongue and then shine a flashlight to check the situation inside the patient's oral cavity, and apply medicine according to the situation. However, this process is extremely inconvenient for patients to operate by themselves. Although there are already visual oral medicine applicators combined with mobile terminals to achieve visual medicine application, the existing oral medicine applicators still require the operator to carefully judge which part is the medicine application part, making it difficult to apply medicine quickly and accurately. Moreover, the deviation of the image viewing angle and the operation angle position on the terminal leads to a poor user experience. Summary of the Invention

[0003] The present invention aims to provide an oral medicine application device based on image recognition to solve the problem of poor self - medicine application operation experience for current oral patients.

[0004] In a first aspect, the present invention provides an oral medicine application device based on image recognition, including:

[0005] A medicine applicator and a display device with a wireless data link; the medicine applicator includes a handle with a first wireless transmission module, a probe installed on the handle and having a light source and a camera, and a handle with a first wireless transmission module. The display device includes a second wireless transmission module, a processor, and a display screen. Wherein, the probe further includes a medicine application part for applying medicine to the target affected area. The first wireless transmission module is used to wirelessly transmit the image acquired by the camera to the second wireless transmission module. The processor is used to perform recognition processing on the image acquired by the second wireless transmission module, and give a medicine application prompt message according to the target affected area recognized from the processed image.

[0006] Optionally, the medicine applicator further includes a distance sensor, and the distance sensor is used to obtain the distance range between the probe and the front - end obstacle. The medicine application prompt message includes marking the recognized affected area with a symbol and emitting a prompt sound when the distance range is within a preset range.

[0007] Optionally, the medicine application prompt message including marking the recognized affected area with a symbol and emitting a prompt sound within a preset range includes:

[0008] Step 1: Establish an experience database of oral ulcer images, set the picture pixels, select a class model with abstract features of the same type for the image to be recognized, and preprocess the image to be recognized. The preprocessing includes making the components of each pixel of the image to be recognized R = G = B, so as to obtain the grayscale value feature vector of the image to be recognized.

[0009] Step 2: Make

[0010] and

[0011] where is the transposed vector of v k , v k is the vector of the class model, v k =(v k1 , v k2 ,......, v kN ), N is the dimension of the class model vector, obtain the adjoint vector so that it satisfies the following formula

[0012] where v k′ represents the adjoint vector corresponding to the feature vector in v k , k' is the serial number corresponding to the feature vector in v k , k = 1......M, M is the number of class models, M ≤ N;

[0013] Step 3: Input the initial value q(0) of the feature vector of the model to be tested for the image to be recognized, and use the formula to find the initial value ζ k of the ordered parameter ζ k (0);

[0014] Step 4: Use the parameter ζ k to establish the following iterative formula of ζ k :

[0015] where λ is selected according to the similarity between the class model and the vector of the model to be tested, b and c are coefficients preset according to experience, and d satisfies:

[0016] Step 5: Establish a three-layer neural network model. The first layer to the third layer of the neural network model are the input layer, the second layer and the output layer respectively. Each receiving unit of the input layer receives the component q j (0) of the feature vector q(0) of the model to be tested for the image to be recognized; the second layer of the neural network model is each parameter ζ k neuron, where the parameter ζk is each q j (0) multiplied by the one associated with this q j (0) connected The sum after the product, where is the adjoint vector The j-th component of; The second layer iteratively runs starting from ζ k (0) as described in the formula in step 4, and returns to step 4 to run again when executes step 6 to obtain the final image recognition result from the output layer;

[0017] Step 6: Obtain the final image recognition result according to the following formula:

[0018] where q j (t) is the element of the output layer unit j, and ζ k (t) is the t-th vector of the parameter ζ k , t = 0, 1, 2, …… T, and T is the total number of vectors of ζ k , and M is the number of the class models;

[0019] Input the recognized q j (t) into the oral ulcer image experience database, and compare it with the experience data in the database to determine whether the image to be recognized is an image of the target affected area.

[0020] Optionally, the medicine application prompt information includes displaying the required medicine amount for this time according to the preset medicine amount and / or the previous disease condition, and the required medicine amount for this time includes the amount of medicine released each time and the remaining number of times of medicine release.

[0021] Optionally, the medicine delivery part of the probe is a spoon-shaped part and / or a hollow part. When it is a hollow part, it jointly realizes the drug supply to the target affected area in combination with the medicine storage part and the pressure generation module in the handle.

[0022] Optionally, the preset range is obtained based on the infrared sensor and the processor.

[0023] Optionally, the infrared sensor is integrated in the camera or arranged in the probe.

[0024] Optionally, the processor is an FPGA or an MCU.

[0025] In the present invention, since the affected area, i.e., the target drug application position information, is obtained after processing the acquired oral cavity image, drug application prompt information can be given according to this information, avoiding the inconvenience of direction operation caused by the angle deviation between the handle and the image displayed by the display device during user operation, and the inconvenience caused by the inability to timely obtain the target drug application position due to vision or lack of professional judgment ability. In addition, the prompts given by the present invention include visual prompts (marking the affected area with symbols) and auditory prompts (emitting prompt sounds), which can effectively improve the accuracy of drug application when the operator applies the drug by himself. To make the drug application device applicable to a wider range, the probe of the present invention can be set as a spoon shape and / or a hollow part, so that the present invention is applicable to solid or liquid drugs. The affected area positioning of the present invention combines a neural network model for image processing. First, it solves the drawback that the existing drug application devices cannot obtain the affected area information. Second, it solves the problems that the model used in the existing image recognition has a long training time and insufficient accuracy. Moreover, the distance acquisition of the present invention combines temperature information to achieve precise positioning of the affected area, more efficiently obtain the affected area information, and better improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention can be further understood from the following description in conjunction with the accompanying drawings. The components in the drawings are not necessarily drawn to scale, but the emphasis is placed on showing the principles of the embodiments.

[0027] Figure 1 It is a system structure diagram of an oral drug application device based on image recognition provided by the present invention;

[0028] Figure 2 It is a three-layer neural network structure diagram provided in Embodiment 1 of the present invention;

[0029] Figure 3 It is a display effect diagram after obtaining the target affected area provided by the present invention;

[0030] Figure 4 It is one of the circuit connection schematic diagrams provided by the present invention;

[0031] Figure 5 It is a schematic flow diagram of the image recognition method provided in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with its embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. For those skilled in the art, after referring to the following detailed description, other systems, methods and / or features of this embodiment will become obvious. It is intended that all such additional systems, methods, features and advantages be included within this specification, be included within the scope of the present invention, and be protected by the appended claims. Additional features of the disclosed embodiments are described in the following detailed description and will be obvious from the following detailed description.

[0033] In the drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", etc. indicating the orientation or positional relationship, it is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or component referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the drawings are only for illustrative purposes and cannot be understood as a limitation of this patent. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0034] Embodiment 1

[0035] Figure 1An oral medicine application device based on image recognition provided in the first embodiment of the present invention. In this embodiment, by combining image recognition of the affected area, it is convenient for the operator to quickly locate the affected area and apply medicine when applying medicine by himself. The oral medicine application device in this embodiment includes a medicine applicator and a display device with a wireless data link; the medicine applicator includes a handle with a first wireless transmission module, a probe installed on the handle and having a light source and a camera, and a handle with a first wireless transmission module. The display device includes a second wireless transmission module, a processor, and a display screen; wherein, the probe further includes a medicine application part for applying medicine to the target affected area. The first wireless transmission module is used to wirelessly transmit the image acquired by the camera to the second wireless transmission module. The processor is used to perform recognition processing on the image acquired by the second wireless transmission module, and give a medicine application prompt message according to the target affected area recognized from the processed image. A microcontroller and a speaker are further provided in the handle, and a user operation switch (which can be optionally set or not set). The light source, the camera, the first wireless transmission module, the speaker, and the user operation switch are all connected to the microcontroller. The microcontroller is used for processing all data in the medicine applicator, and the speaker is used for emitting the medicine application prompt message. Optionally, the speaker is not in the probe but is integrated with the display device. For example, the display device can be a device such as a mobile phone, a tablet computer, or a computer, and the prompt message can be realized through the speaker on the mobile phone, the tablet computer, or the computer. The first wireless transmission module is wirelessly connected to the second wireless transmission module, and the display screen and the second wireless transmission module are both connected to the processor. The attached drawing Figure 4 shows a schematic diagram of the circuit module connection of the speaker in the handle. The left half of the figure is the medicine applicator, and the right half is the display device.

[0036] The medicine applicator further includes a distance sensor. The distance sensor is connected to the microcontroller. The distance sensor is used to obtain the distance range between the probe and the front obstacle. The medicine application prompt message includes marking the recognized affected area with a symbol and emitting a prompt sound when the distance range is within a preset range. The distance sensor in this embodiment can be an infrared sensor or an ultrasonic sensor.

[0037] The medicine application prompt message includes marking the recognized affected area with a symbol and emitting a prompt sound within a preset range, including the following process steps as shown in the attached drawing Figure 5 as follows:

[0038] Step 1: Establish an experience database for oral ulcer images, set the picture pixels, and segment the image obtained by the camera into the image to be recognized. Specifically, the image can be segmented into several images to be recognized, or the semantic segmentation method can be used to segment the image to obtain the image to be recognized. Or, according to the visual saliency map, use the spatial domain filtering and threshold segmentation algorithms to separate two regions, combine the morphological operations to obtain a tripartite graph, calculate the gradient of each pixel in the unknown region according to the tripartite graph, sample according to the gradient direction and saliency magnitude to obtain the sample point set of the current unknown region pixels, then calculate the opacity and confidence of each sample point, and take the sample with the highest confidence as the best sample pair for the final matte extraction. Then smooth the local region of the opacity to obtain the finally estimated opacity. According to the finally estimated opacity and the color values of the best sample pair, perform a matte extraction operation on the original image to extract the image to be recognized. Select a class model with the abstract features of the same type of pattern for the image to be recognized. This class model reflects the common attributes of the same type of pattern. Preprocess the image to be recognized so that the component of each pixel R = G = B, where R, G, and B represent the color brightness of the red, green, and blue channels respectively, so as to obtain the gray value feature vector of the image to be recognized. Specifically, it can be: extract the texture features of the image to be recognized and extract the feature values to form the feature vector; or use the convolutional neural network after sample training to obtain the feature vector for the image to be recognized; or start from the preset pixel points in the grayscale image, establish a selection area in the grayscale image according to the preset size, and judge whether the gray value of the pixel point is less than the gray mean value based on the gray mean value and standard deviation value of the selection area. If it is less, mark the feature value of this point as 0. If it is greater, mark the feature value of this point as a non-zero value; concatenate the feature values in the selection area row / column by row / column to obtain the feature vector of the current preset pixel point; obtain the next pixel point of the preset pixel point along the image coordinate system direction, and repeat the above steps for obtaining the feature vector of the preset pixel point based on this next pixel point until the feature vector of the last preselected pixel point is obtained; only retain the feature vectors with the deviation value greater than the preset value W, and perform weighted processing on the retained feature vectors, and concatenate all the weighted feature vectors to finally obtain the global feature vector, where the feature vector is represented by the pixel gray value;

[0039] Step 2: Make

[0040]

[0041] where is the transpose vector of v k , v k is the vector of the class model, v k =(v k1 , v k2 ,......, vkN ), where N is the dimension of the class model vector of the image to be recognized, and an adjoint vector is obtained to satisfy the following formula: where v k′ represents the adjoint vector corresponding to v k in the eigenvector, k' is the serial number corresponding to the eigenvector in v k , k = 1... M, M is the number of the class models, and M ≤ N;

[0042] Step 3: Input the initial value q(0) of the eigenvector of the model to be tested for the image to be recognized, and use the formula to find the initial value ζ k of the ordered parameter ζ k (0);

[0043] Step 4: Use the parameter ζ k to establish the following iterative formula for ζ k :

[0044]

[0045] where λ is selected according to the similarity between the class model and the vector of the model to be tested, b and c are coefficients preset according to experience, usually b = c > 0, and b = c = 1 can be taken in specific applications. d satisfies: λ is obtained by those skilled in the art based on historical experience and a large number of repeated experimental trainings, which will not be elaborated here.

[0046] Step 5: Establish a three-layer neural network model. The first layer to the third layer of the neural network model are the input layer, the second layer, and the output layer respectively. Each receiving unit of the input layer receives the component q j (0) of the eigenvector of the model to be tested for the image to be recognized; the second layer of the neural network model is each parameter ζ k neuron, where the parameter ζ k is the sum of the products of each q j (0) multiplied by the j connected to this q , where is the adjoint vector of the jth component; the second layer starts iterative operation from ζ k (0) according to the formula in Step 4, and returns to Step 4 to run again when , and executes Step 6 to obtain the final image recognition result from the output layer until ;

[0047] Step 6: Obtain the final image recognition result according to the following formula:

[0048]

[0049] where q j (t) is the t-th element of the output layer unit j, and ζ k (t) is the t-th vector of the parameter ζ k , t = 0, 1, 2, …… T, T is the total number of vectors of ζ k , and M is the number of the class models;

[0050] Input the recognized q j (t) into the oral ulcer image experience database, and compare it with the experience data in the database to determine whether the image to be recognized is an image of the target affected area. Specifically, calculate the Euclidean distance ||q j (t) - Q(t)| between q j (t) and the feature vector Q(t) of the m-th image in the database. Determine whether the Euclidean distance is less than the threshold τ. If it is less, it is determined that the image to be recognized and the m-th image belong to the same class of images. When the m-th image is an oral ulcer image, it is determined that the image to be recognized is an image of the target affected area. When the result is an image of the target affected area, mark the image recognition result with a symbol. The attached drawing Figure 3 is a schematic diagram after marking, and the symbol can also be a circle, an ellipse or a square.

[0051] The medicine application prompt information includes displaying the required medicine amount for this time according to the preset medicine amount and / or the previous illness condition. The required medicine amount for this time includes the amount of medicine released each time and the remaining number of medicine release times.

[0052] The medicine delivery part of the probe is a spoon-shaped part and / or a hollow part. When it is a hollow part, it combines with the medicine storage part and the pressure generation module in the handle to jointly realize the drug supply to the target affected area. In this embodiment, when the user uses it, after hearing the prompt sound and confirming the position according to the prompt information displayed by the display device, press the switch, and the handle generates a jet pressure to spray the solid or liquid medicine onto the target affected area. To reduce the cost of the medicine application device, the switch can be not set, and the probe can be set as a spoon-shaped part. In this embodiment, when the probe is set as a spoon-shaped part, the user can apply the medicine originally placed in the spoon part to the target affected area after hearing the prompt sound and confirming the position according to the prompt information displayed by the display device.

[0053] The preset range is obtained based on the infrared sensor or the ultrasonic sensor and the processor. The processor judges whether it is within the preset range according to the range data obtained by the infrared sensor or the ultrasonic sensor. If it is, a signal to start sending the medicine application prompt information is sent to the speaker.

[0054] The infrared sensor is integrated on the camera or arranged in the probe.

[0055] The processor is FPGA or MCU.

[0056] The image recognition method used in this embodiment not only enables the existing medication application device to have the function of automatically identifying the affected area, but also improves the efficiency of image recognition of oral affected areas, avoiding problems such as noise interference, image defects, and angle changes that exist in ordinary image recognition algorithms. The algorithm of the present invention is simple and convenient for the development and application of microprocessors, thereby improving the feasibility of image recognition of oral affected areas in medication application devices.

[0057] Embodiment 2

[0058] This embodiment can be combined with the first embodiment or implemented separately. This embodiment realizes accurate distance acquisition of the affected part and improves the user experience. The oral medication device of this embodiment includes a medication applicator and a display device with a wireless data link; the medication applicator includes a handle with a first wireless transmission module and a probe installed on the handle and having a light source and a camera, and the display device includes a second wireless transmission module, a processor and a display screen; wherein the probe also includes a drug delivery part for applying the drug to the target affected part, the first wireless transmission module is used to transmit the image acquired by the camera to the second wireless transmission module by wireless mode, and the processor is used to identify and process the image acquired by the second wireless transmission module, and identify the target affected part according to the processed image and give medication prompt information. Wherein, a microcontroller and a speaker are also provided in the probe, and a user operation switch (optionally set or not set), the light source, the camera, the first wireless transmission module, the speaker and the user operation switch are all connected to the microcontroller, the microcontroller is used to process all data in the medication applicator, and the speaker is used to send the medication prompt information. The first wireless transmission module is wirelessly connected to the second wireless transmission module, and the display screen and the second wireless transmission module are both connected to the processor.

[0059] The medicine applicator also includes a distance sensor, which is connected to the microcontroller. The medicine application prompt information includes marking the identified affected area with a symbol and issuing a prompt sound within a preset range. Since infrared imaging can better reflect the temperature difference between the background and the target, and the imaging of different parts of the human body under different conditions is different, the distance sensor of this embodiment can be an infrared sensor that can obtain infrared images. The sensor can be combined with a camera and implemented using an infrared camera in the prior art.

[0060] The above medicine application prompt information includes marking the identified affected area with symbols and emitting a prompt sound within a preset range, including: converting the acquired image into a grayscale image, identifying the pixel point area with grayscale values within the threshold range in the grayscale image as the target area, and other pixel point areas with grayscale values as the background area; calculating the centroid of the target area, and emitting a prompt sound if the centroid pixel point meets the preset distance condition.

[0061] Specifically, the pixel point area with grayscale values within the threshold range [t1, t2] in the grayscale image is identified as the target area, and other pixel point areas with grayscale values are the background area; the formula for calculating the centroid of the target area is:

[0062] X m = ∑x i / |D|, Y m = ∑y i / |D|, where D is the range of the horizontal and vertical coordinates of the target area; (X m , Y m ) is the centroid coordinate of the target area D; |D| is the number of all pixel points within the target area D; ∑x i is the sum of the horizontal coordinates of all pixel points within the target area D; ∑y i is the sum of the vertical coordinates of all pixel points within the target area D; assuming the size of the acquired image is X*Y, and its central pixel point is the coordinate origin (X 0 , Y 0 ), let the maximum deviation area be a rectangular area of △x*△y centered on the origin. If the centroid pixel point (X m , Y m ) satisfies: Then a prompt sound is emitted. Among them, t1, t2, △x, and △y are respectively obtained by those skilled in the art based on historical experience and a large number of repeated experimental trainings, and will not be elaborated here.

[0063] The medicine application part of the probe is a spoon-shaped component and / or a hollow part. When it is a hollow part, it combines with the medicine storage part and the pressure generation module in the handle to jointly realize the drug supply to the target affected area. In this embodiment, when the user is using it, after hearing the prompt sound and confirming the position according to the prompt information displayed on the display device, pressing the switch, the handle will generate a jet pressure to spray the solid or liquid medicine onto the target affected area. To reduce the cost of the medicine application device, the switch can be not set, and the probe can be set as a spoon-shaped part. In this embodiment, when the probe is set as a spoon-shaped part, the user can apply the medicine originally placed in the spoon part to the target affected area after hearing the prompt sound and confirming the position according to the prompt information displayed on the display device.

[0064] The preset range is obtained based on the infrared sensor and the processor. The processor determines whether it is within the preset range according to the range data obtained by the infrared sensor. If so, a signal is sent to the speaker to start sending the medicine application prompt information.

[0065] The infrared sensor is integrated in the camera or arranged inside the probe.

[0066] The processor is an FPGA or an MCU.

[0067] Since the infrared image of the affected area is significantly different from that of the normal area, in this embodiment, the affected area location is obtained based on the image detected by the infrared sensor. On the basis of facilitating the user to quickly obtain the affected area, when the medicine application part is within the effective medicine application range, the user is prompted in time, improving the medicine application efficiency.

[0068] Embodiment 3

[0069] This embodiment can be combined with the previous embodiments or implemented independently. This embodiment further limits the prompt information, which can facilitate the operator to quickly locate the affected area and apply medicine when applying medicine by himself. The oral medicine application device of this embodiment includes a medicine applicator and a display device with a wireless data link; the medicine applicator includes a handle with a first wireless transmission module and a probe installed on the handle and having a light source and a camera, and the display device includes a second wireless transmission module, a processor and a display screen; wherein, the probe further includes a medicine application part for applying medicine to the target affected area, the first wireless transmission module is used to wirelessly transmit the image obtained by the camera to the second wireless transmission module, the processor is used to identify and process the image obtained by the second wireless transmission module, identify the target affected area according to the processed image and give a medicine application prompt information. A microcontroller and a speaker are also arranged inside the probe, and a user operation switch (which can be optionally set or not set). The light source, the camera, the first wireless transmission module, the speaker and the user operation switch are all connected to the microcontroller. The microcontroller is used for processing all data in the medicine applicator, and the speaker is used for emitting the medicine application prompt information. The first wireless transmission module is wirelessly connected to the second wireless transmission module, and the display screen and the second wireless transmission module are both connected to the processor.

[0070] The medicine applicator further includes a distance sensor, the distance sensor is connected to the microcontroller, and the medicine application prompt information includes marking the identified affected area with a symbol and emitting a prompt sound within a preset range. The distance sensor in this embodiment can be an infrared sensor or an ultrasonic sensor. In this embodiment, it can be determined whether the medicine application part is within the effective medicine application range by whether the distance between the medicine application part and the obstacle fed back by the infrared sensor or the ultrasonic sensor is within the preset range, and no limitation is made on this.

[0071] The above-mentioned medicine application prompt information includes marking the identified affected area with symbols and emitting a prompt sound within a preset range, including: the above-mentioned medicine application prompt information includes displaying the required medicine amount for this time according to the preset medicine amount obtained and / or the previous illness condition. The required medicine amount for this time includes the amount of medicine released each time and the remaining number of medicine release times. Specifically, the processor can save the image of the affected area obtained for the first time. When applying medicine subsequently, the processor can compare the image of the previous affected area obtained with the image of the current affected area and send out prompt information according to the comparison result, or send out prompt information according to the preset medicine release amount, times, and the situation of the current medicine release times. For example, a total of 6 times of medicine need to be released, twice a day, 0.3g each time, and this is the third time. The speaker will emit "Arrived at the affected area, medicine can be applied. This is the third time of applying medicine. Please apply the fourth time after three hours" or "Arrived at the affected area, medicine can be applied. This is the first time of applying medicine today. Please apply the second time after three hours" when the user uses it. If the result of comparing the image of the previous affected area obtained with the image of the current affected area is that the affected area becomes smaller, the speaker will emit a prompt of "There has been improvement. Please continue to apply medicine according to the treatment course. Wish you a speedy recovery" when the user uses it. Such information is not limited in the present invention.

[0072] The medicine delivery part of the probe is a spoon-shaped component and / or a hollow part. When it is a hollow part, it combines with the medicine storage part and the pressure generation module in the handle to jointly achieve the drug supply to the target affected area. In this embodiment, when the user uses it, after hearing the prompt sound and confirming the position according to the prompt information displayed on the display device, the user presses the switch, and the handle generates a jet pressure to spray the solid or liquid medicine onto the target affected area. To reduce the cost of the medicine application device, the switch can be not set, and the probe is set as a spoon-shaped part. In this embodiment, when the probe is set as a spoon-shaped part, the user can apply the medicine originally placed in the spoon part to the target affected area after hearing the prompt sound and confirming the position according to the prompt information displayed on the display device.

[0073] The preset range is obtained based on an infrared sensor or an ultrasonic sensor and the processor. The processor judges whether it is within the preset range according to the range data obtained by the infrared sensor or the ultrasonic sensor. If so, it sends a signal to the speaker to start emitting the medicine application prompt information.

[0074] The infrared sensor is integrated on the camera or arranged inside the probe.

[0075] The processor is an FPGA or an MCU.

[0076] This embodiment realizes intelligent voice prompt on the basis of the previous embodiment, which is convenient for users to obtain medicine application information and recovery situation, and further improves the user experience.

[0077] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments only. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An oral medication device based on image recognition, characterized in that: The oral medication device comprises: A medicine applicator and a display device with a wireless data link; the medicine applicator includes a handle with a first wireless transmission module and a probe installed on the handle and having a light source and a camera, and the display device includes a second wireless transmission module, a processor and a display screen; wherein the probe also includes a drug administration portion for applying the medicine to the target affected area, the first wireless transmission module is used to wirelessly transmit the image acquired by the camera to the second wireless transmission module, and the processor is used to recognize and process the image acquired by the second wireless transmission module, recognize the target affected area according to the processed image, and provide medicine application prompt information.

2. The oral medication device according to claim 1, characterized in that: The medicine applicator also includes a distance sensor, which is used to obtain the distance range between the probe and the front obstacle. The medicine application prompt information includes marking the identified affected area with a symbol and issuing a prompt sound when the distance range is within a preset range.

3. The oral medication device according to claim 2, characterized in that: The step of marking the identified affected area with a symbol and issuing a warning sound within a preset range includes: Step 1: Establish an empirical database of oral ulcer images, set the image pixels, segment the image acquired by the camera into the image to be identified, select a class model with abstract features of the same pattern for the image to be identified, and preprocess the image to be identified. The preprocessing includes making the component R=G=B of each pixel of the image to be identified, thereby obtaining the feature vector of the gray value of the image to be identified Step 2: Make and in Yes k The transposed vector, v k is the vector of the class model, v k =(v k1 , v k2 ,......,v kN ), N is the dimension of the class model vector of the image to be identified, and the accompanying vector is obtained So that it satisfies the following formula: where v k′ represents the adjoint vector Corresponding to v k The eigenvector in v, k' is k The serial number corresponding to the inner feature vector, k=1...M, M is the number of the class models, M≤N; Step 3: Input the initial value q(0) of the feature vector of the image to be identified and use the formula Find the ordered parameter ζ k The initial value ζ k (0); Step 4: Using parameter ζ k Establish the following k The iterative formula is: Where λ is selected according to the similarity between the class model and the model vector to be tested, b and c are coefficients preset based on experience, and d satisfies: Step 5: Establish a three-layer neural network model, wherein the first to third layers of the neural network model are input layer, second layer and output layer respectively, and each receiving unit of the input layer receives the component q of the feature vector q(0) of the model to be identified j (0); The second layer of the neural network model is each parameter ζ k neuron, where the parameter ζ k Each q j (0) multiplied by q j (0) Connected The sum of the products, where is the adjoint vector The second layer is obtained from ζ according to the formula in step 4. k (0) starts the iteration and Return to step 4 and run again until When executing step 6, the final image recognition result is obtained from the output layer; Step 6: Obtain the final image recognition result according to the following formula: where q j (t) is the t-th element of output layer unit j, ζ k (t) is the parameter ζ k The tth vector of , t=0,1,2,……T, T is ζ k The total number of vectors, M is the number of class models; The identified q j (t) inputting the image to an empirical database of oral ulcer images to compare it with the empirical data in the database to determine whether the image to be identified is an image of the target affected area; when the result is an image of the target affected area, marking the image recognition result with a symbol.

4. The oral medication device according to claim 3, characterized in that: The medicine application prompt information includes marking the identified affected area with a symbol and issuing a prompt sound within a preset range, including: converting the acquired image into a grayscale image, identifying the pixel point area in the grayscale image whose grayscale value is within the threshold area as the target area, and the pixel point area with other grayscale values ​​as the background area; obtaining the center of mass of the target area, and issuing a prompt sound if the center of mass pixel meets the preset distance condition.

5. The oral medicine dispensing device according to claim 4, characterized in that: The medication reminder information includes displaying the required medication amount based on the preset medication amount and / or the previous illness condition, and the required medication amount includes the medication amount for each dose and the current remaining number of medication doses.

6. The oral medicine dispensing device according to claim 5, characterized in that: The drug delivery part of the probe is a spoon-shaped part and / or a hollow part. When it is a hollow part, it is combined with the drug storage part and the pressure generating module in the handle to achieve drug delivery to the target affected part.

7. The oral medication device according to claim 6, characterized in that: The preset range is acquired based on the infrared sensor and the processor.

8. The oral medication device according to claim 7, characterized in that: The infrared sensor is integrated on the camera or arranged in the probe.

9. The oral medication device according to claim 8, characterized in that: The processor is FPGA or MCU.

Images