System for assessing the quality of swallowing jelly and method for assessing the quality of swallowing jelly

Abstract

Provided are a system for determining the quality of deglutition jelly and a method for determining the quality of deglutition jelly, which make it possible to non-invasively determine the quality of deglutition jelly on the basis of an ultrasonic image. This system for determining the quality of deglutition jelly comprises: an ultrasonic probe; an image generation unit that generates an ultrasonic image from a reception signal which has been obtained by using the ultrasonic probe to transmit an ultrasonic beam to, and receive an ultrasonic beam from, a subject; an image analysis unit that analyzes an ultrasonic image, which has been generated in a state where the ultrasonic probe is in contact with the outer surface of a sealed package enclosing deglutition jelly having gas bubbles, to acquire a first gas bubble pattern in the deglutition jelly; and a quality determination unit that determines the quality of the deglutition jelly on the basis of the first gas bubble pattern.

Description

[Technical Field]

[0001] The present invention relates to a swallowing jelly quality assessment system and a swallowing jelly quality assessment method for assessing the quality of swallowing jelly used when examining dysphagia. [Background technology]

[0002] As a method for testing swallowing disorders in elderly people during eating, the possibility of performing a non-invasive test for swallowing disorders using ultrasound is being investigated. When testing for swallowing disorders using ultrasound, the subject is asked to swallow a test meal, and an ultrasound image of the subject's pharynx during swallowing is observed to evaluate the presence or absence of swallowing disorders in the subject, for example, the presence or absence of test meal residue in the subject's pharynx, the presence or absence of aspiration, etc.

[0003] Here, prior art documents that serve as references for the present invention include, for example, Patent Documents 1 to 3.

[0004] Patent Document 1 describes an ultrasonic tomography aid that is obtained by cooling an aerated, molten gel-like food to form a gel. Patent Document 2 describes a semi-solid jelly-like test material containing a suspension of microbubbles or the like for use in testing contact swallowing function using an ultrasonic test method. Patent Document 3 discloses that an ultrasonic probe is placed against one outer surface of a package filled with contents such as liquid food, and ultrasonic waves are emitted and reflected waves are received, and the presence and degree of deterioration of the contents is determined based on the detected values. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-147757 [Patent Document 2] International Publication No. 2013 / 187283 [Patent Document 3] Japanese Patent Application Laid-Open No. 63-186141 Summary of the Invention [Problem to be solved by the invention]

[0006] Patent Documents 1 and 2 disclose swallowing jelly as a test food when examining swallowing disorders using ultrasound, but do not disclose assessing the quality of the swallowing jelly. Patent Document 3 discloses that ultrasound is used to determine whether or not the contents packed in a package have deteriorated and to what extent. However, this determination is made based on at least one of the difference in sound speed between the transmitted wave and the received wave, or the sound speed of the received wave, the time from transmission to reception, or the attenuation of ultrasonic energy, and does not disclose that the determination is made based on an ultrasound image.

[0007] Due to the principles of ultrasound examination, it is difficult to capture the residual jelly in an ultrasound image. According to the research of the present inventor, it has been found that by incorporating granular air bubbles into the swallowing jelly to facilitate the reflection of ultrasound, it becomes easier to detect the residue of the swallowing jelly in ultrasound images. Furthermore, according to the research of the present inventor, it has become clear that the shape of the air bubble pattern incorporated into the swallowing jelly changes with time and due to changes in the temperature of the storage location, etc. This change in the shape of the bubble pattern is reflected on the ultrasound image, which may have a negative impact on the user's assessment of the swallowing jelly residue and the machine learning model's ability to detect the swallowing jelly residue. Therefore, it is necessary to determine the quality of the swallowing jelly before performing a swallowing disorder test.

[0008] An object of the present invention is to provide a system and a method for determining the quality of swallowing jelly that can non-invasively determine the quality of swallowing jelly based on ultrasound images. [Means for solving the problem]

[0009] In order to achieve the above object, the present invention provides an ultrasonic probe, an image generating unit that generates an ultrasound image from a received signal obtained by transmitting and receiving an ultrasound beam to and from a subject using an ultrasound probe; an image analysis unit that acquires a first air bubble pattern in the swallowable jelly by analyzing an ultrasound image generated when an ultrasound probe is in contact with an outer surface of an unopened package in which the swallowable jelly containing air bubbles is enclosed; A system for determining the quality of swallowable jelly is provided, comprising: a quality determining unit that determines the quality of the swallowable jelly based on a first air bubble pattern.

[0010] Here, it is preferable that the quality determination unit detects a change in the shape of the first air bubble pattern relative to the air bubble pattern in the swallowing jelly when it is sealed in the package, and determines the quality of the swallowing jelly based on the change in the shape of the first air bubble pattern.

[0011] The device further includes a memory in which a plurality of reference ultrasound images in which the bubble pattern changes sequentially according to the deterioration of the quality of the swallowing jelly are stored, Preferably, the quality determining unit detects a change in the shape of the first bubble pattern by comparing the ultrasound image with a plurality of reference ultrasound images stored in the memory.

[0012] Preferably, the quality determining unit includes a machine learning model that receives the first air bubble pattern as an input and outputs an estimation result of the quality of the swallowing jelly, and determines the quality of the swallowing jelly based on the estimation result.

[0013] The device also includes an optical image acquisition unit that acquires an optical image of the unopened package, The image analysis unit further analyzes the optical image to obtain optical features of the swallowing jelly, It is preferable that the quality determining unit determines the quality of the swallowable jelly based on the first air bubble pattern and the optical feature amount.

[0014] It is also preferable that the quality assessment unit detects a change in the shape of the first air bubble pattern relative to the air bubble pattern in the swallowing jelly when it is sealed in the packaging, detects a change in the optical characteristics of the swallowing jelly relative to the optical characteristics of the swallowing jelly when it is sealed in the packaging, and assesses the quality of the swallowing jelly based on the change in the shape of the first air bubble pattern and the change in the optical characteristics of the swallowing jelly.

[0015] Also, a memory is provided in which a plurality of reference ultrasound images in which the bubble pattern changes sequentially according to the deterioration of the quality of the swallowing jelly and a plurality of reference optical images in which the optical feature amount of the swallowing jelly when sealed in the package changes sequentially according to the deterioration of the quality of the swallowing jelly are stored, It is preferable that the quality assessment unit detects changes in the shape of the first bubble pattern by comparing the ultrasound image with multiple reference ultrasound images stored in memory, and detects changes in the optical features of the swallowing jelly by comparing the optical image with multiple reference optical images stored in memory.

[0016] In addition, it is preferable that the quality determination unit includes a first machine learning model that takes the first bubble pattern as input and outputs a first estimation result that estimates the quality of the swallowing jelly, and a second machine learning model that takes the optical features of the swallowing jelly as input and outputs a second estimation result that estimates the quality of the swallowing jelly, and that the quality of the swallowing jelly is determined by integrating the first determination result determined based on the first estimation result and the second determination result determined based on the second estimation result.

[0017] In addition, the optical feature amount preferably includes the color of the swallowing jelly.

[0018] Preferably, the optical feature includes a second bubble pattern in the swallowable jelly.

[0019] In addition, it is preferable that the quality determination unit determines the quality of the swallowing jelly by weighting and integrating a first determination result of the quality of the swallowing jelly determined based on the first bubble pattern and a second determination result of the quality of the swallowing jelly determined based on the optical feature.

[0020] In addition, it is preferable that the quality determination unit includes a multimodal model that receives the first bubble pattern and optical features as input and outputs an estimation result of the quality of the swallowing jelly, and determines the quality of the swallowing jelly based on the estimation result.

[0021] The ultrasound probe is brought into contact with the pharynx of the subject who has swallowed the swallowing jelly, and an ultrasound image is generated by analyzing the ultrasound image, thereby obtaining a bubble pattern in the residue of the swallowing jelly, and detecting at least one of the presence or absence of the residue and the area of ​​the residue based on the bubble pattern in the residue. It is preferable that the quality determination unit optimizes the residue detection unit based on the determination result of the quality of the swallowing jelly.

[0022] Also, an ultrasound diagnostic device, a server connected to the ultrasound diagnostic device via a network; The ultrasound diagnostic device includes an ultrasound probe and an image generating unit, Preferably, the server includes at least one of an image analysis unit, a quality determination unit, and a residue detection unit.

[0023] The present invention also provides a method for generating an ultrasound image from a received signal obtained by transmitting and receiving an ultrasound beam while an ultrasound probe is in contact with the outer surface of an unopened package containing swallowable jelly containing gas bubbles; The first bubble pattern in the swallowing jelly is obtained by analyzing the ultrasound image. A method for determining the quality of a swallowable jelly is provided, which determines the quality of the swallowable jelly based on a first bubble pattern. [Effects of the Invention]

[0024] In the present invention, the quality of the swallowing jelly can be determined non-invasively based on the first air bubble pattern in the swallowing jelly shown in the ultrasound image. Furthermore, by using a residue detection unit optimized based on the result of the swallowing jelly quality determination, it is possible to perform a highly accurate examination of the subject for swallowing disorders based on the air bubble pattern in the swallowing jelly residue shown in the ultrasound image of the subject's pharynx. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram illustrating the configuration of an ultrasound system according to a first embodiment of the present invention. [Figure 2] 1 is a block diagram showing the configuration of an ultrasound diagnostic apparatus according to a first embodiment. [Figure 3] FIG. 2 is a block diagram illustrating a configuration of a transmission / reception circuit according to the first embodiment. [Figure 4] FIG. 2 is a block diagram illustrating a configuration of a quality determination processing unit according to the first embodiment. [Figure 5] 4 is a flowchart of one embodiment showing the operation of the ultrasound system of the first embodiment when capturing an ultrasound image. [Figure 6] 10 is a flowchart of one embodiment showing the operation of the ultrasound system of the first embodiment when examining dysphagia. [Figure 7] FIG. 10 is a conceptual diagram showing the configuration of an ultrasound system according to a second embodiment of the present invention. [Figure 8] FIG. 10 is a block diagram illustrating a configuration of a quality determination processing unit according to a second embodiment. [Figure 9] 10 is a flowchart of an embodiment showing the operation of the ultrasound system of the second embodiment when examining dysphagia. [Figure 10] FIG. 1 is a conceptual diagram showing a package containing swallowable jelly. [Figure 11] FIG. 1 is a conceptual diagram showing how ultrasonic waves are reflected on the surface of a bubble. [Figure 12] FIG. 1 is a conceptual diagram showing a bubble pattern captured in an ultrasound image. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, the swallowing jelly quality assessment system and swallowing jelly quality assessment method of the present invention will be described in detail based on preferred embodiments shown in the accompanying drawings.

[0027] Fig. 1 is a conceptual diagram showing the configuration of an ultrasound system according to a first embodiment of the present invention. The ultrasound system shown in Fig. 1 includes an ultrasound diagnostic device having an ultrasound probe 1 and a handheld information terminal 3 connected to the ultrasound probe 1 by wire or wirelessly. The ultrasound diagnostic device of this embodiment is realized by the ultrasound probe 1, the handheld information terminal 3, and an application program for ultrasound diagnosis that runs on the information terminal 3.

[0028] The ultrasonic probe 1 scans a subject with an ultrasonic beam to capture an ultrasonic image, and outputs data corresponding to the ultrasonic image, which in this embodiment is image information data of the ultrasonic image. As shown in Fig. 2, the ultrasonic probe 1 includes a transducer array 11, a transmitting / receiving circuit 14, a signal processing unit 16, an image processing unit 17, a probe-side communication circuit 18, a probe control unit 21, and a battery 24.

[0029] A transmitting / receiving circuit 14 is bidirectionally connected to the transducer array 11. A signal processing unit 16, an image processing unit 17, and a probe-side communication circuit 18 are sequentially connected in series to the transmitting / receiving circuit 14. The signal processing unit 16 and the image processing unit 17 constitute an image information data generation unit 19. A probe control unit 21 is connected to the transmitting / receiving circuit 14, the signal processing unit 16, the image processing unit 17, and the probe-side communication circuit 18. The ultrasound probe 1 also has a built-in battery 24.

[0030] The transmitting and receiving circuit 14, the image information data generating unit 19 (the signal processing unit 16 and the image processing unit 17), and the probe control unit 21 constitute a probe-side processor 25.

[0031] The transducer array 11 has a plurality of ultrasound transducers arranged one-dimensionally or two-dimensionally. Each of these transducers transmits ultrasound waves in accordance with a drive signal supplied from the transmission / reception circuit 14, and receives reflected waves from the subject and outputs an analog reception signal. Each vibrator is constructed using an element in which electrodes are formed on both ends of a piezoelectric body made of, for example, a piezoelectric ceramic such as PZT (Lead Zirconate Titanate), a polymer piezoelectric element such as PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal such as PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0032] The transmission / reception circuit 14, under the control of the probe control unit 21, causes the transducer array 11 to transmit ultrasonic waves and generates sound ray signals by performing reception focusing processing on reception signals output from the transducer array 11 that have received ultrasonic echoes. As shown in Fig. 3, the transmission / reception circuit 14 has a pulser 51 connected to the transducer array 11, and an amplifier 52, an AD (Analog-to-Digital) converter 53, and a beamformer 54 that are connected in series from the transducer array 11 in this order.

[0033] The pulser 51 includes, for example, a plurality of pulse generators, and adjusts the delay amount of each drive signal and supplies it to the plurality of transducers of the transducer array 11 so that the ultrasound waves transmitted from the plurality of transducers form an ultrasound beam based on the transmission delay pattern selected by the probe control unit 21. In this way, when a pulsed or continuous wave voltage is applied to the electrodes of the transducers of the transducer array 11, the piezoelectric material expands and contracts, and each transducer generates a pulsed or continuous wave ultrasound wave, and an ultrasound beam is formed from the composite wave of these ultrasound waves.

[0034] The transmitted ultrasonic beam is reflected by an object such as a part of the subject, and propagates toward the transducer array 11 of the ultrasonic probe 1. Each transducer constituting the transducer array 11 expands and contracts upon receiving the ultrasonic echo propagating toward the transducer array 11 in this manner, generating received signals which are electrical signals, and outputs these received signals to the amplifier unit 52.

[0035] The amplifier 52 amplifies the signals input from the respective transducers constituting the transducer array 11 and transmits the amplified signals to the AD converter 53. The AD converter 53 converts the signals transmitted from the amplifier 52 into digital received data and outputs the received data to the beamformer 54.

[0036] The beam former 54 performs so-called reception focusing processing by adding each delay to each piece of reception data converted by the AD conversion unit 53 in accordance with the sound speed or sound speed distribution set based on the reception delay pattern selected by the probe control unit 21. By this reception focusing processing, each piece of reception data converted by the AD conversion unit 53 is phased and added, and a sound ray signal in which the focus of the ultrasonic echo is narrowed is generated.

[0037] The image information data generator 19 generates image information data based on the sound ray signals generated by the transmitting and receiving circuit 14. The image information data generator 19 includes the signal processor 16 and the image processor 17, as described above.

[0038] The signal processing unit 16 generates image signal data before imaging into an ultrasound image based on the sound ray signals generated by the transmitting and receiving circuit 14 under the control of the probe control unit 21. More specifically, the signal processing unit 16 performs signal processing on the sound ray signals generated by the beam former 54 of the transmitting and receiving circuit 14, for example, performs correction for attenuation caused by the propagation distance in accordance with the depth of the position where the ultrasound is reflected, and then performs envelope detection processing to generate a signal representing tomographic image information regarding tissue in the subject as image signal data before imaging.

[0039] Under the control of the probe control unit 21, the image processing unit 17 generates an ultrasound image as image information data generated by the image information data generation unit 19 based on the image signal data generated by the signal processing unit 16. More specifically, the image processing unit 17 raster-converts the pre-imaging image signal data generated by the signal processing unit 16 into an image signal conforming to a normal television signal scanning method, and performs various image processes on the thus converted image signal, such as brightness correction, tone correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction, conforming to the display format of the monitor 34, to generate an ultrasound image (ultrasound image signal), and then outputs the ultrasound image generated by the image information data generation unit 19 to the probe side communication circuit 18 as image information data.

[0040] The transmitting / receiving circuit 14, the signal processing unit 16, and the image processing unit 17 constitute an image generating unit of the present invention. The image generation unit generates an ultrasound image, in this embodiment, as image information data, from the received signals obtained by transmitting and receiving ultrasound beams to and from the subject using the ultrasound probe 1 (more precisely, the transducer array 11).

[0041] The probe-side communication circuit 18 transmits the image information data generated by the image processing unit 17 by wire or wirelessly under the control of the probe control unit 21. In this embodiment, the probe-side communication circuit 18 includes an antenna for transmitting and receiving radio waves, and transmits the ultrasound image wirelessly by modulating a carrier based on the ultrasound image generated by the image processing unit 17 to generate a transmission signal, supplying the transmission signal to the antenna, and transmitting the radio wave from the antenna. Carrier modulation methods include ASK (Amplitude Shift Keying), PSK (Phase Shift Keying), QPSK (Quadrature Phase Shift Keying), and 16QAM (16 Quadrature Amplitude Modulation). The probe-side communication circuit 18 can also connect the ultrasonic probe 1 and the information terminal 3 by wire using a cable such as a USB (Universal Serial Bus) cable.

[0042] The probe control unit 21 controls each unit of the ultrasonic probe 1 based on pre-stored programs, etc. More specifically, the probe control unit 21 controls the transmission / reception circuit 14 so that ultrasonic beams are transmitted and ultrasonic echoes are received based on a pre-set examination mode and scanning method. The probe control unit 21 also controls the signal processing unit 16 and image processing unit 17 of the image information data generation unit 19 so that pre-set signal processing is performed on the sound ray signal and pre-set image processing is performed on the image signal data. Furthermore, the probe control unit 21 controls the probe-side communication circuit 18 so that image signal data is transmitted at a pre-set transmission radio wave intensity.

[0043] Here, the examination mode refers to any of the examination modes available in an ultrasound diagnostic device, such as B (brightness) mode, CF (color Doppler) mode, PD (power Doppler) mode, M (motion) mode, PW (pulsed Doppler) mode, CW (continuous wave Doppler) mode, etc., and the scanning method refers to any of the scanning methods, such as electronic sector scanning method, electronic linear scanning method, electronic convex scanning method, etc.

[0044] The battery 24 is built into the ultrasonic probe 1 and supplies power to each circuit of the ultrasonic probe 1.

[0045] Next, the information terminal 3 is a handheld terminal device such as a smartphone or tablet PC (Personal Computer), and displays an ultrasound image based on data corresponding to the ultrasound image captured by the ultrasound probe 1. As shown in FIG. 2 , the information terminal 3 includes a terminal-side communication circuit 32, a display control unit 33, a quality assessment processing unit 35, a residue detection unit 38, a terminal control unit 36, a monitor 34, and an input device 37.

[0046] A display control unit 33 and a monitor 34 are connected in series to the terminal-side communication circuit 32. A quality assessment processing unit 35 and a residue detection unit 38 are also connected to the terminal-side communication circuit 32, and the residue detection unit 38 is connected to the quality assessment processing unit 35. A display control unit 33 is also connected to the quality assessment processing unit 35 and the residue detection unit 38. A terminal control unit 36 ​​is connected to the terminal-side communication circuit 32, the display control unit 33, the quality assessment processing unit 35, and the residue detection unit 38, and an input device 37 is also connected to the terminal control unit 36.

[0047] In this embodiment, the probe-side communication circuit 18 of the ultrasonic probe 1 and the terminal-side communication circuit 32 of the information terminal 3 are wirelessly connected via wireless communication, thereby connecting the ultrasonic probe 1 and the information terminal 3 to enable bidirectional information exchange.

[0048] The terminal-side communication circuit 32 receives, by wire or wirelessly, image information data transmitted from the probe-side communication circuit 18 of the ultrasound probe 1 under the control of the terminal control unit 36. In this embodiment, the terminal-side communication circuit 32 includes an antenna for transmitting and receiving radio waves, receives a transmission signal transmitted wirelessly from the probe-side communication circuit 18 via the antenna, and demodulates the received transmission signal to output an ultrasound image (ultrasound image signal), which is image information data.

[0049] The display control unit 33, under the control of the terminal control unit 36, causes the monitor 34 to display various types of information. For example, the display control unit 33 performs predetermined processing on an ultrasound image, which is image information data received by the terminal-side communication circuit 32, and causes the monitor 34 to display the image. The display control unit 33 also causes the monitor 34 to display various messages, various operation screens, and the like.

[0050] The monitor 34 displays various types of information. As described above, the monitor 34 displays various messages, various operation screens, and the like in addition to ultrasound images under the control of the display control unit 33. Examples of the monitor 34 include an LCD (Liquid Crystal Display) and an organic EL (Electro-Luminescence) display.

[0051] The input device 37 is used by the user to input various instructions through input operations, and in this embodiment includes a touch panel or the like that allows the user to input various instructions through touch operations.

[0052] The quality assessment processing unit 35 performs various processes related to the assessment of the quality of the swallowing jelly under the control of the terminal control unit 36. As shown in Fig. 4, the quality assessment processing unit 35 has an image analysis unit 60, a quality assessment unit 62, and a reference image memory 64. The image analysis unit 60 is connected to the terminal-side communication circuit 32. The image analysis unit 60 and the reference image memory 64 are connected to the quality assessment unit 62, and the display control unit 33 and the residue detection unit 38 are connected to the quality assessment unit 62.

[0053] The image analysis unit 60 analyzes an ultrasound image generated when the ultrasound probe 1 is in contact with the outer surface of an unopened package containing swallowable jelly containing air bubbles, thereby obtaining a first air bubble pattern within the swallowable jelly shown in the ultrasound image.

[0054] The swallowing jelly is, for example, a typical edible jelly with a 6-month expiration date, and is stored in a cool, dark place or a refrigerator at 10°C or below. The bubbles contained in the swallowing jelly are, but are not particularly limited to, granular bubbles of carbon dioxide or the like with a diameter of about 1 to 2 mm. Furthermore, a predetermined number of bubbles are enclosed in a predetermined region of interest within the swallowing jelly at a predetermined density and a predetermined uniformity. The number, size, density, and uniformity of the bubbles are not particularly limited.

[0055] For example, as shown in Fig. 10, a package of swallowable jelly has a cup 80 and an upper lid 81. Swallowable jelly 82 is contained in the cup 80, which has an upper opening, and is enclosed in the package by sealing the upper opening of the cup 80 with the upper lid 81.

[0056] The user can take an ultrasound image of the swallowable jelly 82 sealed in the package with the ultrasound probe 1 in contact with the outer surface of the cup 80, or with the ultrasound probe 1 in contact with the outer surface of the top lid 81 when the swallowable jelly 82 and the top lid 81 are in contact with each other and there is no air layer between them. During the image capture, ultrasound waves of, for example, about 5 MHz to 10 MHz are transmitted to individually image the air bubbles 83 in the swallowable jelly 82.

[0057] The image analysis unit 60 acquires, as the first bubble pattern, at least one of the bubble feature amounts, for example, the number of bubbles, the bubble size, the bubble density, and the bubble uniformity. Note that the image analysis unit 60 may acquire bubble feature amounts other than these.

[0058] The image analysis unit 60 can acquire the first bubble pattern by analyzing the ultrasound image using, but is not limited to, a machine learning model trained using deep learning, or various known image analysis techniques.

[0059] For example, the machine learning model of the image analysis unit 60 is a trained model that uses training ultrasound images of unopened packages containing swallowing jelly containing air bubbles and the air bubble patterns of the swallowing jelly shown in these training ultrasound images as training data, and learns the relationship between the training ultrasound images and the air bubble patterns of the swallowing jelly shown in these training ultrasound images using multiple training data. The machine learning model takes as input an ultrasound image of an unopened package containing swallowable jelly containing air bubbles, and outputs an estimated result of the first air bubble pattern of the swallowable jelly shown in the ultrasound image. The image analysis unit 60 acquires a first air bubble pattern in the swallowing jelly based on the estimation result estimated by the machine learning model.

[0060] The reference image memory 64 is a memory for storing a plurality of reference ultrasound images in which the bubble pattern in the swallowing jelly when sealed in the package changes sequentially according to the deterioration of the quality of the swallowing jelly.

[0061] Here, deterioration in the quality of the swallowing jelly includes, for example, deterioration over time, deterioration due to temperature changes, deterioration due to exposure to ultraviolet rays or direct sunlight, etc. Other factors of quality deterioration may also be included. The reference image memory 64 can store multiple reference ultrasound images that have changed sequentially due to a combination of one or more quality deterioration factors by combining one or more quality deterioration factors.

[0062] For example, the bubble pattern in swallowing jelly deteriorates over time, causing its shape to change. For example, bubbles come into contact with each other and merge, gradually increasing the size of the bubbles, decreasing the number of bubbles, decreasing the bubble density, and decreasing the uniformity of the bubbles. The same is true for other deterioration factors. The deterioration of the bubble pattern accelerates with changes in temperature or with longer exposure to ultraviolet light or direct sunlight, and may make the product unusable for swallowing disorder testing, even if it is within its expiration date.

[0063] The quality determination unit 62 determines the quality of the swallowing jelly based on the first air bubble pattern acquired by the image analysis unit 60. In addition, the quality determination unit 62 optimizes the residue detection unit 38 based on the determination result of the quality of the swallowing jelly.

[0064] Similarly, the quality determination unit 62 can determine the quality of the swallowing jelly using a machine learning model or various known image analysis techniques.

[0065] For example, the machine learning model of the quality judgment unit 62 is a trained model that has learned the relationship between the air bubble pattern in the swallowing jelly shown in the training ultrasound image and the quality of the swallowing jelly using multiple training data, which are the air bubble pattern in the swallowing jelly shown in the training ultrasound image and the quality of the swallowing jelly. The machine learning model takes the first bubble pattern in the swallowing jelly shown in the ultrasound image as input and outputs an estimation result of the quality of the swallowing jelly. The quality determination unit 62 determines the quality of the swallowing jelly based on the estimation result estimated by the machine learning model.

[0066] The residue detection unit 38, under the control of the terminal control unit 36, analyzes ultrasound images generated when the ultrasound probe 1 is in contact with the pharynx of the subject who has swallowed the swallowing jelly, to obtain a pattern of air bubbles in the residue of the swallowing jelly that remains in the subject's pharynx, for example, the pyriform sinus, during swallowing, and detects the presence or absence of residue based on the pattern of air bubbles in the residue of the swallowing jelly. The residue detection unit 38 can also detect the area of ​​residue using semantic segmentation, bounding box detection, etc.

[0067] Similarly, the residue detection unit 38 can detect the presence or absence of swallowed residue, the area of ​​residue, etc., using a machine learning model or various known image analysis techniques, etc.

[0068] For example, the machine learning model of the residue detection unit 38 is a trained model that uses a training ultrasound image of the subject's pharynx and at least one of the presence or absence or area of ​​swallowing residue in this training ultrasound image as training data, and has learned the relationship between the training ultrasound image and at least one of the presence or absence or area of ​​swallowing residue in this training ultrasound image using multiple training data. The machine learning model receives an ultrasound image of the subject's pharynx as input and outputs an estimation result that estimates at least one of the presence or absence of swallowing residue and the area of ​​the residue in the ultrasound image. The residue detection unit 38 detects at least one of the presence or absence and area of ​​swallowing residue based on the estimation result estimated by the machine learning model.

[0069] The display control unit 33, the quality determination processing unit 35, the residue detection unit 38, and the terminal control unit 36 ​​constitute a terminal-side processor 39. Furthermore, in the ultrasound system of the first embodiment, at least the ultrasound probe 1 (transducer array 11), the image generation unit, and the quality judgment processing unit 35 including the image analysis unit 60, the quality judgment unit 62, and the reference image memory 64 constitute the quality judgment system for swallowing jelly of the present invention.

[0070] Next, the operation of the ultrasound system of the first embodiment when capturing an ultrasound image will be described with reference to the flowchart of FIG.

[0071] With the ultrasonic probe 1 in contact with the body surface of the subject, the transmission / reception circuit 14 starts transmitting ultrasonic waves based on instructions from the user inputted from the input device 37, and a sound ray signal is generated (step S1).

[0072] That is, under the control of the probe control unit 21, ultrasonic beams are transmitted from the transducers of the transducer array 11 into the subject in accordance with a drive signal from the pulser 51 of the transmission / reception circuit . The ultrasonic echo from the subject based on the ultrasonic beam transmitted from the pulser 51 is received by each transducer of the transducer array 11, and a received signal, which is an analog signal, is output from each transducer of the transducer array 11 that receives the ultrasonic echo. The received signal, which is an analog signal output from each transducer of the transducer array 11, is amplified by the amplifier 52 of the transmission / reception circuit 14 and AD converted by the AD converter 53 to obtain received data. The beamformer 54 performs reception focus processing on this reception data, thereby generating sound ray signals.

[0073] Next, based on the sound ray signals generated by the beam former 54 of the transmission / reception circuitry 14, the image information data generator 19 generates an ultrasound image as image information data (step S2).

[0074] That is, the sound ray signals generated by the beamformer 54 are subjected to various signal processing by the signal processing unit 16 of the image information data generating unit 19, and a signal representing tomographic image information regarding tissues in the subject is generated as image signal data before imaging. The image signal data generated by the signal processing unit 16 is raster converted by the image processing unit 17 and further subjected to various image processing, thereby generating an ultrasound image as image information data.

[0075] The ultrasound image generated by the image processing unit 17 is wirelessly transmitted from the probe side communication circuit 18 to the information terminal 3 (step S3).

[0076] Next, the ultrasound image wirelessly transmitted from the probe-side communication circuit 18 of the ultrasound probe 1 is received by the terminal-side communication circuit 32 under the control of the terminal control unit 36 ​​of the information terminal 3 (step S4).

[0077] Next, the ultrasonic image received by the terminal side communication circuit 32 is subjected to predetermined processing by the display control unit 33 and displayed on the monitor 34 (step S5).

[0078] Next, the operation of the ultrasound system of the first embodiment when examining dysphagia will be described with reference to the flowchart of FIG.

[0079] First, the user photographs a package of swallowable jelly to be subjected to quality assessment, that is, an unopened package in which swallowable jelly containing air bubbles is sealed (Step S11).

[0080] In this case, with the ultrasonic probe 1 in contact with the outer surface of the package of the swallowable jelly, the transmitting / receiving circuit 14 starts transmitting ultrasonic waves, and a sound ray signal is generated. Next, based on the sound ray signals generated by the transmitting and receiving circuit 14, the image information data generating unit 19 generates an ultrasonic image of the package of the swallowable jelly. The ultrasonic image of the swallowable jelly is transmitted from the ultrasonic probe 1 to the information terminal 3 and received by the terminal-side communication circuit 32 of the information terminal 3. Next, the ultrasonic image of the swallowing jelly received by the terminal side communication circuit 32 is subjected to predetermined processing by the display control unit 33 and displayed on the monitor 34.

[0081] The ultrasonic probe 1 can be any of linear, sector, or convex types, but since the size of the swallowing jelly packaging is relatively small, it is preferable to use a linear or sector ultrasonic probe 1, which has a relatively small contact surface.

[0082] When capturing an ultrasound image of the swallowing jelly, the difference in impedance between the carbon dioxide gas in the bubbles 83 and the swallowing jelly is large, so the ultrasound is totally reflected on the surface of the bubbles 83, and the crescent-shaped area surrounded by the dashed line is depicted as a high-brightness ultrasound tomographic image, as shown in Fig. 11. As a result, the ultrasound image of the swallowing jelly includes a plurality of crescent-shaped high-brightness ultrasound tomographic images corresponding to the plurality of bubbles 83, as shown in Fig. 12.

[0083] Next, the image analysis unit 60 analyzes the ultrasound image of the swallowing jelly received by the terminal side communication circuit 32, and obtains a first air bubble pattern in the swallowing jelly that appears in the ultrasound image of the swallowing jelly (step S12).

[0084] Next, the quality of the swallowable jelly is determined by the quality determining unit 62 based on the first bubble pattern acquired by the image analyzing unit 60 (step S13).

[0085] For example, the quality determination unit 62 detects a change in the shape of the first air bubble pattern relative to the air bubble pattern in the swallowing jelly when sealed in the package, and determines the quality of the swallowing jelly based on the change in the shape of the first air bubble pattern. In other words, the quality determination unit 62 detects the similarity between the shape of the air bubble pattern in the swallowing jelly when sealed in the package and the shape of the first air bubble pattern, and determines the quality of the swallowing jelly based on the similarity.

[0086] For example, the quality determination unit 62 determines that the quality of the swallowing jelly is more degraded as the air bubbles become larger. Also, the quality determination unit 62 determines that the quality of the swallowing jelly is more degraded as the number of air bubbles becomes smaller, as the density of the air bubbles becomes lower, and as the uniformity of the air bubbles becomes worse.

[0087] The method for detecting the change in the shape of the first gas bubble pattern is not particularly limited, but for example, a plurality of reference ultrasound images in which the gas bubble pattern changes sequentially according to the deterioration of the quality of the swallowing jelly are stored in the reference image memory 64, and the change in the shape of the first gas bubble pattern can be detected by comparing the ultrasound image of the swallowing jelly with the plurality of reference ultrasound images stored in the reference image memory 64. This makes it possible to judge the quality of the swallowing jelly at a plurality of stages based on the change in the shape of the first gas bubble pattern.

[0088] When the determination of the quality of the swallowing jelly is completed, the quality determination unit 62 optimizes the residue detection unit 38 based on the determination result of the quality of the swallowing jelly (step S14).

[0089] The residue detection unit 38 includes a plurality of machine learning models corresponding to a plurality of bubble patterns that differ, for example, in the number of bubbles, size of bubbles, density of bubbles, and uniformity of bubbles. The quality determination unit 62 optimizes the residue detection unit 38 by switching between the plurality of machine learning models of the residue detection unit 38 based on the swallowing jelly determination result. For example, if the quality determination unit 62 determines that the swallowing jelly has a small number of bubbles, it optimizes the residue detection unit 38 by switching to a machine learning model corresponding to a bubble pattern with a small number of bubbles. In this way, the residue detection unit 38 can detect the presence or absence of residue in the swallowing jelly, the area of ​​the residue, and the like, using the optimal machine learning model corresponding to the quality of the swallowing jelly.

[0090] The method for optimizing the residue detection unit 38 is not particularly limited, and for example, the parameters of the machine learning model may be changed, the number of layers of the machine learning model may be changed, the modules of each layer may be changed, or other optimization methods may be used.

[0091] Next, the display control unit 33 displays the results of the swallowing jelly quality assessment and information regarding the optimization of the residue detection unit 38 on the monitor 34. For example, a message such as "Changes in the number of air bubbles detected. The machine learning model has been optimized" is displayed. This allows the user to check the quality of the swallowing jelly and determine whether or not the swallowing jelly can be used for testing for swallowing disorders. The user can also confirm that the residue detection unit 38 has been optimized.

[0092] There is no particular limitation on what is displayed as the judgment result, but for example, it is possible to display a message indicating which of the multiple stages mentioned above the quality of the swallowing jelly is at, or a message regarding changes in the number of bubbles, changes in the size of the bubbles, changes in the density of the bubbles, and changes in the uniformity of the bubbles. There are no particular limitations on what is displayed as information related to optimization, but it is possible to display various types of information related to optimization of the residue detection unit 38. Note that the information related to optimization may or may not be displayed.

[0093] Here, if the user judges that the swallowing jelly is not of good quality by referring to the judgment result of the quality of the swallowing jelly (No in step S15), the user ends the subsequent processing. In this case, the user can reselect the swallowing jelly to be judged for quality and repeat the processing from the beginning. On the other hand, if the user determines that the swallowing jelly is of good quality (Yes in step S15), an ultrasound image of the subject's pharynx is taken (step S16).

[0094] In this case, when the ultrasonic probe 1 is in contact with the pharynx of a subject who has swallowed a swallowing jelly that has been determined to be a good product, for example, one of the left and right pharynx, the transmitting and receiving circuit 14 starts transmitting ultrasonic waves and an ultrasonic image of the subject's pharynx is taken. The subsequent operations are the same as when taking an image of a package of swallowing jelly, and the ultrasonic image of the subject's pharynx is displayed on the monitor 34.

[0095] Next, the residue detection unit 38 analyzes the ultrasound image of the subject's pharynx to obtain the bubble pattern within the residue of the swallowing jelly (step S17), and detects the presence or absence of residue, the area of ​​residue, etc. based on the bubble pattern within the residue of the swallowing jelly (step S18).

[0096] In the ultrasound system of the first embodiment, the quality of the swallowing jelly can be determined noninvasively based on the first air bubble pattern in the swallowing jelly shown in the ultrasound image. Furthermore, the residue detection unit 38, which is optimized based on the result of the determination of the swallowing jelly quality, can perform a highly accurate examination of the subject for swallowing disorders based on the air bubble pattern in the swallowing jelly residue shown in the ultrasound image of the subject's pharynx.

[0097] Next, Fig. 7 is a conceptual diagram showing the configuration of an ultrasound system according to a second embodiment of the present invention. The ultrasound system according to the second embodiment shown in Fig. 7 includes a quality judgment processing unit 35B instead of the quality judgment processing unit 35 of the ultrasound system according to the first embodiment shown in Fig. 1, and further includes an optical camera 5. The quality judgment processing unit 35B is connected to the optical camera 5. The following mainly describes the optical camera 5 and the quality judgment processing unit 35B.

[0098] The optical camera 5 is an example of the optical image acquisition unit of the present invention, and captures various optical images. For example, the optical camera 5 captures an optical image of an unopened package containing swallowable jelly containing air bubbles. The optical camera 5 is not particularly limited, and may be various digital cameras, such as a digital single-lens reflex camera or a digital camera included in an information terminal 3 such as a smartphone. The optical image may be a still image or a moving image.

[0099] In the ultrasound system of the second embodiment, the cup 80 and the top lid 81 of the swallowing jelly package shown in Fig. 10 are made of a substantially transparent material. It is sufficient that at least a portion of the cup 80 or the top lid 81 is made of a substantially transparent material. The user can take an optical image of the swallowable jelly 82 enclosed in the packaging body while bringing the ultrasonic probe 1 into contact with a substantially transparent portion of the outer surface of the packaging body.

[0100] The quality assessment processing unit 35B performs various processes related to the assessment of the quality of the swallowing jelly under the control of the terminal control unit 36, similar to the quality assessment processing unit 35 of the first embodiment. As shown in Fig. 8, the quality assessment processing unit 35B has an image analysis unit 60B, a reference image memory 64B, and a quality assessment unit 62B. The image analysis unit 60B is connected to the terminal side communication circuit 32. The image analysis unit 60B and the reference image memory 64B are connected to the quality assessment unit 62B, and the display control unit 33 and the residue detection unit 38 are connected to the quality assessment unit 62B.

[0101] The image analysis unit 60B, like the image analysis unit 60 of the first embodiment, analyzes an ultrasound image of the swallowing jelly to obtain a first air bubble pattern in the swallowing jelly shown in the ultrasound image. In addition, the image analysis unit 60B analyzes an optical image of the swallowing jelly taken by the optical camera 5 to obtain optical feature quantities of the swallowing jelly shown in the optical image.

[0102] The optical feature of the swallowing jelly is not particularly limited, but includes at least one of the color of the swallowing jelly and the second bubble pattern within the swallowing jelly. It is also possible to include other optical feature of the swallowing jelly. The color of the swallowing jelly may gradually fade or change depending on the deterioration of its quality. The feature acquired as the second bubble pattern is the same as that of the first bubble pattern.

[0103] The reference image memory 64B stores a plurality of reference ultrasound images in which the bubble pattern in the swallowing jelly when sealed in the package changes sequentially according to the deterioration of the quality of the swallowing jelly, similar to the reference image memory 64 in the first embodiment. In addition, the reference image memory 64B stores a plurality of reference optical images in which the optical feature amount of the swallowing jelly when sealed in the package changes sequentially according to the deterioration of the quality of the swallowing jelly.

[0104] The quality determination unit 62B determines the quality of the swallowing jelly based on the first air bubble pattern and optical feature amount acquired by the image analysis unit 60B. As shown in FIG. 8, the quality determination unit 62B has a first determination unit 70, a second determination unit 71, and a determination result integration unit 72. The first determination unit 70 and the second determination unit 71 are connected to the image analysis unit 60 and the reference image memory 64, respectively. The first determination unit 70 and the second determination unit 71 are also connected to the determination result integration unit 72, which is connected to the display control unit 33 and the residue detection unit 38.

[0105] The first determination unit 70 outputs a first determination result that determines the quality of the swallowing jelly based on the first air bubble pattern in the swallowing jelly analyzed by the image analysis unit 60B.

[0106] The second determination unit 71 outputs a second determination result that determines the quality of the swallowing jelly based on the optical feature amount of the swallowing jelly analyzed by the image analysis unit 60B.

[0107] Similarly, the first determination unit 70 and the second determination unit 71 can determine the quality of the swallowing jelly using a machine learning model or various known image analysis techniques.

[0108] For example, the machine learning model of the first determination unit 70 is similar to the machine learning model of the quality determination unit 62 in the first embodiment.

[0109] The machine learning model of the second judgment unit 71 is a trained model that has learned the relationship between the optical features of the swallowing jelly shown in the training optical image and the quality of the swallowing jelly using multiple training data, which are the optical features of the swallowing jelly shown in the training optical image and the quality of the swallowing jelly. The machine learning model takes the optical features of the swallowed jelly captured in the optical image as input and outputs an estimate of the quality of the swallowed jelly. The second determination unit 71 determines the quality of the swallowing jelly based on the estimation result estimated by the machine learning model.

[0110] The judgment result integrating unit 72 judges the quality of the swallowing jelly by integrating the first judgment result of the quality of the swallowing jelly judged by the first judgment unit 70 based on the first bubble pattern and the second judgment result of the quality of the swallowing jelly judged based on the optical feature amount by the second judgment unit 71. For example, the judgment result integrating unit 72 judges the quality of the swallowing jelly by weighting and integrating the first judgment result and the second judgment result.

[0111] The quality determining unit 62B may determine the quality of the jelly to be swallowed by analyzing both the ultrasonic image and the optical image at once using a multimodal model.

[0112] The multimodal model is a trained model that uses the air bubble patterns of the swallowing jelly shown in the training ultrasound images, the optical features of the swallowing jelly shown in the training optical images, and the quality of the swallowing jelly as training data, and has learned the relationship between the air bubble patterns of the swallowing jelly shown in the training ultrasound images, the optical features of the swallowing jelly shown in the training optical images, and the quality of the swallowing jelly from multiple training data. The multimodal model takes as input the first air bubble pattern of the swallowed jelly shown in the ultrasound image and the optical features of the swallowed jelly shown in the optical image, and outputs an estimation result of the quality of the swallowed jelly. The quality determining unit 62B determines the quality of the swallowing jelly based on the estimation result obtained by the multimodal model.

[0113] In the ultrasound system of the second embodiment, at least an ultrasound probe 1 (transducer array 11), an image generation unit, an image analysis unit 60B, a quality judgment processing unit 35B including a quality judgment unit 62B and a reference image memory 64B, and an optical camera 5 constitute the quality judgment system for swallowing jelly of the present invention.

[0114] Next, the operation of the ultrasound system of the second embodiment when examining dysphagia will be described with reference to the flowchart of FIG.

[0115] First, the user takes an ultrasonic image of the package of the swallowable jelly to be evaluated for quality (step S21).

[0116] Next, the user photographs an optical image of the same package of swallowable jelly (step S22). In this case, the optical camera 5 photographs the package of swallowable jelly, and an optical image of the swallowable jelly is obtained.

[0117] Next, the image analysis unit 60 analyzes the ultrasound image of the swallowing jelly to obtain a first bubble pattern in the swallowing jelly shown in the ultrasound image (step S23). Also, the image analysis unit 60 analyzes the optical image of the swallowing jelly to obtain optical feature quantities of the swallowing jelly shown in the optical image (step S24).

[0118] Next, the quality of the swallowing jelly is determined by the quality determining unit 62B based on the first air bubble pattern and the optical feature amount acquired by the image analyzing unit 60B.

[0119] In this case, the first determination unit 70 determines the quality of the jelly to be swallowed based on the first air bubble pattern in the jelly to be swallowed that is captured in the ultrasound image, and outputs the first determination result (step S25). For example, the first determination unit 70 detects a change in the shape of the first air bubble pattern relative to the air bubble pattern in the swallowable jelly when it is sealed in the package, and determines the quality of the swallowable jelly based on the change in the shape of the first air bubble pattern. The method for detecting the change in the shape of the first bubble pattern is the same as in the first embodiment.

[0120] Furthermore, the second determination unit 71 determines the quality of the jelly to be swallowed based on the optical feature amount of the jelly to be swallowed captured in the optical image, and outputs the second determination result (step S26). For example, the second determination unit 71 detects a change in the optical feature amount of the swallowing jelly relative to the optical feature amount of the swallowing jelly when it is sealed in the package, and determines the quality of the swallowing jelly based on this change in the optical feature amount.

[0121] The method for detecting changes in optical features is not particularly limited, but for example, a plurality of reference optical images in which the optical features of the swallowing jelly at the time of sealing in the package have sequentially changed in accordance with the deterioration of the quality of the swallowing jelly are stored in the reference image memory 64, and changes in the optical features of the swallowing jelly can be detected by comparing the optical image of the swallowing jelly with the plurality of reference optical images stored in the reference image memory 64. This makes it possible to judge the quality of the swallowing jelly at a plurality of stages based on changes in the optical features of the swallowing jelly.

[0122] Next, the judgment result integration unit 72 integrates the first judgment result of the quality of the swallowing jelly judged by the first judgment unit 70 and the second judgment result of the quality of the swallowing jelly judged by the second judgment unit 71 to judge the quality of the swallowing jelly (step S27).

[0123] The subsequent operations are the same as those in the first embodiment. Note that an ultrasonic image may be captured after capturing an optical image.

[0124] In the ultrasound system of the second embodiment, the quality of the swallowing jelly can be determined noninvasively based on both the first air bubble pattern in the swallowing jelly shown in the ultrasound image and the optical feature amount of the swallowing jelly. Also, as in the ultrasound system of the first embodiment, the residue detection unit 38, which is optimized based on the determination result of the quality of the swallowing jelly, can perform a highly accurate examination of the subject for swallowing disorders based on the air bubble pattern in the swallowing jelly residue shown in the ultrasound image of the subject's pharynx.

[0125] The ultrasound system may further include a server, which may execute at least one of the functions of the image analysis unit 60, the quality determination unit 62, and the residue detection unit .

[0126] For example, when the function of the residue detection unit 38 is executed in the server, the server is provided with a residue detection unit equivalent to the residue detection unit 38. Then, an ultrasound image is transmitted from the terminal-side communication circuit 32 of the information terminal 3 to the server via the network, and the presence or absence of residue, the area of ​​residue, etc. detected by analyzing the ultrasound image by the residue detection unit provided in the server are transmitted from the server to the terminal-side communication circuit 32 via the network. As a result, even if the information terminal 3 has low processing power, such as a smartphone, it is possible to use a server with high processing power to execute the function of the residue detection unit 38. Furthermore, the function of the residue detection unit provided in the server can be used by multiple ultrasound diagnostic devices. The same applies when the function of the image analysis unit 60 or the quality determination unit 62 is executed in the server.

[0127] The present invention is not limited to handheld ultrasound systems, but can also be applied to stationary ultrasound systems or portable ultrasound systems in which the information terminal is realized by a laptop-type terminal device. Furthermore, the image information data generator 19 may be provided in the ultrasound probe 1 or the information terminal 3.

[0128] In the device of the present invention, the hardware configuration of processing units that perform various processes, such as the transceiver circuit 14, signal processing unit 16, image processing unit 17, probe control unit 21, display control unit 33, quality assessment processing units 35, 35B, residue detection unit 38, and terminal control unit 36, may be dedicated hardware or various processors or computers that execute programs. Furthermore, the hardware configuration of reference image memories 64, 64B may be dedicated hardware or may be a memory such as a semiconductor memory and a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0129] Various types of processors include CPUs (Central Processing Units), which are general-purpose processors that execute software (programs) and function as various processing units, programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as FPGAs (Field Programmable Gate Arrays), and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors with circuit configurations designed specifically for performing specific processes.

[0130] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types, for example, a combination of multiple FPGAs, or a combination of an FPGA and a CPU, etc. Also, multiple processing units may be configured with one of the various processors, or two or more of the multiple processing units may be combined into one processor.

[0131] For example, as typified by server and client computers, one processor is configured by combining one or more CPUs and software, and this processor functions as multiple processing units. Another form is the use of a processor that realizes the functions of an entire system including multiple processing units on a single IC (Integrated Circuit) chip, as typified by system-on-chip (SoC).

[0132] Furthermore, the hardware configuration of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0133] The method of the present invention can be implemented by, for example, a program that causes a computer to execute each step. Also, a computer-readable recording medium on which this program is recorded can be provided.

[0134] Although the present invention has been described in detail above, the present invention is not limited to the above-described embodiments, and various improvements and modifications may be made without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0135] 1 ultrasound probe, 3 information terminal, 5 optical camera, 11 transducer array, 14 transmission / reception circuit, 16 signal processing unit, 17 image processing unit, 18 probe side communication circuit, 19 image information data generation unit, 21 probe control unit, 24 battery, 25 probe side processor, 32 terminal side communication circuit, 33 display control unit, 34 monitor, 35, 35B quality judgment processing unit, 36 terminal control unit, 37 input device, 38 residue detection unit, 39 terminal side processor, 51 pulser, 52 amplifier unit, 53 AD conversion unit, 54 beam former, 60, 60B image analysis unit, 62, 62B quality judgment unit, 64, 64B reference image memory, 70 first judgment unit, 71 second judgment unit, 72 judgment result integration unit, 80 cup, 81 upper lid, 82 swallowing jelly, 83 air bubbles.

Claims

1. an ultrasound probe; an image generating unit that generates an ultrasound image from a received signal obtained by transmitting and receiving an ultrasound beam to and from a subject using the ultrasound probe; an image analysis unit that acquires a first air bubble pattern in the swallowable jelly by analyzing an ultrasound image generated when the ultrasound probe is in contact with an outer surface of an unopened package in which the swallowable jelly containing air bubbles is enclosed; A quality determination system for swallowing jelly, comprising: a quality determination unit that determines the quality of the swallowing jelly based on the first bubble pattern.

2. The swallowable jelly quality assessment system of claim 1, wherein the quality assessment unit detects a change in the shape of the first air bubble pattern relative to the air bubble pattern in the swallowable jelly when it is sealed in the packaging, and assesses the quality of the swallowable jelly based on the change in the shape of the first air bubble pattern.

3. a memory in which a plurality of reference ultrasound images in which the bubble pattern sequentially changes according to deterioration of the quality of the swallowing jelly are stored; The swallowable jelly quality assessment system of claim 2, wherein the quality assessment unit detects changes in the shape of the first bubble pattern by comparing the ultrasound image with the plurality of reference ultrasound images stored in the memory.

4. The swallowing jelly quality assessment system of claim 1, wherein the quality assessment unit includes a machine learning model that uses the first bubble pattern as input and outputs an estimation result of the quality of the swallowing jelly, and determines the quality of the swallowing jelly based on the estimation result.

5. an optical image acquisition unit that acquires an optical image of the unopened package; The image analysis unit further analyzes the optical image to obtain an optical feature of the swallowing jelly, The swallowable jelly quality determination system according to claim 1 , wherein the quality determination unit determines the quality of the swallowable jelly based on the first bubble pattern and the optical feature amount.

6. The quality assessment system for swallowing jelly described in claim 5, wherein the quality assessment unit detects changes in the shape of the first gas bubble pattern relative to the gas bubble pattern in the swallowing jelly when sealed in the packaging, detects changes in the optical characteristics of the swallowing jelly relative to the optical characteristics of the swallowing jelly when sealed in the packaging, and assesses the quality of the swallowing jelly based on the changes in the shape of the first gas bubble pattern and the changes in the optical characteristics of the swallowing jelly.

7. a memory in which a plurality of reference ultrasound images in which the bubble pattern has sequentially changed according to the deterioration of the quality of the swallowing jelly and a plurality of reference optical images in which the optical feature amount of the swallowing jelly when enclosed in the package has sequentially changed according to the deterioration of the quality of the swallowing jelly are stored; The quality assessment system for swallowing jelly described in claim 6, wherein the quality assessment unit detects changes in the shape of the first bubble pattern by comparing the ultrasound image with the multiple reference ultrasound images stored in the memory, and detects changes in the optical features of the swallowing jelly by comparing the optical image with the multiple reference optical images stored in the memory.

8. The quality assessment system for swallowing jelly described in claim 5, wherein the quality assessment unit includes a first machine learning model that takes the first bubble pattern as input and outputs a first estimation result that estimates the quality of the swallowing jelly, and a second machine learning model that takes optical features of the swallowing jelly as input and outputs a second estimation result that estimates the quality of the swallowing jelly, and assesses the quality of the swallowing jelly by integrating a first assessment result determined based on the first estimation result and a second assessment result determined based on the second estimation result.

9. The system for determining quality of a swallowable jelly according to claim 5 , wherein the optical feature amount includes a color of the swallowable jelly.

10. The system for determining the quality of swallowable jelly according to claim 5 , wherein the optical feature includes a second bubble pattern in the swallowable jelly.

11. A quality assessment system for swallowing jelly described in any one of claims 5 to 10, wherein the quality assessment unit assesses the quality of the swallowing jelly by weighting and integrating a first assessment result of the quality of the swallowing jelly assessed based on the first bubble pattern and a second assessment result of the quality of the swallowing jelly assessed based on the optical feature.

12. The swallowing jelly quality assessment system of claim 5, wherein the quality assessment unit includes a multimodal model that uses the first bubble pattern and the optical features as inputs and outputs an estimation result of the quality of the swallowing jelly, and determines the quality of the swallowing jelly based on the estimation result.

13. a residue detection unit that obtains an air bubble pattern in the residue of the swallowing jelly by analyzing an ultrasound image generated while an ultrasound probe is in contact with the pharynx of a subject who has swallowed the swallowing jelly, and detects at least one of the presence or absence of the residue and the area of ​​the residue based on the air bubble pattern in the residue; The swallowing jelly quality determination system according to claim 1 , wherein the quality determination unit optimizes the residue detection unit based on the determination result of the quality of the swallowing jelly.

14. an ultrasound diagnostic device; a server connected to the ultrasound diagnostic device via a network, the ultrasonic diagnostic device includes the ultrasonic probe and the image generating unit, The swallowing jelly quality assessment system according to claim 13, wherein the server includes at least one of the image analysis unit, the quality assessment unit, and the residue detection unit.

15. an ultrasonic probe is brought into contact with the outer surface of an unopened package containing swallowable jelly containing bubbles, and ultrasonic beams are transmitted and received to generate an ultrasonic image from the received signal; obtaining a first bubble pattern in the swallowing jelly by analyzing the ultrasound image; A method for determining the quality of a swallowable jelly, the method comprising determining the quality of the swallowable jelly based on the first bubble pattern.

Images