Image information acquisition device

Abstract

The embodiment of the invention discloses an image information acquisition device. According to one specific implementation mode of the method, an imaging moving device comprises a first mechanical arm, a second mechanical arm, an ultrasonic imaging probe, an optical scanning imaging probe and an excitation probe; the image display device comprises an ultrasonic image acquisition card and an optical scanning image acquisition card; the imaging control device comprises an optical imaging device, an ultrasonic imaging device and an excitation probe control device; the driving control device and the imaging control device are both connected with the imaging moving device. According to the embodiment, the detection comprehensiveness of the to-be-detected object and the detection depth and precision of the to-be-detected object can be improved, the detection steps of the to-be-detected object are simplified, and the detection time of the to-be-detected object is shortened.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of medical devices, and in particular to an image information acquisition device. Background Art

[0002] OCT images and ultrasound images can be used to detect skin tissue. At present, the commonly used methods for detecting skin tissue can be: using only an OCT probe to detect the state of skin tissue or using only an ultrasound imaging probe to detect the state of skin tissue, or first using an OCT probe to detect the state of skin tissue and then using an ultrasound imaging probe to detect the state of skin tissue.

[0003] However, it is found in practice that when the above method is used to detect the state of skin tissue, the following technical problems often occur:

[0004] When only OCT images are used, the depth of skin tissue detection is relatively small. When only ultrasound imaging probes are used to detect the condition of skin tissue, the detection accuracy of ultrasound imaging probes is relatively low, resulting in lower accuracy or smaller range of skin tissue detection, leading to lower accuracy of skin condition detection. When OCT probes are used for detection first and then ultrasound imaging probes are used for detection, the operation steps are cumbersome, resulting in a longer time-consuming detection of skin condition.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose an image information acquisition device to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide an image information acquisition device, characterized in that the image information acquisition device includes an imaging mobile device, an imaging control device, a drive control device and an image display device, wherein the imaging mobile device includes a first mechanical arm, a second mechanical arm, an ultrasonic imaging probe, an optical scanning imaging probe and an excitation probe, wherein one end of the second mechanical arm is connected to one end of the first mechanical arm, and the other end of the second mechanical arm is connected to the ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe; the image display device includes an ultrasonic image acquisition card and an optical scanning image acquisition card; the imaging control device includes an optical imaging device, an ultrasonic imaging device and an excitation probe control device; the drive control device and the imaging control device are both connected to the imaging mobile device; in a working state, the drive control device controls The first mechanical arm and the second mechanical arm included in the imaging control device move so that the ultrasonic imaging probe and the optical scanning imaging probe included in the imaging moving device are coaxial with the excitation probe in an initial state. The imaging control device controls the optical scanning imaging probe to collect an optical scanning image of the object to be detected through the optical imaging device included therein. The imaging control device controls the ultrasonic imaging probe to collect an ultrasonic image of the object to be detected through the ultrasonic imaging device included therein. The imaging control device controls the excitation probe to collect a shear wave image of the object to be detected through the excitation probe control device included therein. The ultrasonic image, the optical scanning image and the shear wave image are respective images of the same position of the object to be detected collected simultaneously. The image display device displays the ultrasonic image, the optical scanning image and the shear wave image.

[0009] Optionally, it is characterized in that a positioning fixture is provided on the upper part of the other end of the second mechanical arm, and the ultrasonic imaging probe and the optical scanning imaging probe are both embedded in the positioning fixture.

[0010] Optionally, it is characterized in that a connecting plate is provided on the connecting base of the second robot arm.

[0011] Optionally, it is characterized in that the degrees of freedom of the first robotic arm and the second robotic arm are both 3 degrees of freedom, the imaging mobile device includes a connecting base, one end of the first robotic arm is connected to the connecting base by a threaded connection, and the second robotic arm is connected to the first robotic arm by a threaded connection.

[0012] Optionally, it is characterized in that the ultrasonic imaging probe and the optical scanning imaging probe are both coaxial with the excitation probe in an initial state.

[0013] Optionally, it is characterized in that the excitation probe is wrapped with a water bag.

[0014] Optionally, it is characterized in that the frequency of the excitation probe is 5 MHz, the frequency of the ultrasonic imaging probe is 50 MHz, the diameter of the ultrasonic imaging probe is about 5 mm, and the diameter of the excitation probe is about 5 cm.

[0015] Optionally, it is characterized in that the ultrasonic imaging probe included in the above-mentioned imaging movement device is connected to the ultrasonic imaging device included in the above-mentioned imaging control device, the optical scanning imaging probe included in the above-mentioned imaging movement device is connected to the optical imaging device included in the above-mentioned imaging control device, and the excitation probe included in the above-mentioned imaging movement device is connected to the excitation probe control device included in the above-mentioned imaging control device.

[0016] Optionally, it is characterized in that the above-mentioned ultrasonic imaging probe and the above-mentioned optical scanning imaging probe are both cylindrical in shape.

[0017] Optionally, it is characterized in that the position of the above-mentioned imaging mobile device is calibrated in the initial state through the following steps: coaxially calibrating the above-mentioned excitation probe and the above-mentioned ultrasonic imaging probe and the above-mentioned optical scanning imaging probe included in the above-mentioned positioning fixture through a hydrophone and a calibration instrument to obtain position calibration result information as the first position calibration result information; photographing the position image information corresponding to the above-mentioned ultrasonic imaging probe, the above-mentioned optical scanning imaging probe and the above-mentioned excitation probe; uploading the above-mentioned position image information to the calibration result information generation processor, and the above-mentioned calibration result information generation processor is further configured to: perform feature extraction processing on the above-mentioned position image information to obtain position image feature information; perform first pooling processing on the above-mentioned position image feature information to obtain first maximum pooling result information and first average pooling result information corresponding to the above-mentioned position image information; generate each first feature map corresponding to the above-mentioned position image information according to the above-mentioned first maximum pooling result information and the above-mentioned first average pooling result information; perform weighted processing on the above-mentioned each first feature map to obtain weighted processing result information; perform mapping processing on the above-mentioned weighted processing result information to obtain first weighted processing result information. weight coefficient matrix information; perform a second pooling process on the position image information to obtain second maximum pooling result information and second average pooling result information corresponding to the position image information; perform splicing process on the second maximum pooling result information and the second average pooling result information to obtain splicing feature map information; perform a convolution operation on the splicing feature map information to obtain a convolution feature map; generate a second weight coefficient matrix information according to the convolution feature map; perform denoising process on the position image information according to the first weight coefficient matrix information and the second weight coefficient matrix information to obtain the denoised position image information as the target position image information; determine the position calibration result information between the ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe included in the positioning fixture as the second position calibration result information according to the target position image information; in response to determining that the first position calibration result information and the second position calibration result information meet the preset calibration conditions, determine the calibration success information between the ultrasonic imaging probe and the optical scanning imaging probe and the excitation probe included in the positioning fixture.

[0018] In a second aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0019] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any one of the implementations in the first aspect or the second aspect.

[0020] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through an image information acquisition device of some embodiments of the present disclosure, the accuracy and depth of skin condition detection can be improved, and the operation steps can be simplified, thereby further shortening the time consumed for skin condition detection. The reasons for the low accuracy and depth of skin condition detection, as well as the complicated operation steps and the long time consumed for skin condition detection are: when only OCT images are used, the depth of skin tissue detection is small, and when only ultrasound imaging probes are used to detect the condition of skin tissue, the detection accuracy of ultrasound imaging probes is low, resulting in low accuracy or small range of skin tissue detection, resulting in low accuracy of skin condition detection; when using OCT probes for detection first and then ultrasound imaging probes for detection, the operation steps are complicated, resulting in a long time consumed for skin condition detection. Based on this, the image information acquisition device of some embodiments of the present disclosure is characterized in that the above-mentioned image information acquisition device includes an imaging mobile device, an imaging control device, a drive control device and an image display device, wherein the above-mentioned imaging mobile device includes a first mechanical arm, a second mechanical arm, an ultrasonic imaging probe, an optical scanning imaging probe and an excitation probe, wherein one end of the above-mentioned second mechanical arm is connected to one end of the above-mentioned first mechanical arm, and the other end of the above-mentioned second mechanical arm is connected to the above-mentioned ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe; the above-mentioned image display device includes an ultrasonic image acquisition card and an optical scanning image acquisition card; the above-mentioned imaging control device includes an optical imaging device, an ultrasonic imaging device and an excitation probe control device; the above-mentioned drive control device and the above-mentioned imaging control device are both connected to the above-mentioned imaging mobile device; in a working state, the above-mentioned drive control device controls the above-mentioned imaging The control device includes a first mechanical arm and a second mechanical arm that move so that the ultrasonic imaging probe and the optical scanning imaging probe included in the imaging moving device are coaxial with the excitation probe in the initial state. The imaging control device controls the optical scanning imaging probe to collect the optical scanning image of the object to be detected through the optical imaging device included. The imaging control device controls the ultrasonic imaging probe to collect the ultrasonic image of the object to be detected through the ultrasonic imaging device included. The imaging control device controls the excitation probe to collect the shear wave image of the object to be detected through the excitation probe control device included. The ultrasonic image, the optical scanning image and the shear wave image are images of the same position of the object to be detected collected at the same time. The image display device displays the ultrasonic image, the optical scanning image and the shear wave image. Also, because the state of the object to be detected is not detected only by the OCT probe (optical scanning imaging probe) or only by the ultrasonic imaging probe, but the ultrasonic image and the OCT image of the object to be detected are detected by using the OCT probe and the ultrasonic imaging probe at the same time, the depth and accuracy of detecting the object to be detected are improved.Also, because the OCT probe is not used to detect the state of the object to be detected first and then the ultrasonic imaging probe is used to detect the state of the object to be detected, the steps of detecting the object to be detected can be simplified, thereby shortening the time spent on detecting the object to be detected. Also, because not only the OCT image and the ultrasonic image are detected at the same time, but also the shear wave image is detected, the elasticity and viscosity of the object to be detected can be detected, thereby improving the detection range of the object to be detected and making the detection of the object to be detected more comprehensive. Thus, the comprehensiveness of the detection of the object to be detected can be improved, and the depth and accuracy of the detection of the object to be detected can be improved, and the steps of detecting the object to be detected can be further simplified, thereby shortening the time spent on detecting the object to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0022] Figure 1 is a schematic diagram of the structure of some embodiments of the image information acquisition device according to the present disclosure;

[0023] Figure 2 is a structural schematic diagram of an imaging mobile device of an image information acquisition device according to the present disclosure;

[0024] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0025] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0026] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0027] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0028] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0029] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0030] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0031] Figure 1 A schematic diagram 100 of an image information acquisition device according to the present disclosure is shown. The image information acquisition device 1 comprises an imaging movement device 14, an imaging control device 12, a drive control device 13, and an image display device.

[0032] In some embodiments, the image information acquisition device 1 may include an imaging mobile device 14, an imaging control device 12, a drive control device 13 and an image display device 11. The imaging mobile device 14 may be used to acquire an ultrasonic image, an optical scanning image and a shear wave image of the object to be detected. The imaging control device 12 may be used to control the ultrasonic imaging probe 148, the optical scanning imaging probe 144 and the excitation probe 146 included in the imaging mobile device 14 to acquire images. The drive control device 13 may be used to control the movement of the first mechanical arm 142 and the second mechanical arm 143 included in the imaging mobile device 14 so that the ultrasonic imaging probe 148 and the optical scanning imaging probe 144 are aligned with the object to be detected to acquire images. The image display device 11 may be used to display the acquired ultrasonic image, shear wave image and the acquired optical scanning image.

[0033] In some embodiments, the imaging mobile device 14 may include a first mechanical arm 142, a second mechanical arm 143, an ultrasonic imaging probe 148, an optical scanning imaging probe 144 and an excitation probe 146. Among them, one end of the second mechanical arm 143 may be connected to one end of the first mechanical arm 142. The other end of the second mechanical arm 143 may be connected to the ultrasonic imaging probe 148, and the optical scanning imaging probe 144 may be connected to the excitation probe 146. The number of joints of the first mechanical arm 142 and the second mechanical arm 143 is not limited here, and can be adjusted according to specific experimental needs. For example, the number of joints of the first mechanical arm 142 and the number of joints of the second mechanical arm 143 may both be 3. The maximum rotation angle of the first mechanical arm 142 and the second mechanical arm 143 may both be 90°. The ultrasonic imaging probe 148 may be an ultrasonic probe. The optical scanning imaging probe 144 may be an OCT probe. The frequency of the ultrasonic imaging probe 148 may be 50MHz. The diameter of the ultrasonic imaging probe 148 is about 5mm. The frequency of the excitation probe 146 may be 5 MHz. The diameter of the excitation probe 146 is about 5 cm. The wavelength of the optical scanning imaging probe 144 is 1310 nm. The diameter of the optical scanning imaging probe 144 is about 2 mm. The degrees of freedom of the first mechanical arm 142 and the second mechanical arm 143 may both be 3 degrees of freedom. The imaging mobile device 14 may include a connecting base 141. One end of the first mechanical arm 142 may be connected to the connecting base 141 by a thread. The second mechanical arm 143 may be connected to the first mechanical arm 142 by a thread.

[0034] In some embodiments, the image display device 11 may include an ultrasound image acquisition card and an optical scanning image acquisition card. The ultrasound image acquisition card may be used to acquire ultrasound images and shear wave images. The optical scanning image acquisition card may be used to acquire OCT images generated by the optical scanning imaging probe 144. The optical scanning image acquisition card may represent an OCT image acquisition card. Here, the specific types of the ultrasound image acquisition card and the optical scanning image acquisition card are not limited and may be adjusted according to experimental requirements.

[0035] In some embodiments, the imaging control device 12 may include an optical imaging device, an ultrasonic imaging device, and an excitation probe control device. The optical imaging device may be an OCT optical imaging device. The ultrasonic imaging device may be a US ultrasonic imaging device. The excitation probe control device may represent a power transmitter. The transmission power of the power transmitter may be 20W. Here, the specific types of the OCT optical imaging device, the US ultrasonic imaging device, and the power transmitter are not limited. They may be adjusted according to experimental requirements. For example, the power transmitter may be a medical ultrasonic imaging power transmitter.

[0036] In some embodiments, the driving control device 13 and the imaging control device 12 can both be connected to the imaging moving device 14. In the working state, the driving control device 13 can control the movement of the first mechanical arm 142 and the second mechanical arm 143 included in the imaging control device 12 so that the ultrasonic imaging probe 148 and the optical scanning imaging probe 144 included in the imaging moving device 14 can be coaxial with the excitation probe 146 in the initial state. The imaging control device 12 can control the optical scanning imaging probe 144 to collect the optical scanning image of the object to be detected through the included optical imaging device. The imaging control device 12 can control the ultrasonic imaging probe 148 to collect the ultrasonic image of the object to be detected through the included ultrasonic imaging device. The imaging control device 12 can control the excitation probe 146 to collect the shear wave image of the object to be detected through the included excitation probe control device. The ultrasonic image, the optical scanning image and the shear wave image are images of the same position of the object to be detected collected simultaneously. The image display device 11 can display the ultrasound image, the optical scanning image and the shear wave image. The initial state can represent the initial state of the imaging mobile device 14. The object to be detected can represent skin tissue. The optical scanning image can represent an OCT image. The shear wave image can represent an image generated by using an excitation probe 146 and shear wave elastic imaging technology. The drive control device 13 and the imaging mobile device 14 can be connected via 03.

[0037] Alternatively, if Figure 2 As shown, a positioning fixture 145 may be provided at the upper portion of the other end of the second mechanical arm 143. The ultrasonic imaging probe 148 and the optical scanning imaging probe 144 may be embedded in the positioning fixture 145. The positioning fixture 145 may allow the ultrasonic imaging probe 148 and the optical scanning imaging to be fixedly placed side by side, and may allow the detection ranges of the ultrasonic imaging probe 148 and the optical scanning imaging to have an overlapping area, thereby realizing that the image of the same part of the object to be detected is simultaneously detected in the overlapping area.

[0038] Alternatively, if Figure 2 As shown, a connecting plate 147 may be provided on the connecting base of the second mechanical arm 143. The excitation probe 146 may be embedded in the connecting plate 147. Here, the connection method between the connecting plate 147 and the second mechanical arm 143 is not limited. For example, the connection method between the connecting plate 147 and the second mechanical arm 143 may be a detachable connection. The connecting plate 147 may be used to place the object to be detected and may be used to support the object to be detected.

[0039] Alternatively, if Figure 1 or Figure 2 As shown, the degrees of freedom of the first mechanical arm 142 and the second mechanical arm 143 are both 3 degrees of freedom, and the imaging moving device 14 includes a connecting base 141, one end of the first mechanical arm 142 is connected to the connecting base 141 by a thread, and the second mechanical arm 143 is connected to the first mechanical arm 142 by a thread. Here, the connection method between one end of the first mechanical arm 142 and the connecting base 141 and the connection method between the second mechanical arm 143 and the first mechanical arm 142 are not specifically limited and can be adjusted according to specific experimental requirements.

[0040] Alternatively, if Figure 2 As shown, the ultrasound imaging probe 148 and the optical scanning imaging probe 144 can be coaxial with the excitation probe 146 in the initial state. Thus, the positioning accuracy can be improved and the subsequent calibration can be facilitated, thereby reducing the operation steps and shortening the time.

[0041] Alternatively, if Figure 2 As shown, the excitation probe 146 may be wrapped with a water bag, thereby improving the coupling effect, protecting the excitation probe 146 and reducing artifacts and interference during operation.

[0042] Alternatively, if Figure 2 As shown, the frequency of the excitation probe 146 may be 5 MHz. The frequency of the ultrasonic imaging probe 148 may be 50 MHz. The diameter of the ultrasonic imaging probe 148 may be approximately 5 mm. The diameter of the excitation probe 146 may be approximately 5 cm. The wavelength of the optical scanning imaging probe 144 may be 1310 nm. The diameter of the optical scanning imaging probe 144 may be 2 mm. It should be noted that the frequency and size of the excitation probe 146 and the ultrasonic imaging probe 148 are not limited here, and can be adjusted according to experimental requirements. The wavelength and size of the optical scanning imaging probe 144 are not limited here, and can be adjusted according to experimental requirements.

[0043] Alternatively, if Figure 1 As shown, the ultrasonic imaging probe 148 included in the imaging mobile device 14 can be connected to the ultrasonic imaging device included in the imaging control device 12. The optical scanning imaging probe 144 included in the imaging mobile device 14 can be connected to the optical imaging device included in the imaging control device 12. The excitation probe 146 included in the imaging mobile device 14 can be connected to the excitation probe control device included in the imaging control device 12. The ultrasonic imaging probe 148 included in the imaging mobile device 14 and the ultrasonic imaging device included in the imaging control device 12 can be connected via a cable 01. The optical scanning imaging probe 144 included in the imaging mobile device 14 can be connected to the optical imaging device included in the imaging control device 12 via an optical fiber 02. The excitation probe 146 included in the imaging mobile device 14 can be connected to the excitation probe control device included in the imaging control device 12 via a cable 04.

[0044] Alternatively, if Figure 1 As shown, the ultrasonic imaging probe 148 and the optical scanning imaging probe 144 may both be cylindrical in shape.

[0045] In the process of adopting technical solutions to solve the above-mentioned technical problem one, the following technical problem two is often accompanied: the complexity of using a calibration instrument to calibrate the ultrasonic imaging probe and optical scanning imaging probe included in the positioning fixture with the above-mentioned excitation probe calibration method is relatively large, and the technical requirements for the operator are relatively high. The steps are relatively cumbersome and are greatly affected by the environment, resulting in low calibration accuracy and a long calibration time. In response to the above-mentioned technical problem two, the conventional solution is generally: use a hydrophone to calibrate the ultrasonic imaging probe and optical scanning imaging probe included in the positioning fixture with the above-mentioned excitation probe calibration method. The inventor took into account the shortcomings of using a hydrophone to calibrate the ultrasonic imaging probe and optical scanning imaging probe included in the positioning fixture with the above-mentioned excitation probe calibration method. The inventor's unit has experience in calibrating the ultrasonic imaging probe and optical scanning imaging probe included in the positioning fixture with the above-mentioned excitation probe calibration method, and combined with the situation of the cooperative unit commissioned for development, we decided to adopt the following solution:

[0046] Optionally, the imaging mobile device calibrates its position in the initial state by the following steps:

[0047] The first step is to capture position image information corresponding to the ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe, wherein the position image information may represent an image of the positions of the ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe included in the positioning fixture in the initial state of the capture.

[0048] The second step is to upload the position image information to the calibration result information generation processor, and the calibration result information generation processor is further configured to:

[0049] The third step is to perform feature extraction processing on the position image information to obtain position image feature information. The calibration result information generation processor may represent a server for generating calibration success information between the positioning fixture and the excitation probe. The image feature information may represent features of an image. In practice, the calibration result information generation processor may perform feature extraction processing on the image information through a convolutional neural network to obtain image feature information.

[0050] The fourth step is to perform a first pooling process on the above-mentioned position image feature information to obtain the first maximum pooling result information and the first average pooling result information corresponding to the above-mentioned position image information. Among them, the above-mentioned first maximum pooling result information can represent the feature map of the above-mentioned position image feature information after the maximum pooling layer performs pooling processing in the spatial dimension. The above-mentioned first average pooling result information can represent the feature map of the above-mentioned position image feature information after the average pooling layer performs pooling processing in the spatial dimension. In practice, the above-mentioned calibration result information generation processor can input the above-mentioned position image feature information into the maximum pooling layer to obtain the first maximum pooling result information. Then, the above-mentioned position image feature information is input into the average pooling layer to obtain the first average pooling result information.

[0051] The fifth step is to generate each first feature map corresponding to the position image information according to the first maximum pooling result information and the first average pooling result information. Among them, each first feature map may include a 1×1×C feature map corresponding to the first maximum pooling result information and a 1×1×C feature map corresponding to the first average pooling result information. In practice, the calibration result information generation processor may input the first average pooling result information and the first maximum pooling result information into a shared multilayer perceptron, respectively, to obtain each first feature map.

[0052] The sixth step is to perform weighted processing on each of the first feature maps to obtain weighted processing result information. The weighted processing result information can represent the feature map after weighting each of the first feature maps. In practice, the calibration result information generation processor can perform weighted operations on each of the first feature maps to obtain the weighted feature map.

[0053] In the seventh step, the weighted processing result information is mapped to obtain the first weight coefficient matrix information. The first weight coefficient matrix information can represent the channel attention weight matrix. In practice, the calibration result information generation processor can map the weighted processing result information through an activation function to obtain the first weight coefficient matrix information. For example, the activation function can be a Sigmoid function.

[0054] In the eighth step, a second pooling process is performed on the above-mentioned position image information to obtain the second maximum pooling result information and the second average pooling result information corresponding to the above-mentioned position image information. Among them, the second maximum pooling result information can represent the feature map of the above-mentioned position image information after the maximum pooling layer performs pooling processing in the channel dimension. The second average pooling result information can represent the feature map after the average pooling layer performs pooling processing in the channel dimension. In practice, the above-mentioned calibration result information generation processor can input the above-mentioned position image feature information into the maximum pooling layer to obtain the second maximum pooling result information. Then, the above-mentioned position image feature information is input into the average pooling layer to obtain the second average pooling result information.

[0055] In the ninth step, the second maximum pooling result information and the second average pooling result information are spliced ​​to obtain spliced ​​feature map information. The spliced ​​feature map information may represent a feature map obtained by splicing the second maximum pooling result information and the second average pooling result information in the channel dimension. The size of the spliced ​​feature map information may be H×W×1.

[0056] In the tenth step, a convolution operation is performed on the above-mentioned spliced ​​feature map information to obtain a convolution feature map. The above-mentioned convolution feature map can represent the spliced ​​feature map information after the convolution process. In practice, the above-mentioned calibration result information generation processor can perform a 7×7 convolution operation on the above-mentioned spliced ​​feature map information to obtain a convolution feature map.

[0057] In the eleventh step, second weight coefficient matrix information is generated according to the convolution feature map. The second weight coefficient matrix information can represent the spatial attention weight matrix coefficient. In practice, the calibration result information generation processor can process the convolution feature map through an activation function to obtain the spatial attention weight matrix coefficient.

[0058] Step 12: De-noise the position image information according to the first weight coefficient matrix information and the second weight coefficient matrix information, and obtain the position image information after de-noising as the target position image information. In practice, the calibration result information generation processor can input the first weight coefficient matrix information and the second weight coefficient matrix information into an image generator, and obtain the image information after de-noising as the target image information. The image generator can be CIG.

[0059] Step 13: Determine the position calibration result information between the ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe included in the positioning fixture according to the target position image information. In practice, the execution subject may detect the target position image information through a position detection algorithm to obtain the position calibration result information. For example, the position detection algorithm may be an image alignment algorithm. The position calibration result information may indicate a successful calibration or a failed calibration.

[0060] Step 14: In response to determining that the position calibration result information satisfies a preset calibration condition, determine the calibration success information between the ultrasonic imaging probe and the optical scanning imaging probe included in the positioning fixture and the excitation probe. The preset calibration condition may be the position calibration result information indicating the calibration success.

[0061] The above technical solution, as an inventive point of an embodiment of the present disclosure, solves the second technical problem: the factors of "low calibration accuracy and long calibration time" are often as follows: the complexity of using a calibration instrument to calibrate the ultrasonic imaging probe and optical scanning imaging probe included in the positioning fixture with the above excitation probe is relatively large, and the technical requirements for the operator are relatively high, the steps are relatively cumbersome and are greatly affected by the environment. If the above factors are solved, the effect of improving the low calibration accuracy and shortening the calibration time can be achieved. In order to achieve this effect, when the ultrasonic imaging probe and optical scanning imaging probe included in the positioning fixture are calibrated with the above excitation probe, the present disclosure first captures an image including the positioning fixture and the excitation probe, and then uploads the captured image to the calibration result information generation processor. The calibration result information generation processor can perform denoising on the captured image, and then determine whether the ultrasonic imaging probe and optical scanning imaging probe included in the positioning fixture are calibrated with the above excitation probe according to the processed image, and the calibration is performed according to the processed image, which further improves the accuracy of the calibration, and the calibration result can be obtained by only uploading the image to the calibration result information generation processor, and the operation steps are simple, thereby shortening the calibration time.

[0062] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through an image information acquisition device of some embodiments of the present disclosure, the accuracy and depth of skin condition detection can be improved, and the operation steps can be simplified, thereby further shortening the time consumed for skin condition detection. The reasons for the low accuracy and depth of skin condition detection, as well as the complicated operation steps and the long time consumed for skin condition detection are: when only OCT images are used, the depth of skin tissue detection is small, and when only ultrasound imaging probes are used to detect the condition of skin tissue, the detection accuracy of ultrasound imaging probes is low, resulting in low accuracy or small range of skin tissue detection, resulting in low accuracy of skin condition detection; when using OCT probes for detection first and then ultrasound imaging probes for detection, the operation steps are complicated, resulting in a long time consumed for skin condition detection. Based on this, the image information acquisition device of some embodiments of the present disclosure is characterized in that the above-mentioned image information acquisition device includes an imaging mobile device, an imaging control device, a drive control device and an image display device, wherein the above-mentioned imaging mobile device includes a first mechanical arm, a second mechanical arm, an ultrasonic imaging probe, an optical scanning imaging probe and an excitation probe, wherein one end of the above-mentioned second mechanical arm is connected to one end of the above-mentioned first mechanical arm, and the other end of the above-mentioned second mechanical arm is connected to the above-mentioned ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe; the above-mentioned image display device includes an ultrasonic image acquisition card and an optical scanning image acquisition card; the above-mentioned imaging control device includes an optical imaging device, an ultrasonic imaging device and an excitation probe control device; the above-mentioned drive control device and the above-mentioned imaging control device are both connected to the above-mentioned imaging mobile device; in a working state, the above-mentioned drive control device controls the above-mentioned imaging The control device includes a first mechanical arm and a second mechanical arm that move so that the ultrasonic imaging probe and the optical scanning imaging probe included in the imaging moving device are coaxial with the excitation probe in the initial state. The imaging control device controls the optical scanning imaging probe to collect the optical scanning image of the object to be detected through the optical imaging device included. The imaging control device controls the ultrasonic imaging probe to collect the ultrasonic image of the object to be detected through the ultrasonic imaging device included. The imaging control device controls the excitation probe to collect the shear wave image of the object to be detected through the excitation probe control device included. The ultrasonic image, the optical scanning image and the shear wave image are images of the same position of the object to be detected collected at the same time. The image display device displays the ultrasonic image, the optical scanning image and the shear wave image. Also, because the state of the object to be detected is not detected only by the OCT probe (optical scanning imaging probe) or only by the ultrasonic imaging probe, but the ultrasonic image and the OCT image of the object to be detected are detected by using the OCT probe and the ultrasonic imaging probe at the same time, the depth and accuracy of detecting the object to be detected are improved.Also, because the OCT probe is not used to detect the state of the object to be detected first and then the ultrasonic imaging probe is used to detect the state of the object to be detected, the steps of detecting the object to be detected can be simplified, thereby shortening the time spent on detecting the object to be detected. Also, because not only the OCT image and the ultrasonic image are detected at the same time, but also the shear wave image is detected, the elasticity and viscosity of the object to be detected can be detected, thereby improving the detection range of the object to be detected and making the detection of the object to be detected more comprehensive. Thus, the comprehensiveness of the detection of the object to be detected can be improved, and the depth and accuracy of the detection of the object to be detected can be improved, and the steps of detecting the object to be detected can be further simplified, thereby shortening the time spent on detecting the object to be detected.

[0063] Reference below Figure 3 , which shows a schematic structural diagram of an electronic device 300 (eg, a computing device) suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0064] like Figure 3 As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0065] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0066] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0067] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0068] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0069] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: captures the position image information corresponding to the ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe; uploads the position image information to the calibration result information generation processor, and the calibration result information generation processor is further configured to: perform feature extraction processing on the position image information to obtain position image feature information; perform first pooling processing on the position image feature information to obtain first maximum pooling result information and first average pooling result information corresponding to the position image information; generate each first feature map corresponding to the position image information according to the first maximum pooling result information and the first average pooling result information; perform weighted processing on each first feature map to obtain weighted processing result information; perform mapping processing on the weighted processing result information to obtain first weight coefficient matrix information; perform weighted processing on the position image information; perform weighted ... Perform a second pooling process to obtain second maximum pooling result information and second average pooling result information corresponding to the above-mentioned position image information; perform splicing processing on the above-mentioned second maximum pooling result information and the above-mentioned second average pooling result information to obtain splicing feature map information; perform a convolution operation on the above-mentioned splicing feature map information to obtain a convolution feature map; generate second weight coefficient matrix information according to the above-mentioned convolution feature map; perform denoising processing on the above-mentioned position image information according to the above-mentioned first weight coefficient matrix information and the above-mentioned second weight coefficient matrix information to obtain the denoised position image information as the target position image information; determine the position calibration result information between the above-mentioned ultrasonic imaging probe, the above-mentioned optical scanning imaging probe and the above-mentioned excitation probe included in the above-mentioned positioning fixture according to the above-mentioned target position image information; in response to determining that the above-mentioned position calibration result information meets the preset calibration condition, determine the calibration success information between the above-mentioned ultrasonic imaging probe and the above-mentioned optical scanning imaging probe and the above-mentioned excitation probe included in the above-mentioned positioning fixture.

[0070] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0071] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0072] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0073] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. An image information acquisition device, characterized in that: The image information acquisition device includes an imaging moving device, an imaging control device, a driving control device and an image display device, wherein: The imaging mobile device comprises a first mechanical arm, a second mechanical arm, an ultrasonic imaging probe, an optical scanning imaging probe and an excitation probe, wherein one end of the second mechanical arm is connected to one end of the first mechanical arm, and the other end of the second mechanical arm is connected to the ultrasonic imaging probe, the optical scanning imaging probe and the excitation probe; The image display device includes an ultrasonic image acquisition card and an optical scanning image acquisition card; The imaging control device includes an optical imaging device, an ultrasonic imaging device and an excitation probe control device; The driving control device and the imaging control device are both connected to the imaging moving device; In the working state, the driving control device controls the movement of the first mechanical arm and the second mechanical arm included in the imaging control device so that the ultrasonic imaging probe and the optical scanning imaging probe included in the imaging moving device are coaxial with the excitation probe in the initial state, the imaging control device controls the optical scanning imaging probe to collect the optical scanning image of the object to be detected through the included optical imaging device, the imaging control device controls the ultrasonic imaging probe to collect the ultrasonic image of the object to be detected through the included ultrasonic imaging device, and the imaging control device controls the excitation probe to collect the shear wave image of the object to be detected through the included excitation probe control device, the ultrasonic image, the optical scanning image and the shear wave image are respective images of the same position of the object to be detected collected simultaneously, and the image display device displays the ultrasonic image, the optical scanning image and the shear wave image.

2. The image information acquisition device according to claim 1, characterized in that: A positioning fixture is provided on the upper part of the other end of the second mechanical arm, and the ultrasonic imaging probe and the optical scanning imaging probe are both embedded in the positioning fixture.

3. The image information acquisition device according to claim 1, characterized in that: A connecting plate is provided on the connecting base of the second mechanical arm, and the excitation probe is embedded in the connecting plate.

4. The image information acquisition device according to claim 1, characterized in that: The first mechanical arm and the second mechanical arm both have three degrees of freedom. The imaging mobile device includes a connecting base. One end of the first mechanical arm is connected to the connecting base via a thread, and the second mechanical arm is connected to the first mechanical arm via a thread.

5. The image information acquisition device according to claim 1, characterized in that: The ultrasonic imaging probe and the optical scanning imaging probe are both coaxial with the excitation probe in an initial state.

6. The image information acquisition device according to claim 1, characterized in that: The excitation probe is wrapped with a water bag.

7. The image information acquisition device according to claim 1, characterized in that: The frequency of the excitation probe is 5 MHz, the frequency of the ultrasonic imaging probe is 50 MHz, the diameter of the ultrasonic imaging probe is about 5 mm, and the diameter of the excitation probe is about 5 cm.

8. The image information acquisition device according to claim 1, characterized in that: The ultrasonic imaging probe included in the imaging mobile device is connected to the ultrasonic imaging device included in the imaging control device, the optical scanning imaging probe included in the imaging mobile device is connected to the optical imaging device included in the imaging control device, and the excitation probe included in the imaging mobile device is connected to the excitation probe control device included in the imaging control device.

9. The image information acquisition device according to claim 1, characterized in that: The ultrasonic imaging probe and the optical scanning imaging probe are both cylindrical in shape.

Images