Non-contact ultrasonic probe detection method and system based on deep learning

Abstract

The present invention discloses a non-contact ultrasonic probe detection method and system based on deep learning. The plane image and the three-dimensional distance measurement matrix of the same position at different time points are matched, and the temporal correlation between the two is established. By constructing a sphere in the three-dimensional distance measurement matrix to predict the collision boundary, a second three-dimensional matrix is ​​obtained, which can use the size to judge the movable range of the rotating distance measurement device. And the correlation features are obtained between the plane image, the three-dimensional distance measurement matrix and the second three-dimensional matrix. The correlation features can not only find the correlation between the displayed image and the detected distance features, but also contain the implicit movement route information according to the changes in the positions of multiple time points.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a non-contact ultrasonic probe detection method and system based on deep learning. Background Art

[0002] The use of non-contact ultrasound probes is becoming more and more common with the development of detection, which can prevent the probe from contacting the prosthesis, soft tissue and bone tissue in the surgical area during use.

[0003] However, currently, the only way to prevent the probe from coming into contact with other surfaces is by manual operation. This method cannot prevent contact events from occurring, and therefore a distance sensing device is usually added to the handheld ultrasound probe device.

[0004] However, how to use the distance sensing device to determine the location of the prosthesis, soft tissue and bone tissue in the surgical area, and use deep learning to predict whether the probe will contact the prosthesis, soft tissue and bone tissue at the next time point based on the movement direction and internal structure of the ultrasonic probe, so as to ensure that an alarm is issued when the handheld ultrasonic probe is about to contact the surface, reminding the operator that the ultrasonic probe is about to come into contact with the surface of other parts. Summary of the invention

[0005] The purpose of the present invention is to provide a non-contact ultrasonic probe detection method and system based on deep learning to solve the above-mentioned problems existing in the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a non-contact ultrasonic probe detection method based on deep learning, comprising:

[0007] Obtaining plane images and three-dimensional distance measurement matrices at multiple time points; the three-dimensional distance measurement matrix represents the distances to surrounding structures detected with the rotating distance measurement device as the center; the plane image represents an image of the movement path of the rotating distance measurement device taken by a micro camera on the rotating distance measurement device;

[0008] Establishing an association relationship based on the planar images and the ranging three-dimensional matrix at the multiple time points to obtain a first association set;

[0009] The first association set includes an associated planar image at one time point and a three-dimensional matrix of range measurement at another time point;

[0010] Based on the ranging three-dimensional matrix, the collision boundary is predicted to obtain a second three-dimensional matrix;

[0011] Based on the plane image, the three-dimensional distance measurement matrix and the second three-dimensional matrix, the correlation features are obtained; and multiple correlation features are obtained corresponding to multiple time points;

[0012] According to the time points from early to late, the related features of multiple time points are input into the temporal convolutional network in sequence to predict the position of the next time point and determine whether contact will occur;

[0013] If contact is likely to occur, a warning signal is sent.

[0014] Optionally, predicting the collision boundary based on the ranging three-dimensional matrix to obtain the second three-dimensional matrix includes:

[0015] Taking the midpoint of the three-dimensional distance measurement matrix as the origin, constructing a three-dimensional coordinate axis;

[0016] Project the position in the distance measurement three-dimensional matrix onto the position in the three-dimensional coordinate axis; mark the value in the distance measurement three-dimensional matrix at the corresponding position on the three-dimensional coordinate axis;

[0017] In the three-dimensional coordinate axis, based on the three-dimensional distance measurement matrix, a minimum obstacle distance is obtained;

[0018] Take the center point of the three-dimensional distance measurement matrix as the center of the sphere and the radius of the sphere with the minimum obstacle distance to obtain the collision sphere;

[0019] Take the largest inscribed cube in the collision sphere as the first cube;

[0020] A second three-dimensional matrix is ​​constructed using the values ​​contained in the first cube; the values ​​in the second three-dimensional matrix are 0; the number of rows, columns and layers of the second three-dimensional matrix represent the positions that can collide.

[0021] Optionally, obtaining the associated features based on the plane image, the ranging three-dimensional matrix and the second three-dimensional matrix includes:

[0022] The distance measurement three-dimensional matrix is ​​cut by the surface of the number of rows and layers corresponding to the center point to obtain a first cut distance matrix;

[0023] Based on the plane image, multiple first features are obtained through a plane convolution network;

[0024] Based on the first cutting distance matrix, a plurality of second features are obtained through a first ranging convolutional network;

[0025] One first feature corresponds to one second feature;

[0026] Based on the second three-dimensional matrix, the first feature and the corresponding second feature, an association feature is obtained through an association network.

[0027] Optionally, obtaining the associated feature through an associated network based on the second three-dimensional matrix, the first feature and the corresponding second feature includes:

[0028] The associated network includes a first associated network and a second associated network;

[0029] Subtract the first feature from the second feature to obtain a first associated feature; obtain multiple first associated features corresponding to multiple first features;

[0030] constructing the plurality of first correlation features into a first correlation matrix according to the columns of the first cutting distance matrix corresponding to the plurality of first correlation features;

[0031] Inputting the first association matrix into the first association network to obtain a second association feature;

[0032] Based on the second three-dimensional matrix, obtaining a third association feature through a second association network;

[0033] The second association feature and the third association feature are superimposed to obtain an association feature.

[0034] Optionally, obtaining a third association feature based on the second three-dimensional matrix through a second association network includes:

[0035] Acquire a plurality of second three-dimensional matrices in a time period between the first time point and the second time point;

[0036] The plurality of second three-dimensional matrices are input into the second association network to obtain third association features.

[0037] Optionally, the method of obtaining a plurality of first features based on the planar image through a planar convolutional network includes:

[0038] Taking the quotient of the length of the plane image divided by the number of columns of the first cutting distance matrix as the plane difference value;

[0039] The length of the plane image is divided sequentially according to the plane difference value to obtain a plurality of plane column images;

[0040] The planar convolution network includes a three-dimensional convolution kernel of m*2*3; m represents the quotient of the length of the planar image divided by the number of columns of the first cutting distance matrix; the 3 in the three-dimensional convolution kernel of the planar convolution network represents 3 RGB values; the 2 in the three-dimensional convolution kernel of the planar convolution network represents 2 rows in the planar column image;

[0041] According to the direction of the column corresponding to the planar column image, with a step size of 1, convolve the three-dimensional convolution kernel corresponding to the planar convolution network on the planar column image to obtain a first feature;

[0042] A plurality of first features are obtained corresponding to the plurality of planar array images.

[0043] Optionally, the obtaining of multiple second features based on the first cutting distance matrix through a first ranging convolutional network includes:

[0044] The first cutting distance matrix is ​​divided into columns to obtain a plurality of first cutting column matrices; one first cutting column matrix corresponds to one column of the first cutting distance matrix;

[0045] The first distance measurement convolution network includes a 2*n two-dimensional convolution kernel; n represents the number of layers of the first cutting distance matrix; 2 in the two-dimensional convolution kernel of the first distance measurement convolution network represents 2 rows in the first cutting column matrix;

[0046] According to the direction of the column corresponding to the first cutting column matrix, the two-dimensional convolution kernel corresponding to the first ranging convolution is convolved on the first cutting column matrix with a step size of 1 to obtain a second feature;

[0047] A plurality of first cutting column matrices correspondingly obtain a plurality of second features.

[0048] Optionally, establishing an association relationship based on the planar images at the multiple time points and the ranging three-dimensional matrix to obtain a first association set includes:

[0049] Obtaining a first time point and a second time point; the first time point is a time point corresponding to a plane image; the second time point is a time point when the rotating distance measuring device reaches a position corresponding to the plane image at the first time point;

[0050] An association relationship is established between the distance measurement three-dimensional matrix corresponding to the second time point and the plane image at the first time point.

[0051] Optionally, obtaining the minimum obstacle distance in the three-dimensional coordinate axis based on the three-dimensional distance measurement matrix includes:

[0052] Taking the position of 1 in the ranging three-dimensional matrix as the obstacle position;

[0053] Detect the distance between the origin and the obstacle position in the three-dimensional coordinate axis to obtain multiple obstacle distances;

[0054] An obstacle distance that is smaller than other obstacle distances among the plurality of obstacle distances is taken as the minimum obstacle distance.

[0055] In a second aspect, an embodiment of the present invention provides a non-contact ultrasonic probe detection system based on deep learning, comprising:

[0056] An acquisition module is used to obtain a plane image and a three-dimensional distance measurement matrix at multiple time points; the three-dimensional distance measurement matrix represents the distance to the surrounding structures detected with the rotating distance measurement device as the center; the plane image represents the image of the movement path of the rotating distance measurement device taken by the micro camera on the rotating distance measurement device;

[0057] An association module is used to establish an association relationship based on the plane images and the ranging three-dimensional matrix at the multiple time points to obtain a first association set; the first association set includes the plane image at one associated time point and the ranging three-dimensional matrix at another associated time point;

[0058] A boundary module is used to predict the collision boundary based on the ranging three-dimensional matrix to obtain a second three-dimensional matrix;

[0059] The correlation feature module obtains the correlation features based on the plane image, the three-dimensional matrix of distance measurement and the second three-dimensional matrix; multiple correlation features are obtained corresponding to multiple time points;

[0060] The prediction module is used to input the associated features of multiple time points into the temporal convolutional network in order from early to late, predict the position of the next time point, and determine whether contact will occur;

[0061] The reminder module is used to send a reminder signal if contact is about to take place.

[0062] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects:

[0063] An embodiment of the present invention also provides a non-contact ultrasonic probe detection method and system based on deep learning.

[0064] In the present invention, because the shooting range of the camera device cannot directly obtain the plane image of the current rotating distance measuring device point, it is necessary to match the plane image and the distance measuring three-dimensional matrix of the same position at different time points, and establish the temporal correlation between the two. By constructing a sphere in the distance measuring three-dimensional matrix to predict the collision boundary, a second three-dimensional matrix that can use the size to judge the movable range of the rotating distance measuring device is obtained. And the correlation features are obtained between the plane image, the distance measuring three-dimensional matrix and the second three-dimensional matrix. The correlation features can not only find the correlation between the displayed image and the detected distance feature, but also contain the information of the implicit movement route according to the change of the position at multiple time points. Through the temporal and characteristic correlation and the movement direction of the ultrasonic probe, it is possible to achieve a more accurate prediction of whether the probe will contact the prosthesis, soft tissue and bone tissue at the next time point, ensure that the handheld ultrasonic probe is about to contact the surface and send an alarm, and remind the operator that the ultrasonic probe is about to contact the surface of other parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flow chart of a non-contact ultrasonic probe detection method based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0067] Example 1

[0068] like Figure 1 As shown, an embodiment of the present invention provides a non-contact ultrasonic probe detection method based on deep learning, the method comprising:

[0069] S101: Obtaining plane images and a three-dimensional ranging matrix at multiple time points; the three-dimensional ranging matrix represents the distance to the surrounding structures detected with the rotating ranging device as the center; the plane image represents the image of the movement path of the rotating ranging device taken by the micro camera on the rotating ranging device.

[0070] In this embodiment, the rotating distance measuring device is provided with a rotatable distance sensing device and a fixed camera device at the front of the rotating distance measuring device. The distance sensing device is used for distance measurement, and the camera device is a miniature camera for photographing a plane image.

[0071] In this embodiment, the distance sensing device detects the location where the obstacle is encountered.

[0072] The three-dimensional distance measurement matrix sets the position where there is an obstacle to 1, and sets the position where there is no obstacle to 0.

[0073] The distance measurement three-dimensional matrix is ​​a three-dimensional matrix of fixed size. In this embodiment, the distance measurement three-dimensional matrix is ​​512*512*512. The position (256, 256, 256) is the center point of the three-dimensional matrix, that is, the position of the rotating distance measurement device.

[0074] S102: Establishing an association relationship based on the planar images at the multiple time points and the ranging three-dimensional matrix to obtain a first association set.

[0075] The first association set includes an associated planar image at one time point and a range-measured three-dimensional matrix at another time point.

[0076] S103: predicting the collision boundary based on the ranging three-dimensional matrix to obtain a second three-dimensional matrix;

[0077] S104: obtaining correlation features based on the plane image, the three-dimensional distance measurement matrix, and the second three-dimensional matrix; obtaining multiple correlation features corresponding to multiple time points;

[0078] S105: Input the associated features of multiple time points into the temporal convolutional network in sequence from early to late, predict the position of the next time point, and determine whether contact will occur.

[0079] Among them, the temporal convolutional network (TCN). The associated features of multiple time points are sequentially input into the temporal convolutional network, and the position of the next time point is predicted as the position of the next time point. The distance sensing device is used to detect the distance of the collision to the surrounding structure in the direction of the position of the next time point as the obstacle distance at the next time point. If the obstacle distance at the next time point is less than or equal to the distance between the position at the next time point and the rotating distance measuring device, it indicates that the surgical area is contacted. If the obstacle distance at the next time point is greater than the distance between the position at the next time point and the rotating distance measuring device, it indicates that the surgical area is not contacted.

[0080] S106: If contact will occur, send a reminder signal.

[0081] Optionally, predicting the collision boundary based on the ranging three-dimensional matrix to obtain the second three-dimensional matrix includes:

[0082] A three-dimensional coordinate axis is constructed with the midpoint of the three-dimensional distance measurement matrix as the origin.

[0083] In this embodiment, the horizontal axis of the three-dimensional coordinate axis corresponds to the rows of the ranging three-dimensional matrix, the vertical axis of the three-dimensional coordinate axis corresponds to the columns of the ranging three-dimensional matrix, and the vertical axis of the three-dimensional coordinate axis corresponds to the height of the ranging three-dimensional matrix.

[0084] Project the positions in the ranging three-dimensional matrix onto the positions in the three-dimensional coordinate axis; and mark the values ​​in the ranging three-dimensional matrix at the corresponding positions on the three-dimensional coordinate axis.

[0085] In this embodiment, the position of the first row, the first column, and the first layer in the ranging three-dimensional matrix corresponds to (1, 1, 1) in the three-dimensional coordinate axis, and the position of the second row, the second column, and the second layer in the ranging three-dimensional matrix corresponds to (2, 2, 2) in the three-dimensional coordinate axis.

[0086] In the three-dimensional coordinate axis, the minimum obstacle distance is obtained based on the distance measurement three-dimensional matrix.

[0087] Take the center point of the three-dimensional distance measurement matrix as the center of the sphere and the radius of the sphere with the minimum obstacle distance to obtain the collision sphere;

[0088] Take the largest inscribed cube in the collision sphere as the first cube;

[0089] A second three-dimensional matrix is ​​constructed using the values ​​contained in the first cube; the values ​​in the second three-dimensional matrix are 0; the number of rows, columns and layers of the second three-dimensional matrix represent the positions that can collide.

[0090] Optionally, obtaining the first correlation feature based on the plane image, the ranging three-dimensional matrix and the second three-dimensional matrix includes:

[0091] The distance measurement three-dimensional matrix is ​​cut with the surface of the number of rows and layers corresponding to the center point to obtain a first cut distance matrix.

[0092] Wherein, the first cutting distance matrix is ​​a two-dimensional matrix;

[0093] Based on the plane image, multiple first features are obtained through a plane convolution network;

[0094] Based on the first cutting distance matrix, a plurality of second features are obtained through a first ranging convolutional network;

[0095] One first feature corresponds to one second feature.

[0096] Wherein, the first feature and the second feature have the same size.

[0097] Based on the second three-dimensional matrix, the first feature and the corresponding second feature, an association feature is obtained through an association network.

[0098] Optionally, obtaining the associated feature through an associated network based on the second three-dimensional matrix, the first feature and the corresponding second feature includes:

[0099] The associated network includes a first associated network and a second associated network;

[0100] Subtract the first feature from the second feature to obtain a first associated feature; obtain multiple first associated features corresponding to multiple first features;

[0101] constructing the plurality of first correlation features into a first correlation matrix according to the columns of the first cutting distance matrix corresponding to the plurality of first correlation features;

[0102] The first association matrix is ​​input into the first association network to obtain a second association feature.

[0103] Wherein, the first association network is a pyramid structure.

[0104] Based on the second three-dimensional matrix, obtaining a third association feature through a second association network;

[0105] The second association feature and the third association feature are superimposed to obtain an association feature.

[0106] Optionally, obtaining a third association feature based on the second three-dimensional matrix through a second association network includes:

[0107] Acquire a plurality of second three-dimensional matrices in a time period between the first time point and the second time point;

[0108] The plurality of second three-dimensional matrices are input into the second association network to obtain third association features.

[0109] In this embodiment, the second association network is a deconvolutional convolutional neural network (CNN).

[0110] Optionally, the method of obtaining a plurality of first features based on the planar image through a planar convolutional network includes:

[0111] The quotient of the length of the plane image divided by the number of columns of the first cutting distance matrix is ​​taken as the plane difference value.

[0112] In this embodiment, the length of the plane image is 512, the number of columns of the first cutting distance matrix is ​​256, and the plane difference value obtained is 2.

[0113] The length of the plane image is divided in sequence according to the plane difference value to obtain multiple plane column images.

[0114] In this embodiment, the width and number of layers of the planar array image are 512, and the length is 2.

[0115] In this embodiment, the number of the planar array images is 256.

[0116] The planar convolution network includes a three-dimensional convolution kernel of m*2*3; m represents the quotient of the length of the planar image divided by the number of columns of the first cutting distance matrix; the 3 in the three-dimensional convolution kernel of the planar convolution network represents 3 RGB values; the 2 in the three-dimensional convolution kernel of the planar convolution network represents 2 rows in the planar column image.

[0117] Wherein, m is a positive integer.

[0118] Wherein, the planar convolutional network is a three-dimensional convolutional neural network (3D Convolutional Neural Networks, CNN).

[0119] According to the direction of the column corresponding to the planar column image, with a step size of 1, convolve the three-dimensional convolution kernel corresponding to the planar convolution network on the planar column image to obtain a first feature;

[0120] A plurality of first features are obtained corresponding to the plurality of planar array images.

[0121] Optionally, the obtaining of multiple second features based on the first cutting distance matrix through a first ranging convolutional network includes:

[0122] The first cutting distance matrix is ​​divided into columns to obtain a plurality of first cutting column matrices; one first cutting column matrix corresponds to one column of the first cutting distance matrix.

[0123] The first ranging convolution network includes a 2*n two-dimensional convolution kernel; n represents the number of layers of the first cutting distance matrix; and 2 in the two-dimensional convolution kernel of the first ranging convolution network represents 2 rows in the first cutting column matrix.

[0124] Among them, the first ranging convolutional network is a convolutional neural network (Convolutional Neural Networks, CNN).

[0125] According to the direction of the column corresponding to the first cutting column matrix, the two-dimensional convolution kernel corresponding to the first ranging convolution is convolved on the first cutting column matrix with a step size of 1 to obtain the second feature.

[0126] A plurality of first cutting column matrices correspondingly obtain a plurality of second features.

[0127] Through the above method, due to the special convolution direction, the two-dimensional convolution kernel is equivalent to the operation of the three-dimensional convolution kernel.

[0128] Optionally, establishing an association relationship based on the planar images at the multiple time points and the ranging three-dimensional matrix to obtain a first association set includes:

[0129] A first time point and a second time point are obtained; the first time point is a time point corresponding to a plane image; the second time point is a time point when the rotating ranging device reaches a position corresponding to the plane image at the first time point.

[0130] Through the above method, since the shooting range of the camera device cannot directly obtain the image of the plane of the current rotating distance measuring device point, matching is required.

[0131] An association relationship is established between the distance measurement three-dimensional matrix corresponding to the second time point and the plane image at the first time point.

[0132] Optionally, obtaining the minimum obstacle distance in the three-dimensional coordinate axis based on the three-dimensional distance measurement matrix includes:

[0133] The position of 1 in the ranging three-dimensional matrix is ​​taken as the obstacle position.

[0134] The distance between the origin and the obstacle position is detected in the three-dimensional coordinate axis to obtain multiple obstacle distances.

[0135] An obstacle distance that is smaller than other obstacle distances among the plurality of obstacle distances is taken as the minimum obstacle distance.

[0136] Example 2

[0137] Based on the above-mentioned non-contact ultrasonic probe detection method based on deep learning, an embodiment of the present invention also provides a non-contact ultrasonic probe detection system based on deep learning, and the system includes an acquisition module, an association module, a boundary module, an association feature module and a prediction module.

[0138] An acquisition module is used to obtain a plane image and a three-dimensional distance measurement matrix at multiple time points; the three-dimensional distance measurement matrix represents the distance to the surrounding structures detected with the rotating distance measurement device as the center; the plane image represents the image of the movement path of the rotating distance measurement device taken by the micro camera on the rotating distance measurement device;

[0139] An association module is used to establish an association relationship based on the plane images and the ranging three-dimensional matrix at the multiple time points to obtain a first association set; the first association set includes the plane image at one associated time point and the ranging three-dimensional matrix at another associated time point;

[0140] A boundary module is used to predict the collision boundary based on the ranging three-dimensional matrix to obtain a second three-dimensional matrix;

[0141] The correlation feature module obtains the correlation features based on the plane image, the three-dimensional matrix of distance measurement and the second three-dimensional matrix; multiple correlation features are obtained corresponding to multiple time points;

[0142] The prediction module is used to input the associated features of multiple time points into the temporal convolutional network in order from early to late, predict the position of the next time point, and determine whether contact will occur;

[0143] The reminder module is used to send a reminder signal if contact is about to take place.

[0144] The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus.

[0145] Various general-purpose systems can also be used with the teachings based thereon.

[0146] The structure required to construct such a system should be apparent from the above description.

[0147] Furthermore, the present invention is not specific to any particular programming language.

[0148] It will be appreciated that a variety of programming languages ​​may be used to implement the teachings of the present invention as described herein and that the above descriptions using specific languages ​​are intended to disclose the best mode for carrying out the present invention.

[0149] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof.

[0150] This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0151] Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment.

[0152] Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Claims

1. A non-contact ultrasonic probe detection method based on deep learning, characterized in that: include: Obtaining plane images and three-dimensional matrix of range measurement at multiple time points; The three-dimensional distance measurement matrix represents the distance to the surrounding structures detected with the rotating distance measurement device as the center; the plane image represents the image of the movement path of the rotating distance measurement device taken by the micro camera on the rotating distance measurement device; Establishing an association relationship based on the planar images and the ranging three-dimensional matrix at the multiple time points to obtain a first association set; The first association set includes an associated planar image at one time point and a three-dimensional matrix of range measurement at another time point; Based on the ranging three-dimensional matrix, the collision boundary is predicted to obtain a second three-dimensional matrix; Obtaining correlation features based on the plane image, the ranging three-dimensional matrix, and the second three-dimensional matrix; Multiple time points correspond to multiple associated features; According to the time points from early to late, the related features of multiple time points are input into the temporal convolutional network in sequence to predict the position of the next time point and determine whether contact will occur; If contact is to take place, send a reminder signal; The method of predicting the collision boundary based on the ranging three-dimensional matrix to obtain the second three-dimensional matrix includes: Taking the midpoint of the three-dimensional distance measurement matrix as the origin, constructing a three-dimensional coordinate axis; Project the position in the distance measurement three-dimensional matrix onto the position in the three-dimensional coordinate axis; mark the value in the distance measurement three-dimensional matrix at the corresponding position on the three-dimensional coordinate axis; In the three-dimensional coordinate axis, based on the three-dimensional distance measurement matrix, a minimum obstacle distance is obtained; Take the center point of the three-dimensional distance measurement matrix as the center of the sphere and the radius of the sphere with the minimum obstacle distance to obtain the collision sphere; Take the largest inscribed cube in the collision sphere as the first cube; Constructing a second three-dimensional matrix with the values ​​contained in the first cube; the values ​​in the second three-dimensional matrix are 0; the number of rows, columns and layers of the second three-dimensional matrix represent the positions that can collide; The obtaining of the associated features based on the plane image, the ranging three-dimensional matrix and the second three-dimensional matrix includes: The distance measurement three-dimensional matrix is ​​cut by the surface of the number of rows and layers corresponding to the center point to obtain a first cut distance matrix; Based on the plane image, multiple first features are obtained through a plane convolution network; Based on the first cutting distance matrix, a plurality of second features are obtained through a first ranging convolutional network; One first feature corresponds to one second feature; Based on the second three-dimensional matrix, the first feature and the corresponding second feature, an association feature is obtained through an association network.

2. The non-contact ultrasonic probe detection method based on deep learning according to claim 1 is characterized in that: The obtaining of the associated feature through an associated network based on the second three-dimensional matrix, the first feature and the corresponding second feature includes: The associated network includes a first associated network and a second associated network; Subtract the first feature from the second feature to obtain a first associated feature; obtain multiple first associated features corresponding to multiple first features; constructing the plurality of first correlation features into a first correlation matrix according to the columns of the first cutting distance matrix corresponding to the plurality of first correlation features; Inputting the first association matrix into the first association network to obtain a second association feature; Based on the second three-dimensional matrix, obtaining a third association feature through a second association network; The second association feature and the third association feature are superimposed to obtain an association feature.

3. The non-contact ultrasonic probe detection method based on deep learning according to claim 2 is characterized in that: The obtaining of the third association feature based on the second three-dimensional matrix through the second association network includes: Acquire a plurality of second three-dimensional matrices in a time period between the first time point and the second time point; The plurality of second three-dimensional matrices are input into the second association network to obtain third association features.

4. The non-contact ultrasonic probe detection method based on deep learning according to claim 1 is characterized in that: Based on the planar image, a plurality of first features are obtained through a planar convolutional network, including: Taking the quotient of the length of the plane image divided by the number of columns of the first cutting distance matrix as the plane difference value; The length of the plane image is divided sequentially according to the plane difference value to obtain a plurality of plane column images; The planar convolution network includes a three-dimensional convolution kernel of m*2*3; m represents the quotient of the length of the planar image divided by the number of columns of the first cutting distance matrix; the 3 in the three-dimensional convolution kernel of the planar convolution network represents 3 RGB values; the 2 in the three-dimensional convolution kernel of the planar convolution network represents 2 rows in the planar column image; According to the direction of the column corresponding to the planar column image, with a step size of 1, convolve the three-dimensional convolution kernel corresponding to the planar convolution network on the planar column image to obtain a first feature; A plurality of first features are obtained corresponding to the plurality of planar array images.

5. The non-contact ultrasonic probe detection method based on deep learning according to claim 1 is characterized in that: The method of obtaining a plurality of second features based on the first cutting distance matrix through a first ranging convolutional network includes: The first cutting distance matrix is ​​divided into columns to obtain a plurality of first cutting column matrices; one first cutting column matrix corresponds to one column of the first cutting distance matrix; The first distance measurement convolution network includes a 2*n two-dimensional convolution kernel; n represents the number of layers of the first cutting distance matrix; 2 in the two-dimensional convolution kernel of the first distance measurement convolution network represents 2 rows in the first cutting column matrix; According to the direction of the column corresponding to the first cutting column matrix, the two-dimensional convolution kernel corresponding to the first ranging convolution is convolved on the first cutting column matrix with a step size of 1 to obtain a second feature; A plurality of first cutting column matrices correspondingly obtain a plurality of second features.

6. The non-contact ultrasonic probe detection method based on deep learning according to claim 1, characterized in that: The establishing of an association relationship based on the planar images and the ranging three-dimensional matrix at the multiple time points to obtain a first association set includes: Obtaining a first time point and a second time point; the first time point is a time point corresponding to a plane image; the second time point is a time point when the rotating distance measuring device reaches a position corresponding to the plane image at the first time point; An association relationship is established between the distance measurement three-dimensional matrix corresponding to the second time point and the plane image at the first time point.

7. The non-contact ultrasonic probe detection method based on deep learning according to claim 1, characterized in that: The step of obtaining the minimum obstacle distance in the three-dimensional coordinate axis based on the three-dimensional distance measurement matrix includes: Taking the position of 1 in the ranging three-dimensional matrix as the obstacle position; Detect the distance between the origin and the obstacle position in the three-dimensional coordinate axis to obtain multiple obstacle distances; An obstacle distance that is smaller than other obstacle distances among the plurality of obstacle distances is taken as the minimum obstacle distance.

8. A non-contact ultrasonic probe detection system based on deep learning, characterized in that: include: An acquisition module, used to obtain plane images and ranging three-dimensional matrices at multiple time points; The three-dimensional distance measurement matrix represents the distance to the surrounding structures detected with the rotating distance measurement device as the center; the plane image represents the image of the movement path of the rotating distance measurement device taken by the micro camera on the rotating distance measurement device; An association module is used to establish an association relationship based on the plane images and the ranging three-dimensional matrix at the multiple time points to obtain a first association set; the first association set includes the plane image at one associated time point and the ranging three-dimensional matrix at another associated time point; A boundary module is used to predict the collision boundary based on the ranging three-dimensional matrix to obtain a second three-dimensional matrix; A correlation feature module, which obtains correlation features based on the plane image, the ranging three-dimensional matrix and the second three-dimensional matrix; Multiple time points correspond to multiple associated features; The prediction module is used to input the associated features of multiple time points into the temporal convolutional network in order from early to late, predict the position of the next time point, and determine whether contact will occur; A reminder module, used to send a reminder signal if contact is about to take place; The method of predicting the collision boundary based on the ranging three-dimensional matrix to obtain the second three-dimensional matrix includes: Taking the midpoint of the three-dimensional distance measurement matrix as the origin, constructing a three-dimensional coordinate axis; Project the position in the distance measurement three-dimensional matrix onto the position in the three-dimensional coordinate axis; mark the value in the distance measurement three-dimensional matrix at the corresponding position on the three-dimensional coordinate axis; In the three-dimensional coordinate axis, based on the three-dimensional distance measurement matrix, a minimum obstacle distance is obtained; Take the center point of the three-dimensional distance measurement matrix as the center of the sphere and the radius of the sphere with the minimum obstacle distance to obtain the collision sphere; Take the largest inscribed cube in the collision sphere as the first cube; Constructing a second three-dimensional matrix with the values ​​contained in the first cube; the values ​​in the second three-dimensional matrix are 0; the number of rows, columns and layers of the second three-dimensional matrix represent the positions that can collide; The obtaining of the associated features based on the plane image, the ranging three-dimensional matrix and the second three-dimensional matrix includes: The distance measurement three-dimensional matrix is ​​cut by the surface of the number of rows and layers corresponding to the center point to obtain a first cut distance matrix; Based on the plane image, multiple first features are obtained through a plane convolution network; Based on the first cutting distance matrix, a plurality of second features are obtained through a first ranging convolutional network; One first feature corresponds to one second feature; Based on the second three-dimensional matrix, the first feature and the corresponding second feature, an association feature is obtained through an association network.

Images