Video auxiliary analysis system and method based on artificial intelligence

By calculating the pixel and structural details loss of ultrasound images, combined with segmented frequency domain image enhancement and boundary loss weight, the segmentation accuracy of ultrasound images is improved, the image edge blur and feature area problems are solved, and the quality of video-assisted analysis is improved.

CN119027946AInactive Publication Date: 2024-11-26HUA PING XIANGSHENG (SHANGHAI) MEDICAL TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411065569.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When segmenting ultrasound images, the image edges are blurred and the feature areas are too large or too small. Deep convolutional network abstraction often introduces background feature interference, resulting in a reduction in the accuracy of the scanning trajectory of the ultrasound probe.

Method used

Using an artificial intelligence-based video-assisted analysis system, the pixel details and structural details losses are calculated, and the image contrast and texture information are enhanced using the segmented frequency domain image enhancement module, combined with real images and predicted images for restoration, increasing the boundary loss weight, and fusing the global Semantic information to improve segmentation accuracy.

Benefits of technology

It improves the segmentation accuracy and clarity of ultrasound images, reduces multiple scans of the same checkpoint by the ultrasound probe, reduces image differences, facilitates overall analysis, and improves the quality of video-assisted analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027946A_ABST
    Figure CN119027946A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of video auxiliary analysis, in particular to a video auxiliary analysis system and method based on artificial intelligence. The invention discloses a video auxiliary analysis method based on artificial intelligence, and the method comprises the steps: 1, carrying out the ultrasonic scanning of a to-be-detected person through an ultrasonic robot, and obtaining an original ultrasonic image; calculating fuzzy pixel detail loss and structure detail loss in the original ultrasonic image to obtain a fine processing ultrasonic image; acquiring a video acquired by the ultrasonic probe in the fine processing ultrasonic image and a scanning track of the ultrasonic probe; and 2, carrying out data recording on a scanning process based on a human body digital twin model, and realizing autonomous ultrasonic scanning through a scanning thought decision maker and a video auxiliary analysis module. According to the method, the image segmentation result is evaluated in a unified manner by calculating the fuzzy pixel detail loss and the structure detail loss in the original ultrasonic image, and the accuracy and definition of the original ultrasonic image used for follow-up video auxiliary analysis are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of video-assisted analysis, and in particular to a video-assisted analysis system and method based on artificial intelligence. Background Art

[0002] With the rapid development of modern science and technology and the advent of the intelligent era, the robot's high precision, high repeatability and controllability can be used to avoid direct contact, and the ultrasonic probe can be driven by the robotic arm to move along a predetermined trajectory to detect the target object.

[0003] Since most of the targets to be detected are in closed spaces, it is impossible to fully understand the position and environment of the target before the ultrasound probe is turned on. With the help of the video-assisted analysis system, after obtaining real-time ultrasound images, the ultrasound images can be further analyzed, and the ultrasound probe movement trajectory can be modified according to the real-time position and environment of the target, so that the ultrasound probe movement trajectory is more in line with the actual conditions of different subjects to be detected, thereby shortening the number of detections and the time required for each detection.

[0004] Image segmentation is mostly aimed at CT and MRI images with fixed imaging methods, but the segmentation effect for non-fixed ultrasound images is not ideal. The analysis of ultrasound images depends on the accuracy and clarity of ultrasound images. In the process of ultrasound image segmentation, the edges of the image will become blurred or even lost in the existing technology. The feature areas in the image are too large or too small. When the deep convolutional network abstracts and restores the semantic information, background features are often introduced to interfere with the image segmentation results, resulting in a decrease in the accuracy of the scanning trajectory of the ultrasound probe obtained by subsequent video-assisted analysis. Summary of the invention

[0005] The purpose of the present invention is to provide a video assisted analysis system and method based on artificial intelligence to solve the existing problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a video-assisted analysis method based on artificial intelligence, the method comprising:

[0007] Step 1: Perform ultrasonic scanning on the person to be tested by the ultrasonic robot to obtain an original ultrasonic image; calculate the blurred pixel detail loss and structural detail loss in the original ultrasonic image to obtain a refined ultrasonic image; and obtain the video obtained by the ultrasonic probe and the scanning trajectory of the ultrasonic probe in the refined ultrasonic image;

[0008] Step 2: Record the data of the scanning process based on the human body digital twin model, and use the learning framework to train the data for reinforcement learning, and realize autonomous ultrasound scanning through the scanning idea decision maker and the video-assisted analysis module;

[0009] Step 3: Analyze the refined ultrasonic image based on the scanning idea decision maker and the video-assisted analysis module, and output the analysis result; the analysis result includes the target part in the refined ultrasonic image, and the target part is used to reflect the part corresponding to the abnormal positioning point in the refined ultrasonic image;

[0010] Step 4: dynamically updating the human body digital twin model in real time according to the displayed positioning points in the refined processed ultrasound image;

[0011] Step 5: Based on the analysis results, the preset learning framework is fed back and updated, the scanning strategy decision maker is updated, and the results are output through the video-assisted analysis module;

[0012] The calculation of blurred pixel detail loss and structural detail loss in the original ultrasound image described in step 1 includes:

[0013] After the original ultrasound image is segmented, the segmented frequency domain image enhancement module is used to enhance the contrast and texture information of the image, and each pixel is judged to be a target pixel or a background pixel. The convergence of detail pixels is accelerated by comparing the average weight of the target pixel and the background pixel, and the pixel detail loss is obtained. The calculation formula is:

[0014] S1=-∑{M(a,b)log[N(a,b)]+[1-M(a,b)]log[1-N(a,b)]}

[0015] Wherein, M represents the real image, M(a,b) represents the real value label of the pixel (a,b), N represents the predicted image, and N(a,b) represents the possibility of predicting the target; since the original ultrasound image itself is converted from the sound wave signal, and the different imaging methods of different machines may cause obvious differences in the generated ultrasound images, if the data source differences are large, it will easily lead to large differences in network training, and it is not easy to fit the training model parameters. The present invention enhances the contrast of the image itself and weakens the differences between images, adopts a restoration method that combines mixed real images with predicted images, enhances the spatial position information and local semantic information during feature restoration, and thus improves the segmentation accuracy of the network.

[0016] By assigning a larger weight to the boundary neighborhood and increasing the loss weight of the boundary, the prediction probability of the boundary and the rest of the target area is guaranteed to be the same, the global semantic information under different receptive fields is fused to enhance the network's ability to capture targets of different sizes, and the boundary channel information is combined with the semantic channel information to enhance the attention of high-level features in key semantic channels and improve the extraction of key target features after cutting. The structural detail loss is obtained, and the calculation formula is:

[0017]

[0018] Among them, x is the vector of the relevant patch cut from the predicted image, y is the vector of the relevant patch cut from the real image, and d x and d y are the means of x and y, respectively, and e x and e y are the standard deviations of x and y respectively, and F1 and F2 are constant parameters that are set. The image quality of the original ultrasound image can be improved, and its edges can be identified more clearly. For the same inspection point, there is no need to repeatedly move the ultrasound probe for multiple inspections, which reduces the time required for inspection; it reduces the differences between the original ultrasound images collected, which is convenient for the overall analysis of the sequence images, because a larger weight is assigned to the boundary neighborhood, the loss weight of the boundary is increased, and the prediction probability of the boundary and the rest of the target area is guaranteed to be the same, and the target feature information is gradually optimized, which improves the accuracy, clarity and completeness of the image segmentation and improves the quality of video-assisted analysis.

[0019] When building the human digital twin model described in step 2, it includes:

[0020] Calibrate and update the human body digital twin model based on the refined ultrasound image standard image;

[0021] The robot arm trajectory of the human digital twin model is normalized and represented based on a chessboard grid.

[0022] In step 2, the scanning process is recorded based on the human body digital twin model, and the data is trained for reinforcement learning using the learning framework, and autonomous ultrasound scanning is achieved through the scanning idea decision maker and the video-assisted analysis module, including:

[0023] According to the finely processed ultrasonic image, obtaining abnormal positioning points of the person to be tested and marking the abnormal positioning points;

[0024] The processed ultrasound image is parsed and semantically analyzed based on image parsing and segmentation, and the analysis results are output, wherein the analysis results include the location of the abnormal feature, the abnormal feature in the next partition, the ultrasound video of the abnormal feature, and the determination result of the abnormal feature.

[0025] In step 5, based on the analysis results, the preset learning framework is fed back and updated, the scanning idea decision maker is updated, and the results are output through the video-assisted analysis module, including:

[0026] When the abnormal features in the next partition and the determination results of the abnormal features are input into the preset learning framework for training, and fed back to the scanning idea decision maker, the scanning idea decision maker is used to judge the abnormal features and their positions;

[0027] When the position of the abnormal feature and the determination result of the abnormal feature are input into the preset learning framework for training, the result is output through the video-assisted analysis module.

[0028] In step 5, the preset learning framework is fed back and updated based on the analysis results, the scanning strategy decision maker is updated, and the results are output through the video-assisted analysis module, which also includes:

[0029] The PPO algorithm is used to obtain the mapping relationship between the abnormal positioning point and the position of the abnormal feature in the analysis result;

[0030] The mapping relationship is input into the preset learning framework for training, and the result is output through the scanning idea decider and the video-assisted analysis module.

[0031] The PPO algorithm adopts a normal distribution model, and uses mean μ and variance σ to represent the probabilities of various reactions that will occur in a continuous action.

[0032] A video-assisted analysis system based on artificial intelligence, the system comprising:

[0033] a scanning trajectory acquisition unit, configured to perform ultrasonic scanning on the person to be tested by the ultrasonic robot to obtain an original ultrasonic image, calculate blurred pixel detail loss and structural detail loss in the original ultrasonic image, obtain a refined ultrasonic image, and obtain a video obtained by the ultrasonic probe in the ultrasonic image and a scanning trajectory of the ultrasonic probe;

[0034] A learning and training unit, which is used to record data of the scanning process based on the human body digital twin model, and use the learning framework to perform reinforcement learning training on the data, and realize autonomous ultrasound scanning through a scanning idea decider and a video-assisted analysis module;

[0035] An analysis unit, configured to analyze the refined processed ultrasound image based on the scanning strategy decision maker and the video-assisted analysis module, and output an analysis result, wherein the analysis result includes a target part in the refined processed ultrasound image, and the target part is used to reflect a part corresponding to an abnormal positioning point in the refined processed ultrasound image;

[0036] The learning and training unit, when also used to construct the human digital twin model, includes:

[0037] Calibrate and update the human body digital twin model based on the refined ultrasound image standard image;

[0038] Normalizing the trajectory of the robot arm of the human digital twin model based on a chessboard grid;

[0039] According to the displayed positioning points in the refined processed ultrasound image, the human body digital twin model is dynamically updated in real time;

[0040] The learning and training unit comprises:

[0041] A marking subunit, used for obtaining abnormal positioning points of the person to be tested and marking the abnormal positioning points according to the finely processed ultrasonic image;

[0042] An analysis subunit, for performing analysis and semantic analysis on the refined ultrasonic image based on image analysis and segmentation, and outputting analysis results, wherein the analysis results include the location of the abnormal feature, the abnormal feature in the next partition, the ultrasonic video of the abnormal feature, and the determination result of the abnormal feature;

[0043] A learning and training subunit, configured to provide feedback and update the preset learning framework based on the analysis results, update the scanning strategy decision maker, and output the results through the video-assisted analysis module;

[0044] The learning and training subunit comprises:

[0045] A first learning and training subunit is used for inputting the abnormal features in the next partition and the determination results of the abnormal features into the preset learning framework for training, and the scanning idea decision maker is used for judging the abnormal features and their positions through processing by the scanning idea decision maker;

[0046] The second learning and training subunit is used to input the position of the abnormal feature and the determination result of the abnormal feature into the preset learning framework for training to obtain the video-assisted analysis module.

[0047] The learning and training subunit comprises:

[0048] A mapping relationship acquisition subunit, used to acquire a mapping relationship between the abnormal positioning point and the position of the abnormal feature in the analysis result by using a PPO algorithm;

[0049] A training subunit is used to input the mapping relationship into the preset learning framework for training, and output the result through the scanning idea decider and the video-assisted analysis module.

[0050] The PPO algorithm adopts a normal distribution model, and uses mean μ and variance σ to represent the probabilities of various reactions that will occur in a continuous action.

[0051] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0052] The present invention enhances the contrast of the image itself and weakens the difference between images, adopts a restoration method that combines a mixed real image with a predicted image, enhances the spatial position information and local semantic information during feature restoration, thereby improving the segmentation accuracy of the network; fuses the global semantic information under receptive fields of different sizes to enhance the network's ability to capture targets of different sizes, combines boundary channel information with semantic channel information, enhances the attention of high-level features in key semantic channels, improves the extraction of key target features after cutting, improves the image quality of the original ultrasound image, can more clearly identify its edges, and does not need to repeatedly move the ultrasound probe for multiple inspections of the same inspection point, thereby reducing the time required for the inspection; reduces the differences between the original ultrasound images collected, and facilitates the overall analysis of the sequence of images, because a larger weight is assigned to the boundary neighborhood, the loss weight of the boundary is increased, and the prediction probability of the boundary and the rest of the target area is guaranteed to be the same, and the target feature information is gradually optimized, thereby improving the accuracy, clarity and completeness of image segmentation, and improving the quality of video-assisted analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0054] Figure 1 is a flow chart of the method of the present invention;

[0055] Figure 2 It is a system composition diagram of the present invention; DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] See also Figure 1 The present invention provides a technical solution: a video-assisted analysis method based on artificial intelligence, the method comprising:

[0058] Step 1: Perform ultrasonic scanning on the person to be tested by the ultrasonic robot to obtain an original ultrasonic image; calculate the blurred pixel detail loss and structural detail loss in the original ultrasonic image to obtain a refined ultrasonic image; and obtain the video obtained by the ultrasonic probe and the scanning trajectory of the ultrasonic probe in the refined ultrasonic image;

[0059] Step 2: Record the data of the scanning process based on the human body digital twin model, and use the learning framework to train the data for reinforcement learning, and realize autonomous ultrasound scanning through the scanning idea decision maker and the video-assisted analysis module;

[0060] Step 3: Analyze the refined ultrasonic image based on the scanning idea decision maker and the video-assisted analysis module, and output the analysis result; the analysis result includes the target part in the refined ultrasonic image, and the target part is used to reflect the part corresponding to the abnormal positioning point in the refined ultrasonic image;

[0061] Step 4: dynamically updating the human body digital twin model in real time according to the displayed positioning points in the refined processed ultrasound image;

[0062] Step 5: Based on the analysis results, the preset learning framework is fed back and updated, the scanning strategy decision maker is updated, and the results are output through the video-assisted analysis module;

[0063] The calculation of blurred pixel detail loss and structural detail loss in the original ultrasound image described in step 1 includes:

[0064] After the original ultrasound image is segmented, the segmented frequency domain image enhancement module is used to enhance the contrast and texture information of the image, and each pixel is judged to be a target pixel or a background pixel. The convergence of detail pixels is accelerated by comparing the average weight of the target pixel and the background pixel, and the pixel detail loss is obtained. The calculation formula is:

[0065] S1=-∑{M(a,b)log[N(a,b)]+[1-M(a,b)]log[1-N(a,b)]}

[0066] Where M represents the real image, M(a,b) represents the real value label of the pixel (a,b), N represents the predicted image, and N(a,b) represents the possibility of predicting the target;

[0067] By assigning a larger weight to the boundary neighborhood, increasing the loss weight of the boundary, and ensuring that the prediction probability of the boundary is the same as that of the rest of the target area, the structural detail loss is obtained, and the calculation formula is:

[0068]

[0069] Among them, x is the vector of the relevant patch cut from the predicted image, y is the vector of the relevant patch cut from the real image, and d x and d y are the means of x and y, respectively, and e x and e y are the standard deviations of x and y respectively, and F1 and F2 are constant parameters set.

[0070] When building the human digital twin model described in step 2, it includes:

[0071] Calibrate and update the human body digital twin model based on the refined ultrasound image standard image;

[0072] The robot arm trajectory of the human digital twin model is normalized and represented based on a chessboard grid.

[0073] In step 2, the scanning process is recorded based on the human body digital twin model, and the data is trained for reinforcement learning using the learning framework, and autonomous ultrasound scanning is achieved through the scanning idea decision maker and the video-assisted analysis module, including:

[0074] According to the finely processed ultrasonic image, obtaining abnormal positioning points of the person to be tested and marking the abnormal positioning points;

[0075] The processed ultrasound image is parsed and semantically analyzed based on image parsing and segmentation, and the analysis results are output, wherein the analysis results include the location of the abnormal feature, the abnormal feature in the next partition, the ultrasound video of the abnormal feature, and the determination result of the abnormal feature.

[0076] In step 5, based on the analysis results, the preset learning framework is fed back and updated, the scanning idea decision maker is updated, and the results are output through the video-assisted analysis module, including:

[0077] When the abnormal features in the next partition and the determination results of the abnormal features are input into the preset learning framework for training, and fed back to the scanning idea decision maker, the scanning idea decision maker is used to judge the abnormal features and their positions;

[0078] When the position of the abnormal feature and the determination result of the abnormal feature are input into the preset learning framework for training, the result is output through the video-assisted analysis module.

[0079] In step 5, the preset learning framework is fed back and updated based on the analysis results, the scanning strategy decision maker is updated, and the results are output through the video-assisted analysis module, which also includes:

[0080] The PPO algorithm is used to obtain the mapping relationship between the abnormal positioning point and the position of the abnormal feature in the analysis result;

[0081] The mapping relationship is input into the preset learning framework for training, and the result is output through the scanning idea decider and the video-assisted analysis module.

[0082] The PPO algorithm adopts a normal distribution model, and uses mean μ and variance σ to represent the probabilities of various reactions that will occur in a continuous action.

[0083] Figure 2 , a video-assisted analysis system based on artificial intelligence, the system comprising:

[0084] a scanning trajectory acquisition unit, configured to perform ultrasonic scanning on the person to be tested by the ultrasonic robot to obtain an original ultrasonic image, calculate blurred pixel detail loss and structural detail loss in the original ultrasonic image, obtain a refined ultrasonic image, and obtain a video obtained by the ultrasonic probe in the ultrasonic image and a scanning trajectory of the ultrasonic probe;

[0085] A learning and training unit, which is used to record data of the scanning process based on the human body digital twin model, and use the learning framework to perform reinforcement learning training on the data, and realize autonomous ultrasound scanning through a scanning idea decider and a video-assisted analysis module;

[0086] An analysis unit, configured to analyze the refined processed ultrasound image based on the scanning strategy decision maker and the video-assisted analysis module, and output an analysis result, wherein the analysis result includes a target part in the refined processed ultrasound image, and the target part is used to reflect a part corresponding to an abnormal positioning point in the refined processed ultrasound image;

[0087] The learning and training unit, when also used to construct the human digital twin model, includes:

[0088] Calibrate and update the human body digital twin model based on the refined ultrasound image standard image;

[0089] Normalizing the trajectory of the robot arm of the human digital twin model based on a chessboard grid;

[0090] According to the displayed positioning points in the refined processed ultrasound image, the human body digital twin model is dynamically updated in real time;

[0091] The learning and training unit comprises:

[0092] A marking subunit, used for obtaining abnormal positioning points of the person to be tested and marking the abnormal positioning points according to the finely processed ultrasonic image;

[0093] An analysis subunit, for performing analysis and semantic analysis on the refined ultrasonic image based on image analysis and segmentation, and outputting analysis results, wherein the analysis results include the location of the abnormal feature, the abnormal feature in the next partition, the ultrasonic video of the abnormal feature, and the determination result of the abnormal feature;

[0094] A learning and training subunit, configured to provide feedback and update the preset learning framework based on the analysis results, update the scanning strategy decision maker, and output the results through the video-assisted analysis module;

[0095] The learning and training subunit comprises:

[0096] A first learning and training subunit is used for inputting the abnormal features in the next partition and the determination results of the abnormal features into the preset learning framework for training, and the scanning idea decision maker is used for judging the abnormal features and their positions through processing by the scanning idea decision maker;

[0097] The second learning and training subunit is used to input the position of the abnormal feature and the determination result of the abnormal feature into the preset learning framework for training to obtain the video-assisted analysis module.

[0098] The learning and training subunit comprises:

[0099] A mapping relationship acquisition subunit, used to acquire a mapping relationship between the abnormal positioning point and the position of the abnormal feature in the analysis result by using a PPO algorithm;

[0100] A training subunit is used to input the mapping relationship into the preset learning framework for training, and output the result through the scanning idea decider and the video-assisted analysis module.

[0101] The PPO algorithm adopts a normal distribution model, and uses mean μ and variance σ to represent the probabilities of various reactions that will occur in a continuous action.

[0102] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video-assisted analysis method based on artificial intelligence, characterized in that: The method includes: Step 1: Perform ultrasonic scanning on the person to be tested by the ultrasonic robot to obtain an original ultrasonic image; calculate the blurred pixel detail loss and structural detail loss in the original ultrasonic image to obtain a refined ultrasonic image; and obtain the video obtained by the ultrasonic probe and the scanning trajectory of the ultrasonic probe in the refined ultrasonic image; Step 2: Record the data of the scanning process based on the human body digital twin model, and use the learning framework to train the data for reinforcement learning, and realize autonomous ultrasound scanning through the scanning idea decision maker and the video-assisted analysis module; Step 3: Analyze the refined ultrasonic image based on the scanning idea decision maker and the video-assisted analysis module, and output the analysis result; the analysis result includes the target part in the refined ultrasonic image, and the target part is used to reflect the part corresponding to the abnormal positioning point in the refined ultrasonic image; Step 4: dynamically updating the human body digital twin model in real time according to the displayed positioning points in the refined processed ultrasound image; Step 5: Based on the analysis results, the preset learning framework is fed back and updated, the scanning strategy decision maker is updated, and the results are output through the video-assisted analysis module.

2. The video-assisted analysis method based on artificial intelligence according to claim 1, characterized in that: The calculation of blurred pixel detail loss and structural detail loss in the original ultrasound image described in step 1 includes: After the original ultrasound image is segmented, the segmented frequency domain image enhancement module is used to enhance the contrast and texture information of the image, and each pixel is judged to be a target pixel or a background pixel. The convergence of detail pixels is accelerated by comparing the average weight of the target pixel and the background pixel, and the pixel detail loss is obtained. The calculation formula is: S1=-∑{M(a,b)log[N(a,b)]+[1-M(a,b)]log[1-N(a,b)]} Where M represents the real image, M(a,b) represents the real value label of the pixel (a,b), N represents the predicted image, and N(a,b) represents the possibility of predicting the target; By assigning a larger weight to the boundary neighborhood, increasing the loss weight of the boundary, and ensuring that the prediction probability of the boundary is the same as that of the rest of the target area, the structural detail loss is obtained, and the calculation formula is: Among them, x is the vector of the relevant patch cut from the predicted image, y is the vector of the relevant patch cut from the real image, and d x and d y are the means of x and y, respectively, and e x and e y are the standard deviations of x and y respectively, and F1 and F2 are constant parameters set.

3. The video-assisted analysis method based on artificial intelligence according to claim 1 is characterized in that: When building the human digital twin model described in step 2, it includes: Calibrate and update the human body digital twin model based on the refined ultrasound image standard image; The robot arm trajectory of the human digital twin model is normalized and represented based on a chessboard grid.

4. The video-assisted analysis method based on artificial intelligence according to claim 1 is characterized in that: In step 2, the scanning process is recorded based on the human body digital twin model, and the data is trained for reinforcement learning using the learning framework, and autonomous ultrasound scanning is achieved through the scanning idea decision maker and the video-assisted analysis module, including: According to the finely processed ultrasonic image, obtaining abnormal positioning points of the person to be tested and marking the abnormal positioning points; The processed ultrasound image is parsed and semantically analyzed based on image parsing and segmentation, and the analysis results are output, wherein the analysis results include the location of the abnormal feature, the abnormal feature in the next partition, the ultrasound video of the abnormal feature, and the determination result of the abnormal feature.

5. The video-assisted analysis method based on artificial intelligence according to claim 4 is characterized in that: In step 5, based on the analysis results, the preset learning framework is fed back and updated, the scanning idea decision maker is updated, and the results are output through the video-assisted analysis module, including: When the abnormal features in the next partition and the determination results of the abnormal features are input into the preset learning framework for training, and fed back to the scanning idea decision maker, the scanning idea decision maker is used to judge the abnormal features and their positions; When the position of the abnormal feature and the determination result of the abnormal feature are input into the preset learning framework for training, the result is output through the video-assisted analysis module.

6. The video-assisted analysis method based on artificial intelligence according to claim 5 is characterized in that: In step 5, based on the analysis results, the preset learning framework is fed back and updated, the scanning idea decision maker is updated, and the results are output through the video-assisted analysis module, and the following is also included: The PPO algorithm is used to obtain the mapping relationship between the abnormal positioning point and the position of the abnormal feature in the analysis result; The mapping relationship is input into the preset learning framework for training, and the result is output through the scanning idea decider and the video-assisted analysis module.

7. The video-assisted analysis method based on artificial intelligence according to claim 6 is characterized in that: The PPO algorithm adopts a normal distribution model, and uses mean μ and variance σ to represent the probabilities of various reactions that will occur in a continuous action.

8. An artificial intelligence-based video-assisted analysis system, applied to an artificial intelligence-based video-assisted analysis method according to any one of claims 1 to 7, characterized in that: The system comprises: a scanning trajectory acquisition unit, configured to perform ultrasonic scanning on the person to be tested by the ultrasonic robot to obtain an original ultrasonic image, calculate blurred pixel detail loss and structural detail loss in the original ultrasonic image, obtain a refined ultrasonic image, and obtain a video obtained by the ultrasonic probe in the ultrasonic image and a scanning trajectory of the ultrasonic probe; A learning and training unit, which is used to record data of the scanning process based on the human body digital twin model, and use the learning framework to perform reinforcement learning training on the data, and realize autonomous ultrasound scanning through a scanning idea decider and a video-assisted analysis module; An analysis unit, configured to analyze the refined processed ultrasound image based on the scanning strategy decision maker and the video-assisted analysis module, and output an analysis result, wherein the analysis result includes a target part in the refined processed ultrasound image, and the target part is used to reflect a part corresponding to an abnormal positioning point in the refined processed ultrasound image; The learning and training unit, when also used to construct the human digital twin model, includes: Calibrate and update the human body digital twin model based on the refined ultrasound image standard image; Normalizing the trajectory of the robot arm of the human digital twin model based on a chessboard grid; According to the displayed positioning points in the refined processed ultrasound image, the human body digital twin model is dynamically updated in real time; The learning and training unit comprises: A marking subunit, used for obtaining abnormal positioning points of the person to be tested and marking the abnormal positioning points according to the finely processed ultrasonic image; An analysis subunit, for performing analysis and semantic analysis on the refined ultrasonic image based on image analysis and segmentation, and outputting analysis results, wherein the analysis results include the location of the abnormal feature, the abnormal feature in the next partition, the ultrasonic video of the abnormal feature, and the determination result of the abnormal feature; A learning and training subunit, configured to provide feedback and update the preset learning framework based on the analysis results, update the scanning strategy decision maker, and output the results through the video-assisted analysis module; The learning and training subunit comprises: A first learning and training subunit is used for inputting the abnormal features in the next partition and the determination results of the abnormal features into the preset learning framework for training, and the scanning idea decision maker is used for judging the abnormal features and their positions through processing by the scanning idea decision maker; The second learning and training subunit is used to input the position of the abnormal feature and the determination result of the abnormal feature into the preset learning framework for training to obtain the video-assisted analysis module.

9. The artificial intelligence-based video-assisted analysis system according to claim 8, characterized in that: The learning and training subunit comprises: A mapping relationship acquisition subunit, used to acquire a mapping relationship between the abnormal positioning point and the position of the abnormal feature in the analysis result by using a PPO algorithm; A training subunit is used to input the mapping relationship into the preset learning framework for training, and output the result through the scanning idea decider and the video-assisted analysis module.

10. The artificial intelligence-based video-assisted analysis system according to claim 9, characterized in that: The PPO algorithm adopts a normal distribution model, and uses mean μ and variance σ to represent the probabilities of various reactions that will occur in a continuous action.

Citation Information

Patent Citations

  • Ultrasonic robot state normalization method and system based on human body characteristics

    CN112132805A

  • Ultrasonic robot online decision-making method and system taking diagnosis result as feedback

    CN112270993A

  • Breast ultrasonic tomographic image segmentation model establishment method and segmentation method

    CN116433586A

  • Shape-based generative adversarial network for segmentation in medical imaging

    US20210038198A1