Inspection report generation method and device, equipment and storage medium

Through automated capsule intestinal video quality evaluation and lesion recognition, accurate and complete examination reports are generated, which solves the human resource consumption and accuracy problems in the traditional report generation process, and improves the efficiency and reliability of the report.

CN118969167APending Publication Date: 2024-11-15XIEHE HOSPITAL ATTACHED TO TONGJI MEDICAL COLLEGE HUAZHONG SCI & TECH UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410988235.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The generation process of traditional capsule intestinal tractoscopy consumes a lot of human resources, and the completeness and accuracy of the report are difficult to guarantee, which can easily lead to misdiagnosis or misdiagnosis of the lesions, affecting the patient's treatment and follow-up.

Method used

By obtaining and quality evaluation of capsule entanglement videos, identifying and detecting lesions, obtaining supplementary information based on lesion categories, and using report templates to generate automated examination reports to improve the accuracy and efficiency of reports.

Benefits of technology

Automatic detection and analysis of capsule intestinal videos is realized, which improves the accuracy and timeliness of lesion recognition, ensures the integrity and reliability of reports, and reduces the time and dependence of manual writing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118969167A_ABST
    Figure CN118969167A_ABST
Patent Text Reader

Abstract

The invention discloses an inspection report generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a collected inspection video, carrying out the quality evaluation of the inspection video, and obtaining a quality score of the inspection video; carrying out focus identification and detection on the examination video to obtain a focus and a focus type contained in the examination video; acquiring a currently used report template, and acquiring supplementary information of the report template according to the focus category; and performing information supplementation on the report template according to the supplementation information and the quality score, and generating an inspection report corresponding to the inspection video. Automatic detection, analysis and quality evaluation are carried out on video images collected by capsule endoscopy and other equipment, the accuracy and timeliness of focus recognition detection are improved, information improvement and supplementation are carried out according to detection and evaluation results when an inspection report is automatically generated, and the accuracy and timeliness of focus recognition detection are improved. And the efficiency, the integrity and the reliability of automatic generation of the inspection report are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of medical assistance technology, and in particular to a method, device, electronic device and storage medium for generating an examination report. Background Art

[0002] Capsule enteroscopy is the first-line diagnostic tool for small intestinal lesions, and a standardized, accurate and comprehensive capsule enteroscopy report is crucial and can provide important information to clinicians, endoscopists and patients.

[0003] Traditionally, when completing a capsule enteroscopy report, it is usually completed manually after the endoscopist has read the entire film. For each capsule enteroscopy video, the examination time is usually 8-10 hours. After the test is completed, the endoscopist usually needs to spend 1-2 hours reading the film, manually keeping the image, and writing the report. In addition to being very time-consuming, this process is also extremely dependent on the endoscopist's own diagnosis and treatment experience and work habits, such as report writing habits, which may lead to missed diagnosis or misdiagnosis of lesions due to long-term viewing and processing of videos, which will have an adverse impact on the patient's subsequent treatment and follow-up.

[0004] Therefore, there is an urgent need for an automatic method for generating capsule enteroscopy reports that can improve the quality, accuracy, and completeness of diagnostic reports. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method, device, electronic device and storage medium for generating an inspection report, so as to solve the technical problems in the related art that a large amount of human resources are required to obtain an inspection report, and the integrity and accuracy of the report cannot be ensured.

[0006] In a first aspect, an embodiment of the present application provides a method for generating an inspection report, comprising:

[0007] Acquire the collected inspection video, and perform quality assessment on the inspection video to obtain a quality score of the inspection video;

[0008] Identify and detect lesions in the inspection video to obtain lesions and lesion categories contained in the inspection video;

[0009] Acquire the currently used report template, and acquire supplementary information of the report template according to the lesion category;

[0010] The report template is supplemented with information according to the supplementary information and the quality score to generate an inspection report corresponding to the inspection video.

[0011] In a second aspect, an embodiment of the present application provides an inspection report generating device, including:

[0012] A quality assessment module is used to obtain the collected inspection video and perform quality assessment on the inspection video to obtain a quality score of the inspection video;

[0013] A lesion detection module, used to identify and detect lesions in the inspection video, and obtain the lesions and lesion categories contained in the inspection video;

[0014] An information acquisition module, used to acquire the currently used report template and acquire supplementary information of the report template according to the lesion category;

[0015] A report generation module is used to supplement the report template according to the supplementary information and the quality score, and generate an inspection report corresponding to the inspection video.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in any one of the above-mentioned inspection report generation methods when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the above-mentioned inspection report generation methods are implemented.

[0018] The embodiment of the present application provides a method, device, electronic device and storage medium for generating an inspection report. In the process of generating the inspection report, the collected inspection video is obtained, and then the inspection video is subjected to quality assessment processing to obtain the quality score of the inspection video, which is used to indicate the quality of the inspection video. Then, the inspection video is subjected to lesion detection and identification to obtain the lesions and lesion categories contained in the inspection video, and relevant supplementary information is obtained in the inspection video according to the lesion category. Finally, after obtaining the currently used report template, the report template is supplemented and improved according to the supplementary information and the quality score to generate the inspection report corresponding to the inspection video. In the process of disease inspection, the inspection video collected by the capsule endoscope device is automatically detected and analyzed to determine the relevant information of the lesions collected by the inspection video, thereby improving the accuracy and timeliness of lesion identification detection. At the same time, the inspection video is subjected to quality assessment to obtain the quality score, and the inspection report is subjected to reliability judgment. Finally, the corresponding report template set in advance is used to improve and supplement the information, and the corresponding inspection report is automatically generated, and the result of the reliability judgment is marked in the inspection report, thereby improving the efficiency and completeness of the automatic generation of the inspection report. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a method for generating an inspection report provided in an embodiment of the present application;

[0020] Figure 2 is a flowchart of the steps of obtaining a quality score provided in an embodiment of the present application;

[0021] Figure 3 is a flowchart of steps for obtaining an integrity score provided in an embodiment of the present application;

[0022] Figure 4 is a flowchart of the steps of obtaining supplementary information provided in an embodiment of the present application;

[0023] Figure 5 It is a report schematic diagram of the inspection report provided in the embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of a block diagram of the inspection report generation provided by an embodiment of the present application;

[0025] Figure 7 It is a structural schematic diagram of an inspection report generating device provided in an embodiment of the present application;

[0026] Figure 8 is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0027] Fig. 9 This is another structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] In related technologies, capsule enteroscopy reports are usually written manually after endoscopists have read the entire film. Since each capsule enteroscopy video usually takes 8-10 hours to examine, and after the examination, endoscopists usually need to spend 1-2 hours to read the film, keep the image manually, and write the report. In addition to being very time-consuming, this process is also extremely dependent on the endoscopist's own diagnosis and treatment experience and work habits, such as report writing habits, which leads to overly personalized report writing. At the same time, due to long-term viewing and processing of videos, lesions may be missed or misdiagnosed, which will have an adverse impact on the patient's subsequent treatment and follow-up.

[0032] In order to solve the technical problems existing in the related art, the present application embodiment provides a method for generating an inspection report, see Figure 1 , Figure 1 It is a flow chart of the inspection report generation method provided in an embodiment of the present application, and the method includes steps 101 to 104.

[0033] Step 101, obtain the collected inspection video, and perform quality assessment on the inspection video to obtain a quality score of the inspection video.

[0034] In one embodiment, during the disease examination process, the examination video collected by the endoscope is obtained, and then the quality of the examination video is evaluated and processed to obtain a quality score of the examination video. In practical applications, the quality score indicates the credibility of the examination report. The higher the quality score, the higher the credibility of the examination report generated, and vice versa.

[0035] For example, taking capsule enteroscopy as an example, during the inspection, a relevant image acquisition device is used to obtain a corresponding inspection video, such as a capsule endoscopy device, and then the inspection video is analyzed to determine the quality of the inspection video. Specifically, when the quality of the inspection video is evaluated to obtain a quality score, reference can be made to Figure 2 , Figure 2 2 is a flow chart of the steps for obtaining a quality score provided in an embodiment of the present application, wherein the steps include steps 201 to 205.

[0036] Step 201, decoding the inspection video to obtain an image set corresponding to the inspection video;

[0037] Step 202, performing integrity detection on the inspection video according to the image set to obtain an integrity score of the inspection video;

[0038] Step 203, performing a cleanliness test on each image in the image set to obtain a cleanliness score of the inspection video;

[0039] Step 204, performing a definition test on each image in the image set to obtain a definition score of the inspection video;

[0040] Step 205, obtaining a quality score of the inspection video according to the completeness score, the cleanliness score and the clarity score.

[0041] Specifically, when performing quality analysis on the inspection video, the inspection video is first decoded to obtain an image set corresponding to the inspection video, and then the quality score of the inspection video is completed by analyzing and processing the image set.

[0042] When obtaining the quality score of the inspection video based on the image set, the quality score is determined based on the integrity, cleanliness and clarity of the inspection image. Therefore, when performing quality assessment, the integrity of the inspection video is determined based on the image set, and at the same time, the cleanliness and clarity of each image in the image set are tested to obtain the cleanliness score and clarity score of each image, and then the cleanliness score and clarity score of the inspection video are determined. Finally, the quality score of the inspection video is determined based on the integrity score, cleanliness score and clarity score.

[0043] In actual processing, it can be determined based on whether the image contains images or features of the video inspection starting point and the video inspection end point, specifically based on the inspected object, etc. Taking the capsule small intestine as an example, when determining the integrity of the inspection video, the image set corresponding to the inspection video needs to include a pylorus image and / or a duodenum image, and also needs to include an ileocecal valve image and / or a colon image, wherein the pylorus image and the duodenum image correspond to the starting point of the inspection video, and the ileocecal valve image and the colon image correspond to the middle end point of the inspection video. Only when the images representing the starting point and the end point are simultaneously identified in the inspection video, is it determined that the inspection video is complete; otherwise, it is determined to be incomplete.

[0044] Therefore, when determining the integrity score of the inspection video, you can refer to Figure 3 , Figure 3 It is a flowchart of the steps of obtaining the integrity score provided in an embodiment of the present application, wherein the steps include steps 301 to 304.

[0045] Step 301, performing landmark detection on each image in the image set to determine the landmark type to which each image in the image set belongs, wherein the landmark type includes a starting point landmark, an intermediate landmark, and an end point landmark;

[0046] Step 302, determining the integrity score of the inspection video according to the landmark type;

[0047] Step 303: if the image set does not include an image whose landmark type is a starting point landmark or an end point landmark, determining that the integrity score of the inspection video is a first score value;

[0048] Step 304: If the image set includes images of landmark types of a starting point landmark and an end point landmark, then the integrity score of the inspection video is determined to be a second score value.

[0049] Exemplarily, when determining the integrity score of the inspection video based on the image set, a pre-trained model is used to perform landmark detection on each image in the image set to determine the landmark type to which each image in the image set belongs, and based on the integrity of the video, the landmark types are divided into starting landmarks, intermediate landmarks, and end landmarks. Obviously, only when the image set corresponding to the inspection video contains both images of the starting landmark and images of the end landmark, the inspection video is determined to be complete, otherwise it is determined that the inspection video is incomplete. Therefore, when determining the integrity score of the inspection video, an evaluation is performed based on the landmark types contained in the image set to obtain the integrity score of the inspection video.

[0050] In the actual detection and analysis process, in addition to ensuring that the image set contains both the starting point landmark and the end point landmark, it is also possible to limit the order of the image corresponding to the starting point landmark and the image corresponding to the end point landmark. For example, it is necessary to ensure that the image corresponding to the starting point landmark is in front of the image corresponding to the end point landmark in terms of timing. If the image corresponding to the starting point landmark is behind the image corresponding to the end point landmark, and there are no other starting point landmarks and end point landmarks, it can be determined that the inspection video is incomplete.

[0051] That is, when determining the integrity score of the inspection video, taking the example that the image set contains at most one starting point landmark and one ending point landmark, it can also include: performing landmark detection on each image in the image set, and determining the landmark type to which each image in the image set belongs; when it is determined that the image set contains a starting point landmark and an ending point landmark, obtaining a display image corresponding to the starting point landmark and a second image corresponding to the ending point landmark; determining the time sequence relationship between the display image and the second image in the inspection video, and obtaining the integrity score of the inspection video based on the time sequence relationship; wherein, when the display image is before the second image, the integrity score of the inspection video is determined to be a first score value, and when the display image is after the second image, the integrity score of the inspection video is determined to be a second score value.

[0052] In addition, when the image set may contain multiple starting landmarks and end landmarks, the integrity score of the inspection video is also determined based on the temporal relationship. At the same time, when there are multiple starting landmarks and end landmarks, the inspection video can be segmented to ensure that each inspection video contains at most one starting landmark and end landmark.

[0053] Furthermore, when the inspection video is processed to obtain the corresponding image set, the video can be decoded at 7 frames per second to obtain the image set corresponding to the inspection video after the decoding process is completed. Then, when performing quality assessment, the quality score of the inspection video is obtained by analyzing and detecting the image set.

[0054] At the same time, when performing quality scoring to analyze and process each image, binary classification processing can be performed. Specifically, when performing integrity assessment to obtain integrity scores, the integrity scores include 1 and 0, that is, the first score value mentioned above is 0, and the second score value is 1. Similarly, when scoring cleanliness and clarity, binary classification is also used for processing, that is, the cleanliness score is divided into 1 and 0, and the cleanliness score is divided into 1 and 0. Of course, in actual applications, multi-classification processing can also be performed based on actual needs. For example, for cleanliness and clarity, the score value is any value in [0, 1].

[0055] Exemplarily, when obtaining a cleanliness score of an inspection video based on an image set, the method includes: using a pre-trained first scoring model to perform a cleanliness score on each image in the image set to obtain a cleanliness score for each image; performing mean calculation based on the cleanliness score of each image, and using the obtained average cleanliness score as the cleanliness score of the inspection video.

[0056] Specifically, when performing cleanliness assessment processing, a pre-trained cleanliness grading model is used for processing. In the actual processing process, the evaluation results for cleanliness can be as above, including clean (score of 1) and unclean (score of 0), that is, the pre-trained cleanliness grading model is used to perform binary classification processing on the image to obtain the cleanliness score of each image, and then the cleanliness score of the entire image set is obtained by calculating the mean, that is, the cleanliness score of the inspection video is obtained.

[0057] Of course, when performing the scoring process, the cleanliness of the image can be graded more finely, that is, using a pre-selected and trained cleanliness grading model to predict a specific score, rather than performing a binary classification process to obtain a specific score between [0, 1].

[0058] In addition, when obtaining the clarity score of the inspection video based on the image set, it includes: using a pre-trained second scoring model to perform a clarity score on each image in the image set to obtain a clarity score for each image; calculating the mean based on the clarity score of each image, and using the obtained average clarity score as the clarity score of the inspection video.

[0059] Specifically, when the clarity score of each image in the image set corresponding to the inspection video is scored, the method used is similar to the method of obtaining the cleanliness score, the difference lies in the difference in the model used, that is, when obtaining the clarity score, the pre-trained clarity grading model is used for processing, and the other processes are the same. Similarly, when determining the clarity score of each image, in addition to using a binary classification method to obtain a score of 0 or 1, detailed processing can also be performed so that the score of each image is a specific value between [0, 1], such as 0.5, 0.7, etc.

[0060] Finally, when the quality score is obtained, the final quality score of the inspected video can be obtained by weighted addition of the completeness score, cleanliness score and clarity score. For example, the quality score S = f1S1+f2S2+f3S3, where f1, f2 and f3 are weight values, and the sum is 1, and S1, S2 and S3 represent the completeness score, cleanliness score and clarity score respectively.

[0061] Step 102, identifying and detecting lesions in the inspection video to obtain the lesions and lesion categories contained in the inspection video.

[0062] In one embodiment, after obtaining the collected inspection video, in order to generate an accurate and complete inspection report, it is necessary to determine the lesion information based on the inspection video, and by detecting and identifying the lesions in the inspection video, the lesions and lesion categories contained in the inspection video are obtained.

[0063] Exemplarily, when processing an inspection video to determine the lesion information of the lesions contained therein, the lesion information is obtained by detecting and identifying the lesions in the images in the image set corresponding to the inspection video, and each image in the image set is processed by utilizing a pre-trained related model to complete the acquisition of the lesion information.

[0064] Taking the capsule small intestine inspection described above as an example, the identifiable lesions can be divided into eight categories, including: erythema, inflammation, hemorrhage, vascular lesions, protrusions, parasites, diverticula, and normal. When determining the category of the lesion, the lesion must first be determined, and then the category of the lesion must be judged. Therefore, for each image in the image set, a pre-trained lesion detection model is used for processing to determine whether each image contains a lesion. Then, when it is determined that a lesion is contained, the lesion area is obtained, and then the lesion area is processed using the pre-trained lesion classification model to output the classification result of the lesion.

[0065] In fact, since there are images with the same lesions and classification results in the image set, and only clearer and cleaner images need to be output when the results are output, in order to facilitate the generation of subsequent reports, after obtaining the lesion information contained in the inspection video, the images can be screened according to the cleanliness score and clarity score used in the quality score to obtain the image for display. Specifically, when the specific score is any value in [0, 1], if there are multiple images belonging to the same classification result, the image with the highest sum of the cleanliness score and clarity score can be selected as the image for display.

[0066] Step 103, obtaining the currently used report template, and obtaining supplementary information according to the lesion information.

[0067] In one embodiment, after determining the lesion information of the lesion contained in the inspection video, in order to facilitate the generation of an appropriate and complete inspection report, it is necessary to obtain a report template suitable for the current scenario, and at the same time obtain supplementary information for supplementing the report template based on the detected lesion information.

[0068] For example, when determining the report template currently in use, it can be set according to the currently targeted site. The disease conditions in different organ sites are different, so at this time, the information of the currently targeted site can be obtained to obtain the corresponding report template.

[0069] At the same time, in order to improve the report template and obtain the final generated inspection report, the information content of the report template needs to be improved. In addition to including routine patient information, the inspection conditions that need to be reflected in the report also need to be accurately and completely displayed.

[0070] Furthermore, when obtaining additional information based on the lesion type, you can refer to Figure 4 , Figure 4 It is a flowchart of the steps of obtaining supplementary information provided in an embodiment of the present application, wherein the steps include steps 401 to 403.

[0071] Step 401, performing set division processing on the image set according to the lesion category to obtain a sub-image set corresponding to the lesion category, wherein one lesion category corresponds to one sub-image set;

[0072] Step 402, performing feature recognition on each image in the sub-image set to obtain lesion features corresponding to the lesion category;

[0073] Step 403 , screen images in the sub-image set to obtain display images associated with the lesion category, and use the lesion category, lesion features and display images as supplementary information.

[0074] Specifically, when the inspection report is improved, the content information that needs to be displayed is obtained from the inspection video to supplement the report template. For the supplementary information, it is usually necessary to include relevant information about the lesion, including the characteristics of the lesion and the image containing the lesion for display in the report. Therefore, when obtaining the supplementary information, the image set is divided into sets according to the lesion category to obtain a sub-image set corresponding to each lesion category, and then feature recognition is performed in the sub-image set to obtain the lesion features corresponding to each lesion category. Finally, when determining the image for display, the image is screened in the sub-image set, such as screening according to the cleanliness score and clarity score obtained above to obtain the display image.

[0075] It should be noted that the report template also needs to include the patient's basic information. The basic information entered before collecting the examination video can be obtained, and then the relevant information for entering into the report template can be obtained from the basic information, including but not limited to: name, gender, age, address, and treatment duration.

[0076] Exemplarily, for the acquired lesion features, different lesion categories correspond to different lesion features. For example, for lesions with erythema category, the lesion features include: the frequency of erythema (individual, scattered, multiple), for another example, for lesions with protrusion category, the lesion features include: size (less than 5*5mm, 5*5-10*10mm, 10*10-20*20mm, more than 20*20mm), surface structure (smooth surface, rough surface) and frequency (individual, scattered, multiple), etc., for another example, for lesions with hemorrhage category, the lesion features include: whether it is active bleeding. Of course, lesions do not necessarily include all categories, but when displaying in the report, it is necessary to fully display the required information according to the actual situation, such as displaying the images and features of all categories of lesions.

[0077] Step 104, supplement the report template according to the supplementary information and the quality score, and generate an inspection report corresponding to the inspection video.

[0078] In one embodiment, after the supplementary information in the report template is obtained, the report template will be supplemented according to the supplementary information and the quality score, and then an inspection report corresponding to the inspection video will be generated after the information is supplemented.

[0079] Specifically, when supplementing information to generate an inspection report, it includes: associating the lesion category with the display image, and entering the display image and the lesion category into the image display area of ​​the report template; obtaining the corresponding text description based on the lesion characteristics; entering the text description into the text description area of ​​the report template based on the area sequence of the display image in the image display area, and generating an inspection report corresponding to the inspection video when the entry is completed.

[0080] For example, when supplementing the information, it is necessary to match the displayed image with the lesion category one by one and display it in the corresponding area. At the same time, it is also necessary to enter the lesion features corresponding to the displayed lesion category or the displayed image in the corresponding area. For details, please refer to Figure 5 , Figure 5 It is a report diagram of the inspection report provided in the embodiment of the present application, such as Figure 5 As shown in the figure, when supplementing information, the information to be displayed is entered in the corresponding area, including the supplement of the displayed image and the supplement of the picture description. The picture description is the description of the lesion characteristics. By matching in the preset corpus, the text description corresponding to the lesion characteristics is obtained and entered in a specific position in the report template to generate the corresponding inspection report. The quality score of the inspection video will also be displayed in the inspection report (in Figure 5 It is not marked in the inspection report shown. In the actual generation process, it can be displayed in the inspection report in any form).

[0081] Further, refer to Figure 6 , Figure 6 It is a block diagram of the inspection report generation provided in the embodiment of the present application.

[0082] exist Figure 6 In the flowchart shown, taking the inspection video as capsule enteroscopy as an example, the generation process of the inspection report includes:

[0083] S1, preprocessing the inspection video, specifically: decoding the inspection video at 7 frames per second to obtain an image set corresponding to the inspection video;

[0084] S2, conducts quality assessment on the inspection video, specifically: utilizes the pre-trained quality control model to analyze and evaluate the quality of the image set to obtain a quality score, where the quality score includes a completeness score, a cleanliness score, and a clarity score. Through analysis and processing in three dimensions, the quality assessment result of the inspection video is obtained.

[0085] S3, identifying the lesion category of the inspection video, specifically: identifying the category of the lesion contained in each image in the image set to obtain the lesion category of each image, wherein the lesion categories include: erythema, inflammation, hemorrhage, vascular lesions, protrusions, parasites, diverticula and normal.

[0086] S4, identifying lesion features in the inspection video, specifically: obtaining lesion features corresponding to different lesion categories in the inspection video, wherein the lesion features are specific descriptions of the lesions, such as size, number, structure, and frequency.

[0087] S5, automatically collecting representative images, specifically: determining the display image corresponding to each lesion category in the image set, such as selecting according to the cleanliness score and / or clarity score of the image.

[0088] S6, automatic generation of the inspection report, specifically: obtaining the currently used report template, adding the quality score, lesion category, lesion characteristics and other supplementary information obtained in the above steps to the report template to generate a corresponding inspection report, wherein the generated inspection report can be as follows: Figure 6 shown.

[0089] In summary, the present application discloses a method for generating an inspection report. In the process of generating the inspection report, the collected inspection video is obtained, and then the inspection video is subjected to quality assessment processing to obtain a quality score of the inspection video, which is used to indicate the quality of the inspection video. Then, the inspection video is subjected to lesion detection and identification to obtain the lesions and lesion categories contained in the inspection video, and relevant supplementary information is obtained in the inspection video according to the lesion category. Finally, after obtaining the currently used report template, the report template is supplemented and improved according to the supplementary information and the quality score to generate an inspection report corresponding to the inspection video. In the process of disease inspection, the inspection video collected by the capsule endoscope device is automatically detected and analyzed to determine the relevant information of the lesions collected by the inspection video, thereby improving the accuracy and timeliness of lesion identification detection. At the same time, the inspection video is subjected to quality assessment to obtain a quality score, and the inspection report is subjected to reliability judgment. Finally, the corresponding report template set in advance is used to improve and supplement the information, and the corresponding inspection report is automatically generated, and the result of the reliability judgment is marked in the inspection report, thereby improving the efficiency and completeness of the automatic generation of the inspection report.

[0090] According to the method described in the above embodiment, this embodiment will be further described from the perspective of the inspection report generating device, which can be implemented as an independent entity or integrated in an electronic device, such as a terminal, which can include a mobile phone, a tablet computer, etc.

[0091] See also Figure 7 , Figure 7 is a structural diagram of an inspection report generating device provided in an embodiment of the present application, such as Figure 7 As shown, the inspection report generating device 700 provided in the embodiment of the present application includes:

[0092] The quality assessment module 701 is used to obtain the collected inspection video and perform quality assessment on the inspection video to obtain a quality score of the inspection video;

[0093] The lesion detection module 702 is used to identify and detect lesions in the inspection video to obtain the lesions and lesion categories contained in the inspection video;

[0094] The information acquisition module 703 is used to acquire the currently used report template and acquire the supplementary information of the report template according to the lesion category;

[0095] The report generation module 704 is used to supplement the report template according to the supplementary information and the quality score, and generate an inspection report corresponding to the inspection video.

[0096] In one embodiment, the quality assessment module 701 is further configured to:

[0097] Decoding the inspection video to obtain an image set corresponding to the inspection video;

[0098] Perform integrity detection on the inspection video according to the image set to obtain an integrity score of the inspection video;

[0099] Perform cleanliness detection on each image in the image set to obtain a cleanliness score of the inspection video;

[0100] Performing clarity detection on each image in the image set to obtain a clarity score of the inspection video;

[0101] The quality score of the inspected video is obtained based on the completeness score, cleanliness score and clarity score.

[0102] In one embodiment, the quality assessment module 701 is further configured to:

[0103] Performing landmark detection on each image in the image set to determine the landmark type to which each image in the image set belongs, wherein the landmark type includes a starting point landmark and an end point landmark;

[0104] Based on the landmark type, the completeness score of the inspection video is determined;

[0105] If the image set does not include an image whose landmark type is a starting point landmark or an end point landmark, determining that the integrity score of the inspection video is a first score value;

[0106] If the image set includes images of landmark types of a starting point landmark and an end point landmark, it is determined that the integrity score of the inspection video is the second score value.

[0107] In one embodiment, the quality assessment module 701 is further configured to:

[0108] Using the pre-trained first scoring model to score the cleanliness of each image in the image set, to obtain a cleanliness score for each image;

[0109] The cleanliness score of each image is averaged and the average cleanliness score is used as the cleanliness score of the inspection video.

[0110] In one embodiment, the quality assessment module 701 is further configured to:

[0111] Using a pre-trained second scoring model to score the clarity of each image in the image set, to obtain a clarity score for each image;

[0112] The average clarity score of each image is calculated, and the obtained average clarity score is used as the clarity score of the inspection video.

[0113] In one embodiment, the information acquisition module 703 is further used to:

[0114] Performing set division processing on the image set according to the lesion category to obtain a sub-image set corresponding to the lesion category, wherein one lesion category corresponds to one sub-image set;

[0115] Perform feature recognition on each image in the sub-image set to obtain lesion features corresponding to the lesion category;

[0116] Image screening is performed in the sub-image set to obtain display images associated with the lesion category, and the lesion category, lesion characteristics and display images are used as supplementary information.

[0117] In one embodiment, the report generation module 704 is further configured to:

[0118] Correspondingly associating the lesion category with the display image, and entering the display image and the lesion category into the image display area of ​​the report template;

[0119] Obtain corresponding text description according to the characteristics of the lesion;

[0120] According to the regional label sequence of the displayed image in the image display area, enter the text description in the text description area of ​​the report template, and generate an inspection report corresponding to the inspection video when the entry is completed.

[0121] Also, see Figure 8 , Figure 8is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, and the electronic device may be a mobile terminal such as a smart phone, a tablet computer, or the like. Figure 8 As shown, the electronic device 800 includes a processor 801 and a memory 802. The processor 801 is electrically connected to the memory 802.

[0122] The processor 801 is the control center of the electronic device 800. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device 800 and processes data by running or loading applications stored in the memory 802 and calling data stored in the memory 802, thereby monitoring the electronic device 800 as a whole.

[0123] In this embodiment, the processor 801 in the electronic device 800 will load the instructions corresponding to the processes of one or more applications into the memory 802 according to the following steps, and the processor 801 will run the application stored in the memory 802, thereby implementing any step in the inspection report generation method provided in the above embodiment.

[0124] The electronic device 800 can implement the steps in any embodiment of the inspection report generation method provided in the embodiments of the present application, and therefore can achieve the beneficial effects that can be achieved by any inspection report generation method provided in the embodiments of the present application. For details, please refer to the previous embodiments and will not be repeated here.

[0125] See also Fig. 9 , Fig. 9 is another structural schematic diagram of an electronic device provided in an embodiment of the present application, such as Fig. 9 As shown, Fig. 9 The specific structural block diagram of the electronic device provided in the embodiment of the present application is shown, and the electronic device can be used to implement the inspection report generation method provided in the above embodiment. The electronic device 900 can be a mobile terminal such as a smart phone or a laptop computer.

[0126] The RF circuit 910 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 910 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 910 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0127] The memory 920 can be used to store software programs and modules, such as the program instructions / modules corresponding to the inspection report generation method in the above embodiment. The processor 980 executes various functional applications and the inspection report generation method by running the software programs and modules stored in the memory 920.

[0128] The memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 920 may further include a memory remotely arranged relative to the processor 980, and these remote memories may be connected to the electronic device 900 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0129] The input unit 930 can be used to receive uploaded digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 930 may include a touch-sensitive surface 931 and other input devices 932. The touch-sensitive surface 931, also known as a touch display screen or touchpad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 931), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 931 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the touch-sensitive surface 931 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch-sensitive surface 931, the input unit 930 may further include other input devices 932. Specifically, the other input devices 932 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0130] The display unit 940 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device 900, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 940 may include a display panel 941, and optionally, the display panel 941 may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 931 may cover the display panel 941, and when the touch-sensitive surface 931 detects a touch operation on or near it, it is transmitted to the processor 980 to determine the type of the touch event, and then the processor 980 provides corresponding visual output on the display panel 941 according to the type of the touch event. Although in the figure, the touch-sensitive surface 931 and the display panel 941 are implemented as two independent components to implement input and output functions, in some embodiments, the touch-sensitive surface 931 and the display panel 941 can be integrated to implement input and output functions.

[0131] The electronic device 900 may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the proximity sensor may generate an interrupt when the flip cover is closed or closed. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured in the electronic device 900, they will not be repeated here.

[0132] The audio circuit 960, the speaker 961, and the microphone 962 can provide an audio interface between the user and the electronic device 900. The audio circuit 960 can transmit the electrical signal converted from the received audio data to the speaker 961, which is converted into a sound signal for output; on the other hand, the microphone 962 converts the collected sound signal into an electrical signal, which is received by the audio circuit 960 and converted into audio data, and then the audio data is output to the processor 980 for processing, and then sent to another terminal through the RF circuit 910, or the audio data is output to the memory 920 for further processing. The audio circuit 960 may also include an earplug jack to provide communication between an external headset and the electronic device 900.

[0133] The electronic device 900 can help users receive requests, send information, etc. through the transmission module 970 (such as a Wi-Fi module), which provides users with wireless broadband Internet access. Although the transmission module 970 is shown in the figure, it is understandable that it is not a necessary component of the electronic device 900 and can be omitted as needed without changing the essence of the invention.

[0134] The processor 980 is the control center of the electronic device 900. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 920, and calling data stored in the memory 920, it executes various functions of the electronic device 900 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 980 may include one or more processing cores; in some embodiments, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 980.

[0135] The electronic device 900 also includes a power supply 990 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 980 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 990 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0136] Although not shown, the electronic device 900 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors to implement any step in the inspection report generation method provided in the above embodiment.

[0137] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0138] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present application provides a storage medium, in which a plurality of instructions are stored, and when the instructions are executed by the processor, any step in the inspection report generation method provided in the above embodiments can be implemented.

[0139] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0140] Since the instructions stored in the storage medium can execute the steps in any embodiment of the inspection report generation method provided in the embodiments of the present application, the beneficial effects that can be achieved by any inspection report generation method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0141] The above is a detailed introduction to an inspection report generation method, device, electronic device and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application. Moreover, for those of ordinary skill in the art, without departing from the principles of the present application, several improvements and modifications can be made, and these improvements and modifications are also regarded as the scope of protection of the present application.

Claims

1. A method for generating an inspection report, characterized in that: include: Acquire the collected inspection video, and perform quality assessment on the inspection video to obtain a quality score of the inspection video; Identify and detect lesions in the inspection video to obtain lesions and lesion categories contained in the inspection video; Acquire the currently used report template, and acquire supplementary information of the report template according to the lesion category; The report template is supplemented with information according to the supplementary information and the quality score to generate an inspection report corresponding to the inspection video.

2. The method according to claim 1, characterized in that The performing quality assessment on the inspection video to obtain a quality score of the inspection video includes: Decoding the inspection video to obtain an image set corresponding to the inspection video; Performing integrity detection on the inspection video according to the image set to obtain an integrity score of the inspection video; Performing a cleanliness test on each image in the image set to obtain a cleanliness score of the inspection video; Performing a clarity test on each image in the image set to obtain a clarity score of the inspection video; A quality score of the inspection video is obtained according to the completeness score, the cleanliness score and the clarity score.

3. The method according to claim 2, characterized in that The performing integrity detection on the inspection video according to the image set to obtain an integrity score of the inspection video includes: Performing landmark detection on each image in the image set to determine the landmark type to which each image in the image set belongs, wherein the landmark type includes a starting point landmark and an end point landmark; Determining a completeness score of the inspection video according to the landmark type; If the image set does not include an image whose landmark type is a starting point landmark or an end point landmark, determining that the integrity score of the inspection video is a first score value; If the image set includes images of landmark types of a starting point landmark and an end point landmark, the integrity score of the inspection video is determined to be a second score value.

4. The method according to claim 2, characterized in that The performing cleanliness detection on each image in the image set to obtain a cleanliness score of the inspection video includes: Using a pre-trained first scoring model to score the cleanliness of each image in the image set, to obtain a cleanliness score for each image; The average cleanliness score of each image is calculated, and the obtained average cleanliness score is used as the cleanliness score of the inspection video.

5. The method according to claim 2, characterized in that The performing clarity detection on each image in the image set to obtain a clarity score of the inspection video includes: Using a pre-trained second scoring model to perform a clarity score on each image in the image set to obtain a clarity score for each image; An average is calculated based on the clarity score of each image, and the obtained average clarity score is used as the clarity score of the inspection video.

6. The method according to claim 1, characterized in that The step of obtaining the supplementary information of the report template according to the lesion category includes: Performing set division processing on the image set according to the lesion category to obtain a sub-image set corresponding to the lesion category, wherein one lesion category corresponds to one sub-image set; Performing feature recognition on each image in the sub-image set to obtain lesion features corresponding to the lesion category; Image screening is performed in the sub-image set to obtain a display image associated with the lesion category, and the lesion category, the lesion characteristics and the display image are used as supplementary information.

7. The method according to claim 6, characterized in that The step of supplementing the report template according to the supplementary information and the quality score to generate an inspection report corresponding to the inspection video includes: Correspondingly associating the lesion category with the display image, and entering the display image and the lesion category into the image display area of ​​the report template; Obtaining a corresponding text description according to the lesion characteristics; According to the area sequence of the display image in the image display area, the text description is entered in the text description area of ​​the report template, and an inspection report corresponding to the inspection video is generated when the entry is completed.

8. An inspection report generating device, characterized in that: include: A quality assessment module is used to obtain the collected inspection video and perform quality assessment on the inspection video to obtain a quality score of the inspection video; A lesion detection module, used to identify and detect lesions in the inspection video, and obtain the lesions and lesion categories contained in the inspection video; An information acquisition module, used to acquire the currently used report template and acquire supplementary information of the report template according to the lesion category; A report generation module is used to supplement the report template according to the supplementary information and the quality score, and generate an inspection report corresponding to the inspection video.

9. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Multi-modal tongue image analysis system

    CN120472504A

  • A multimodal tongue image analysis system

    CN120472504B