Fetal heart monitoring data entry method and device, computer equipment and storage medium
By using image processing technology to identify and store fetal heart rate monitoring data, the problem of low efficiency in fetal heart rate monitoring data management has been solved. This has enabled digital data entry and automatic interpretation by artificial intelligence, thereby improving the accuracy of data management and interpretation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2023-06-30
- Publication Date
- 2026-05-29
AI Technical Summary
Existing methods for managing fetal heart rate monitoring data suffer from low efficiency and an inability to effectively digitize and input data into information systems, hindering the application of artificial intelligence-based automatic interpretation systems for fetal heart rate monitoring curves.
Image processing technology is used to identify scale data and table lines in fetal heart rate monitoring photos through text detection and recognition models and table line detection models. Combined with fetal heart rate curve detection models, fetal heart rate monitoring curve coordinate data is obtained and stored in the database. The U-Net network architecture is used to improve image segmentation efficiency, correct photo rotation and deformation, and enhance model performance.
It enables the digital entry of fetal heart rate monitoring data, reduces errors caused by individual differences in human interpretation, supports subsequent medical research and the application of artificial intelligence automatic interpretation systems, and improves data management efficiency.
Smart Images
Figure CN116824610B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology in artificial intelligence, and more particularly to a method, apparatus, computer device, and storage medium for recording fetal heart rate monitoring data. Background Technology
[0002] Fetal heart rate monitoring, short for Cardiotocography (CTG), is a clinical monitoring method that acquires real-time fetal heart rate (FHR) and maternal uterine contraction (UC) signals. It can identify fetal hypoxia and acidosis in utero and accurately predict perinatal outcomes. Currently, fetal heart rate monitoring has become a routine test in most hospitals during labor. Normal FHR monitoring results can alleviate concerns for both patients and doctors, while abnormal FHR can prompt early intervention, reducing the risk of adverse fetal / neonatal outcomes. For example, the American College of Obstetricians and Gynecologists recommends continuous monitoring during labor in high-risk pregnancies, and the UK National Institute for Health and Care Evaluation recommends continuous fetal heart rate monitoring for abnormalities such as infection and bleeding during labor.
[0003] Currently, most fetal heart rate monitors used in clinical settings print fetal heart rate monitoring results onto monitoring paper tapes, which cannot be directly output to hospital information systems. While using monitoring paper tapes makes it convenient for medical staff to collect and interpret fetal heart rate monitoring data in clinical practice, the lack of digital entry of these results into the information system hinders subsequent research and analysis, and impedes the application of automated fetal heart rate monitoring curve interpretation systems developed based on artificial intelligence and other technologies. Therefore, the existing fetal heart rate monitoring data management method suffers from low management efficiency. Summary of the Invention
[0004] The purpose of this application is to provide a method, device, computer equipment, and storage medium for recording fetal heart rate monitoring data, so as to solve the problem of low management efficiency in existing fetal heart rate monitoring data management methods.
[0005] To address the aforementioned technical problems, this application provides a method for recording fetal heart rate monitoring data, employing the following technical solution:
[0006] Obtain fetal heart rate monitoring photos of the users to be entered;
[0007] The fetal heart rate monitoring photos are input into a text detection and recognition model for scale recognition to obtain scale data.
[0008] The fetal heart rate monitoring image is input into the table line detection model for table line recognition to obtain a binary image of the scale line, wherein the binary image of the scale line includes vertical scale lines and horizontal scale lines.
[0009] By combining the scale data, the vertical scale line, and the horizontal scale line, the scale line coordinates are obtained, wherein the scale line coordinates include the vertical scale line coordinates and the horizontal scale line coordinates;
[0010] The fetal heart rate monitoring photos are input into the fetal heart rate curve detection model to perform fetal heart rate curve recognition operation, and a binary image of the fetal heart rate curve is obtained.
[0011] By combining the coordinates of the vertical scale line, the coordinates of the horizontal scale line, and the binary graph of the fetal heart rate curve, the coordinate data of the fetal heart rate monitoring curve is obtained.
[0012] The fetal heart rate monitoring curve coordinate data is stored in the database corresponding to the user to be entered.
[0013] Furthermore, after the step of obtaining the fetal heart rate monitoring photo of the user to be entered, and before the step of inputting the fetal heart rate monitoring photo into the text detection and recognition model for scale recognition operation to obtain scale data, the following steps are also included:
[0014] Obtain the four corner markers of the fetal heart rate monitoring image;
[0015] The minimum bounding rectangle is obtained by performing bounding rectangle calculation on the four corner point markers using the OpenCV minAreaRect function.
[0016] The rotation transformation matrix is obtained by performing a rotation transformation calculation on the minimum bounding rectangle according to the matrix correction algorithm.
[0017] The fetal heart rate monitoring images are subjected to matrix correction processing based on the rotation transformation matrix.
[0018] Furthermore, after the step of obtaining the fetal heart rate monitoring photo of the user to be entered, and before the step of inputting the fetal heart rate monitoring photo into the text detection and recognition model for scale recognition operation to obtain scale data, the following steps are also included:
[0019] The fetal heart rate monitoring images were converted to grayscale.
[0020] Furthermore, after the step of obtaining the fetal heart rate monitoring photo of the user to be entered, and before the step of inputting the fetal heart rate monitoring photo into the text detection and recognition model for scale recognition operation to obtain scale data, the following steps are also included:
[0021] The fetal heart rate monitoring photos are input into an image segmentation model for segmentation to obtain segmented fetal heart rate monitoring images.
[0022] The step of inputting the fetal heart rate monitoring image into a text detection and recognition model for scale recognition to obtain scale data specifically includes the following steps:
[0023] The fetal heart rate monitoring segmented image is input into the text detection and recognition model for scale recognition to obtain the scale data.
[0024] Furthermore, the image segmentation model uses U-Net as its network architecture.
[0025] Furthermore, before the step of inputting the fetal heart rate monitoring image into the image segmentation model for segmentation to obtain the segmented fetal heart rate monitoring image, the method further includes:
[0026] Construct the downsampling layer of the U-Net;
[0027] Construct the upsampling layer of the U-Net;
[0028] The downsampling layer and the upsampling layer are connected via Skip Connection;
[0029] Construct the Dropout layer;
[0030] Construct the network output module of the U-Net;
[0031] Configure the network parameters of the U-Net.
[0032] To address the aforementioned technical problems, this application also provides a fetal heart rate monitoring data entry device, which employs the following technical solution:
[0033] The photo acquisition module is used to acquire fetal heart rate monitoring photos of the user to be entered.
[0034] The scale recognition module is used to input the fetal heart rate monitoring photos into the text detection and recognition model to perform scale recognition operations and obtain scale data.
[0035] The table line recognition module is used to input the fetal heart rate monitoring photo into the table line detection model to perform table line recognition operation and obtain a binary image of the scale line, wherein the binary image of the scale line includes vertical scale lines and horizontal scale lines.
[0036] The first combining module is used to combine the scale data, the vertical scale line, and the horizontal scale line to obtain the scale line coordinates, wherein the scale line coordinates include the vertical scale line coordinates and the horizontal scale line coordinates.
[0037] The fetal heart rate curve recognition module is used to input the fetal heart rate monitoring photos into the fetal heart rate curve detection model to perform fetal heart rate curve recognition operations and obtain a binary image of the fetal heart rate curve.
[0038] The second combining module is used to combine the coordinates of the vertical scale line, the coordinates of the horizontal scale line, and the binary graph of the fetal heart rate curve to obtain the coordinate data of the fetal heart rate monitoring curve.
[0039] The data entry module is used to store the fetal heart rate monitoring curve coordinate data into a database corresponding to the user to be entered.
[0040] Furthermore, the device also includes:
[0041] The corner marker acquisition module is used to acquire the four corner markers of the fetal heart rate monitoring image;
[0042] The bounding rectangle calculation module is used to perform bounding rectangle calculation operations on the four corner point markers according to the OpenCV minAreaRect function to obtain the minimum bounding rectangle.
[0043] The rotation transformation calculation module is used to perform rotation transformation calculation operations on the minimum bounding rectangle according to the matrix correction algorithm to obtain the rotation transformation matrix.
[0044] The matrix correction module is used to perform matrix correction processing on the fetal heart rate monitoring images according to the rotation transformation matrix.
[0045] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:
[0046] The device includes a memory and a processor. The memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the fetal heart rate monitoring data entry method as described above.
[0047] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:
[0048] The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the fetal heart rate monitoring data entry method as described above.
[0049] This application provides a method for inputting fetal heart rate monitoring data, comprising: acquiring a fetal heart rate monitoring photo of a user to be input; inputting the fetal heart rate monitoring photo into a text detection and recognition model for ruler recognition to obtain ruler data; inputting the fetal heart rate monitoring photo into a table line detection model for table line recognition to obtain a binary graph of ruler lines, wherein the binary graph of ruler lines includes vertical ruler lines and horizontal ruler lines; combining the ruler data, the vertical ruler lines, and the horizontal ruler lines to obtain ruler line coordinates, wherein the ruler line coordinates include vertical ruler line coordinates and horizontal ruler line coordinates; inputting the fetal heart rate monitoring photo into a fetal heart rate curve detection model for fetal heart rate curve recognition to obtain a binary graph of fetal heart rate curves; combining the vertical ruler line coordinates, the horizontal ruler line coordinates, and the binary graph of fetal heart rate curves to obtain fetal heart rate monitoring curve coordinate data; and storing the fetal heart rate monitoring curve coordinate data in a database corresponding to the user to be input. Compared with existing technologies, this application can input a large amount of paper-based fetal heart rate monitoring data generated in clinical practice into an information system. This can provide data for subsequent medical research and enable some automatic fetal heart rate monitoring curve interpretation systems developed based on artificial intelligence technology to be used in prenatal care, reducing errors caused by individual differences in human interpretation. In addition, this application uses a table line detection model and a fetal heart rate curve detection model to detect table lines and fetal heart rate curves. The performance of the models can be improved by increasing the amount of training data, avoiding the instability caused by the traditional OpenCV approach. Attached Figure Description
[0050] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;
[0052] Figure 2 This is a flowchart illustrating the implementation of the fetal heart rate monitoring data entry method provided in Embodiment 1 of this application;
[0053] Figure 3 This is a flowchart illustrating a specific implementation of the corrected fetal heart rate monitoring photograph provided in Embodiment 1 of this application;
[0054] Figure 4 This is a flowchart illustrating a specific implementation of image segmentation provided in Embodiment 1 of this application;
[0055] Figure 5This is a structural diagram of a specific implementation of the U-Net network architecture provided in Embodiment 1 of this application;
[0056] Figure 6 This is a flowchart illustrating the implementation of the U-Net network construction method provided in Embodiment 1 of this application;
[0057] Figure 7 This is a schematic diagram of the fetal heart rate monitoring data entry device provided in Embodiment 2 of this application;
[0058] Figure 8 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation
[0059] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.
[0060] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0061] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0062] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.
[0063] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.
[0064] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.
[0065] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.
[0066] It should be noted that the fetal heart rate monitoring data entry method provided in this application embodiment is generally executed by a server / terminal device, and correspondingly, the fetal heart rate monitoring data entry device is generally set in the server / terminal device.
[0067] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.
[0068] Continue to refer to Figure 2 The diagram shows a flowchart of an embodiment of the fetal heart rate monitoring data entry method according to this application. The fetal heart rate monitoring data entry method includes steps S201, S202, S203, S204, S205, S206, and S207.
[0069] In step S201, fetal heart rate monitoring photos of the user to be entered are obtained.
[0070] In this embodiment, when paper-based fetal heart rate monitoring data needs to be entered, the user can take a photo of the paper-based fetal heart rate monitoring data using a mobile phone or scanner. The photo is then entered into the data entry system via a mobile app or client program for further processing. Only a clear photo that completely includes the fetal heart rate monitoring curve is required. Note that compared to scanner scans, images taken with a mobile phone may have some distortion, but our subsequent process involves first locating the table coordinates and then calculating the fetal heart rate monitoring data point by point, so it is largely unaffected by distortion.
[0071] In step S202, the fetal heart rate monitoring photos are input into the text detection and recognition model for scale recognition to obtain scale data.
[0072] In this embodiment, an input text detection and recognition model is used to detect and recognize the scale data of fetal heart rate monitoring. Common fetal monitoring papers include a fetal heart rate scale (unit: bmp) and a uterine contraction pressure scale (unit: mmHg). By vertically grouping the detected scale data boxes and comparing them with predefined common scales, the current fetal heart rate monitoring scale can be obtained. Specifically, if a certain detected scale data is grouped as follows... , The predefined scales are arranged from top to bottom. , Sort the fetal heart rate scales from top to bottom. For example, common fetal heart rate scales on the market (240, 210, 180, 150, 120, 90, 60, 30). Compare the common numbers between M and each scale, and select the scale with the most common numbers as the current fetal heart rate monitoring scale.
[0073] In step S203, the fetal heart rate monitoring image is input into the table line detection model for table line recognition to obtain a binary scale line image, which includes vertical scale lines and horizontal scale lines.
[0074] In this embodiment, a table line detection model is input to identify table lines. The model outputs a binary image of the same size as the original image. In this image, the pixel containing the table line has a value of 1, and the other pixels have a value of 0. After obtaining the binary image of the table lines, post-processing is performed using morphological operators to obtain the vertical and horizontal lines of the original image. Specifically, this is achieved by calculating the kernel...
[0075] ,
[0076] The etching process is used to obtain vertical and horizontal lines. The pixels on the vertical and horizontal lines are grouped according to their connectivity, corresponding to the vertical and horizontal scale lines on the fetal heart rate monitoring paper, respectively.
[0077] In step S204, the ruler line coordinates are obtained by combining the ruler data, the vertical ruler line, and the horizontal ruler line. The ruler line coordinates include the vertical ruler line coordinates and the horizontal ruler line coordinates.
[0078] In this embodiment of the application, after obtaining the vertical and horizontal scale lines of the fetal heart rate monitoring image, the coordinates of the scale lines (corresponding fetal heart rate / uterine contraction pressure and time) are obtained by combining the scale data obtained in step S202.
[0079] In step S205, the fetal heart rate monitoring image is input into the fetal heart rate curve detection model to perform fetal heart rate curve recognition operation, and a binary image of the fetal heart rate curve is obtained.
[0080] In this embodiment, the fetal heart rate curve detection model is input to identify the fetal heart rate monitoring curve. The fetal heart rate curve detection model outputs a binary image of the same size as the original image. In this image, the pixel where the fetal heart rate monitoring curve is located has a value of 1, and the other pixels have a value of 0. After obtaining the segmented image of the fetal heart rate monitoring curve, post-processing is performed to remove noise points. Specifically, the sum of the values of the eight pixels surrounding each pixel of the fetal heart rate monitoring curve is calculated. If the sum is less than a given threshold, the point is considered a noise point.
[0081] In step S206, the coordinates of the vertical scale line, the coordinates of the horizontal scale line, and the binary graph of the fetal heart rate curve are combined to obtain the coordinate data of the fetal heart rate monitoring curve.
[0082] In this embodiment of the application, the coordinates of each fetal heart rate monitoring curve point are calculated by combining the scale line coordinates and the fetal heart rate monitoring curve coordinate data. Specifically, for each pixel point on the fetal heart rate monitoring curve ( Search for the nearest upper horizontal line in the ruler horizontal line pixels. ,in The position of a pixel in the image. For the corresponding ruler value; similarly, search for the nearest upper horizontal line in the vertical ruler pixels. ,in The position of a pixel in the image. The corresponding scale value is shown. The coordinates of this fetal heart rate monitoring point are:
[0083]
[0084]
[0085] In step S207, the fetal heart rate monitoring curve coordinate data is stored in the database corresponding to the user to be entered.
[0086] In this embodiment, medical staff can input the corresponding patient ID, monitoring time, etc., as prompted on the page, and then input the results into the hospital information system or store them independently using the hospital information system interface.
[0087] This application provides a method for inputting fetal heart rate monitoring data, including: acquiring a fetal heart rate monitoring photo of the user to be input; inputting the fetal heart rate monitoring photo into a text detection and recognition model for ruler recognition to obtain ruler data; inputting the fetal heart rate monitoring photo into a table line detection model for table line recognition to obtain a binary graph of ruler lines, wherein the binary graph of ruler lines includes vertical ruler lines and horizontal ruler lines; combining the ruler data, vertical ruler lines, and horizontal ruler lines to obtain ruler line coordinates, wherein the ruler line coordinates include vertical ruler line coordinates and horizontal ruler line coordinates; inputting the fetal heart rate monitoring photo into a fetal heart rate curve detection model for fetal heart rate curve recognition to obtain a binary graph of fetal heart rate curves; combining the vertical ruler line coordinates, horizontal ruler line coordinates, and the binary graph of fetal heart rate curves to obtain fetal heart rate monitoring curve coordinate data; and storing the fetal heart rate monitoring curve coordinate data in a database corresponding to the user to be input. Compared with existing technologies, this application can input a large amount of paper-based fetal heart rate monitoring data generated in clinical practice into an information system. This can provide data for subsequent medical research and enable some automatic fetal heart rate monitoring curve interpretation systems developed based on artificial intelligence technology to be used in prenatal care, reducing errors caused by individual differences in human interpretation. In addition, this application uses a table line detection model and a fetal heart rate curve detection model to detect table lines and fetal heart rate curves. The performance of the models can be improved by increasing the amount of training data, avoiding the instability caused by the traditional OpenCV approach.
[0088] Continue reading Figure 3 The flowchart shows a specific implementation of the corrected fetal heart rate monitoring photograph provided in Embodiment 1 of this application. For ease of explanation, only the parts related to this application are shown.
[0089] In some optional implementations of this embodiment, after step S201 and before step S202, the steps S301, S302, S303 and S304 are also included.
[0090] In step S301, the four corner markers of the fetal heart rate monitoring image are obtained.
[0091] In step S302, the bounding rectangle of the four corner points is calculated using the OpenCV minAreaRect function to obtain the minimum bounding rectangle.
[0092] In step S303, the rotation transformation calculation operation is performed on the minimum bounding rectangle according to the matrix correction algorithm to obtain the rotation transformation matrix.
[0093] In step S304, the fetal heart rate monitoring images are subjected to matrix correction processing based on the rotation transformation matrix.
[0094] In this embodiment, since the image was taken under natural conditions, it may capture some background content and may be rotated, such as by 90 or 180 degrees. Therefore, the fetal heart rate monitoring image needs to be corrected. This application detects the region R of the fetal heart rate monitoring data in the image and the rotation angle θ using a detection model. Specifically, the fetal heart rate monitoring data region is marked by the four corner points of the region. The minimum bounding rectangle of the region is calculated using the OpenCV minAreaRect function: center and width and height Obtain the rotation transformation matrix. Rotate the fetal heart rate monitoring data area to the correct position, and... Extract a portion of size centered at the specified size. The fetal heart rate monitoring area was used as data for subsequent analysis.
[0095] In some optional implementations of this embodiment, after step S201 and before step S202, the following steps are further included:
[0096] The fetal heart rate monitoring images were converted to grayscale.
[0097] In this embodiment of the application, in the RGB model, if R=G=B, then color represents a grayscale color, where the value of R=G=B is called the grayscale value. Therefore, each pixel of a grayscale image only needs one byte to store the grayscale value (also known as the intensity value or brightness value), and the grayscale range is 0-255.
[0098] Specifically, the formula for grayscale conversion is:
[0099]
[0100] Note the powers of 2.2 and the root of 2.2. RGB color values cannot be simply added together; they must be converted to physical light power using the power of 2.2. This is because the relationship between RGB values and power is not a simple linear one, but rather a power function. The exponent of this function is called the Gamma value, typically 2.2, and this conversion process is called Gamma correction.
[0101] In this embodiment of the application, by converting the fetal heart rate monitoring photos to grayscale, the inference speed of the entire recognition process is made faster.
[0102] Continue reading Figure 4 The diagram shows a flowchart of a specific implementation of image segmentation provided in Embodiment 1 of this application. For ease of explanation, only the parts related to this application are shown.
[0103] In some optional implementations of this embodiment, after step S201 and before step S202, the method further includes step S401, and step S202 includes step S402.
[0104] In step S401, the fetal heart rate monitoring photo is input into the image segmentation model for segmentation to obtain the segmented fetal heart rate monitoring image;
[0105] In step S402, the fetal heart rate monitoring segmented image is input into the text detection and recognition model for scale recognition operation to obtain scale data.
[0106] In some optional implementations of this embodiment, the image segmentation model described above uses U-Net as the network architecture.
[0107] In this embodiment of the application, the U-Net network architecture is as follows: Figure 5 As shown, the U-Net network architecture is simpler than the traditional FCN network. The first half, the left side of the diagram, is for feature extraction, while the second half, the right side, is for upsampling. U-Net employs a completely different feature fusion method: concatenation. U-Net concatenates features along the channel dimension to form a thicker feature set. In contrast, FCN uses point-to-point addition for fusion, which does not create a thicker feature set.
[0108] In this embodiment, U-Net is built on the FCN network architecture, with modifications and expansions to enable accurate segmentation results using very few training images. An upsampling stage is added, along with numerous feature channels, allowing more original image texture information to propagate through high-resolution layers. U-Net lacks fully connected (FC) layers and uses valid convolutions throughout, ensuring that the segmentation results are based on contextual features without missing information.
[0109] In this embodiment, since the proportion of the target object to be segmented is increased, the segmentation task is simpler than before. After adopting the U-Net network architecture, the image segmentation model can achieve faster segmentation efficiency.
[0110] Continue reading Figure 6 The diagram shows the implementation flowchart of the U-Net network construction method provided in Embodiment 1 of this application. For ease of explanation, only the parts related to this application are shown.
[0111] In some optional implementations of this embodiment, before step S501, the steps S601, S602, S603, S604, S605, and S606 are also included.
[0112] In step S601, the downsampling layer of U-Net is constructed.
[0113] In this embodiment of the application, the downsampling layer consists of multiple (e.g., 4) convolutional modules, each of which consists of two 3*3 convolutional layers, one ReLU layer, and one 2*2 max pooling layer.
[0114] In step S602, the upsampling layer of U-Net is constructed.
[0115] In this embodiment of the application, the upsampling layer consists of multiple (e.g., 4) deconvolution modules, each of which consists of a 2*2 up-convolution layer, two 3*3 convolution layers, and a ReLU.
[0116] In step S603, the downsampling layer and the upsampling layer are connected based on Skip Connection.
[0117] In this embodiment, the feature layer output by the convolution module in the downsampling layer is connected to the deconvolution module of the corresponding upsampling layer, and is connected in series with the input of the deconvolution module of the previous layer, and together they are used as input features.
[0118] In step S604, the Dropout layer is built.
[0119] In this embodiment of the application, a Dropout layer is added to the downsampling layer to avoid overfitting during network training.
[0120] In step S605, the network output module of U-Net is built.
[0121] In this embodiment, the output of the upsampling layer is passed through a 2x2 and a 1x1 convolutional layer to obtain the final output of U-Net. This ensures that U-Net can perform end-to-end pixel segmentation, meaning that the input image is the same size as the output image.
[0122] In step S606, the network parameters of U-Net are set.
[0123] In this embodiment, the network parameters of U-Net include the number of convolutional and deconvolutional modules, the optimizer, the loss function, the activation function, and Dropout. In this embodiment, the ISAR image dataset is 128*128 pixels. Therefore, the downsampling layer of the U-Net network is set to 5 convolutional modules, with the feature dimensions after convolution processing set to 16-32-64-128-256 respectively. The corresponding upsampling layer consists of 5 deconvolutional modules, with the output feature dimensions set to 128-64-32-16 respectively, forming a symmetrical structure. Since the output layer outputs an image, the activation function for all layers except the output layer (which uses the Sigmoid function) is the ReLU function. The optimizer used is the Adam optimizer, which combines the advantages of AdaGrad and RMSProp optimization algorithms, requiring less memory and being more computationally efficient. To prevent overfitting during training, the Dropout layer is set to a 50% dropout ratio, meaning that the Dropout layer will randomly disconnect 50% of the input neurons each time parameters are updated during training. Finally, the binary cross-entropy function was chosen as the loss function for the network.
[0124] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned fetal heart rate monitoring curve coordinate data, the aforementioned fetal heart rate monitoring curve coordinate data can also be stored in a blockchain node.
[0125] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.
[0126] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0127] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.
[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).
[0129] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0130] Example 2
[0131] Further reference Figure 7 As a response to the above Figure 2 The present application provides an embodiment of a fetal heart rate monitoring data entry device, which is similar to the method described above. Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0132] like Figure 7 As shown, the fetal heart rate monitoring data entry device 200 of this embodiment includes: a photo acquisition module 210, a ruler recognition module 220, a table line recognition module 230, a first combination module 240, a fetal heart rate curve recognition module 250, a second combination module 260, and a data entry module 270. Wherein:
[0133] Photo acquisition module 210 is used to acquire fetal heart rate monitoring photos of the user to be entered;
[0134] The scale recognition module 220 is used to input fetal heart rate monitoring photos into the text detection and recognition model to perform scale recognition operations and obtain scale data.
[0135] The table line recognition module 230 is used to input fetal heart monitoring photos into the table line detection model to perform table line recognition operations and obtain a binary image of the scale line, wherein the binary image of the scale line includes vertical scale lines and horizontal scale lines.
[0136] The first combining module 240 is used to combine the scale data, the vertical scale line and the horizontal scale line to obtain the scale line coordinates, wherein the scale line coordinates include the vertical scale line coordinates and the horizontal scale line coordinates.
[0137] The fetal heart rate curve recognition module 250 is used to input fetal heart rate monitoring photos into the fetal heart rate curve detection model to perform fetal heart rate curve recognition operations and obtain a binary image of the fetal heart rate curve.
[0138] The second combining module 260 is used to combine the coordinates of the vertical scale line, the coordinates of the horizontal scale line, and the binary graph of the fetal heart rate curve to obtain the coordinate data of the fetal heart rate monitoring curve.
[0139] The data entry module 270 is used to store the fetal heart rate monitoring curve coordinate data into the database corresponding to the user to be entered.
[0140] In this embodiment, a fetal heart rate monitoring data entry device 200 is provided, including: a photo acquisition module 210 for acquiring fetal heart rate monitoring photos of the user to be entered; a scale recognition module 220 for inputting the fetal heart rate monitoring photos into a text detection and recognition model to perform scale recognition operations and obtain scale data; a table line recognition module 230 for inputting the fetal heart rate monitoring photos into a table line detection model to perform table line recognition operations and obtain a binary image of scale lines, wherein the binary image of scale lines includes vertical scale lines and horizontal scale lines; and a first combination module 240 for combining scale data. The system obtains scale line coordinates from the vertical and horizontal scale lines. The scale line coordinates include both vertical and horizontal scale line coordinates. The fetal heart rate curve recognition module 250 is used to input fetal heart rate monitoring photos into the fetal heart rate curve detection model for fetal heart rate curve recognition, resulting in a binary image of the fetal heart rate curve. The second combination module 260 is used to combine the vertical scale line coordinates, the horizontal scale line coordinates, and the binary image of the fetal heart rate curve to obtain fetal heart rate monitoring curve coordinate data. The data entry module 270 is used to store the fetal heart rate monitoring curve coordinate data in a database corresponding to the user to be entered. Compared with existing technologies, this application can input a large amount of paper-based fetal heart rate monitoring data generated in clinical practice into an information system. This can provide data for subsequent medical research and enable some automatic fetal heart rate monitoring curve interpretation systems developed based on artificial intelligence technology to be used in prenatal care, reducing errors caused by individual differences in human interpretation. In addition, this application uses a table line detection model and a fetal heart rate curve detection model to detect table lines and fetal heart rate curves. The performance of the models can be improved by increasing the amount of training data, avoiding the instability caused by the traditional OpenCV approach.
[0141] In some optional implementations of this embodiment, the fetal heart rate monitoring data entry device 200 further includes: a corner point identification acquisition module, an circumscribed rectangle calculation module, a rotation transformation calculation module, and a matrix correction module, wherein:
[0142] The corner marker acquisition module is used to acquire the four corner markers of the fetal heart rate monitoring image;
[0143] The bounding rectangle calculation module is used to perform bounding rectangle calculation operations on the four corner point markers according to the OpenCV minAreaRect function to obtain the minimum bounding rectangle.
[0144] The rotation transformation calculation module is used to perform rotation transformation calculations on the minimum bounding rectangle according to the matrix correction algorithm to obtain the rotation transformation matrix.
[0145] The matrix correction module is used to perform matrix correction processing on fetal heart rate monitoring images based on the rotation transformation matrix.
[0146] In some optional implementations of this embodiment, the fetal heart rate monitoring data entry device 200 further includes: a grayscale conversion module, wherein:
[0147] The grayscale module is used to perform grayscale processing on fetal heart rate monitoring images.
[0148] In some optional implementations of this embodiment, the fetal heart rate monitoring data entry device 200 further includes a segmentation module, and the scale recognition module 220 includes a scale recognition submodule, wherein:
[0149] The segmentation module is used to input fetal heart rate monitoring photos into the image segmentation model for segmentation operations, thereby obtaining segmented fetal heart rate monitoring images.
[0150] The scale recognition submodule is used to input the segmented fetal heart rate monitoring image into the text detection and recognition model to perform scale recognition operations and obtain scale data.
[0151] In some optional implementations of this embodiment, the image segmentation model described above uses U-Net as the network architecture.
[0152] In some optional implementations of this embodiment, the above-mentioned fetal heart rate monitoring data input device 200 further includes: a downsampling layer construction module, an upsampling layer construction module, a connection module, a Dropout layer construction module, an output module construction module, and a network parameter setting module.
[0153] The downsampling layer construction module is used to build the downsampling layer of U-Net;
[0154] The upsampling layer construction module is used to build the upsampling layer of U-Net;
[0155] The connection module is used to connect the downsampling layer and the upsampling layer based on Skip Connection;
[0156] The Dropout layer building module is used to build the Dropout layer;
[0157] Output module building module, used to build the network output module of U-Net;
[0158] The network parameter setting module is used to set the network parameters of U-Net.
[0159] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 8 , Figure 8 This is a basic structural block diagram of the computer device in this embodiment.
[0160] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected via a system bus. It should be noted that only the computer device 6 with components 61-63 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0161] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.
[0162] The memory 61 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as the hard disk or memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 6. Of course, the memory 61 may include both the internal storage unit and its external storage device of the computer device 6. In this embodiment, the memory 61 is typically used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions for a fetal heart rate monitoring data entry method. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or will be output.
[0163] In some embodiments, the processor 62 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 62 is typically used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to execute computer-readable instructions stored in the memory 61 or to process data, for example, to execute computer-readable instructions for the fetal heart rate monitoring data entry method.
[0164] The network interface 63 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 6 and other electronic devices.
[0165] The computer equipment provided in this application can input a large amount of paper-based fetal heart rate monitoring data generated in clinical practice into an information system. This can provide data for subsequent medical research and enable some automatic fetal heart rate monitoring curve interpretation systems developed based on artificial intelligence technology to be used in prenatal care, reducing errors caused by individual differences in human interpretation. In addition, this application uses a table line detection model and a fetal heart rate curve detection model to detect table lines and fetal heart rate curves. The performance of the models can be improved by increasing the amount of training data, avoiding the instability caused by the traditional OpenCV method.
[0166] This application also provides another embodiment, namely, a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the fetal heart rate monitoring data entry method described above.
[0167] The computer-readable storage medium provided in this application can input a large amount of paper-based fetal heart rate monitoring data generated in clinical practice into an information system. This can provide data for subsequent medical research and enable some automatic fetal heart rate monitoring curve interpretation systems developed based on artificial intelligence technology to be used in prenatal care, reducing errors caused by individual differences in human interpretation. In addition, this application uses a table line detection model and a fetal heart rate curve detection model to detect table lines and fetal heart rate curves. The performance of the models can be improved by increasing the amount of training data, avoiding the instability caused by the traditional OpenCV method.
[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0169] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.
Claims
1. A method for entering fetal heart rate monitoring data, characterized in that, Includes the following steps: Obtain fetal heart rate monitoring photos of the users to be entered; The fetal heart rate monitoring photos are input into a text detection and recognition model for scale recognition to obtain scale data. The fetal heart rate monitoring image is input into the table line detection model for table line recognition to obtain a binary image of the scale line, wherein the binary image of the scale line includes vertical scale lines and horizontal scale lines. By combining the scale data, the vertical scale line, and the horizontal scale line, the scale line coordinates are obtained, wherein the scale line coordinates include the vertical scale line coordinates and the horizontal scale line coordinates; The fetal heart rate monitoring photos are input into the fetal heart rate curve detection model to perform fetal heart rate curve recognition operation, and a binary image of the fetal heart rate curve is obtained. By combining the coordinates of the vertical scale line, the coordinates of the horizontal scale line, and the binary image of the fetal heart rate curve, fetal heart rate monitoring curve coordinate data is obtained, wherein, for each pixel point on the binary image of the fetal heart rate curve ( Search for the nearest upper vertical line in the horizontal pixels of the ruler. ,in, Indicates the position of a pixel in the image. express The corresponding ruler value; search for the nearest upper horizontal line in the vertical ruler pixels. ,in, Indicates the position of a pixel in the image. express The corresponding scale values; the fetal heart rate monitoring curve coordinate data are represented as follows: The fetal heart rate monitoring curve coordinate data is stored in the database corresponding to the user to be entered.
2. The method for entering fetal heart rate monitoring data according to claim 1, characterized in that, After the step of obtaining the fetal heart rate monitoring photo of the user to be entered, and before the step of inputting the fetal heart rate monitoring photo into the text detection and recognition model for scale recognition operation to obtain scale data, the following steps are also included: Obtain the four corner markers of the fetal heart rate monitoring image; The minimum bounding rectangle is obtained by performing bounding rectangle calculation on the four corner point markers using the OpenCV minAreaRect function. The rotation transformation matrix is obtained by performing a rotation transformation calculation on the minimum bounding rectangle according to the matrix correction algorithm. The fetal heart rate monitoring images are subjected to matrix correction processing based on the rotation transformation matrix.
3. The method for entering fetal heart rate monitoring data according to claim 1, characterized in that, After the step of obtaining the fetal heart rate monitoring photo of the user to be entered, and before the step of inputting the fetal heart rate monitoring photo into the text detection and recognition model for scale recognition operation to obtain scale data, the following steps are also included: The fetal heart rate monitoring images were converted to grayscale.
4. The method for entering fetal heart rate monitoring data according to claim 1, characterized in that, After the step of obtaining the fetal heart rate monitoring photo of the user to be entered, and before the step of inputting the fetal heart rate monitoring photo into the text detection and recognition model for scale recognition operation to obtain scale data, the following steps are also included: The fetal heart rate monitoring photos are input into an image segmentation model for segmentation to obtain segmented fetal heart rate monitoring images. The step of inputting the fetal heart rate monitoring image into a text detection and recognition model for scale recognition to obtain scale data specifically includes the following steps: The fetal heart rate monitoring segmented image is input into the text detection and recognition model for scale recognition to obtain the scale data.
5. The method for entering fetal heart rate monitoring data according to claim 4, characterized in that, The image segmentation model uses U-Net as its network architecture.
6. The method for entering fetal heart rate monitoring data according to claim 5, characterized in that, Before the step of inputting the fetal heart rate monitoring image into the image segmentation model for segmentation to obtain the segmented fetal heart rate monitoring image, the method further includes: Construct the downsampling layer of the U-Net; Construct the upsampling layer of the U-Net; The downsampling layer and the upsampling layer are connected via Skip Connection; Build the Dropout layer; Construct the network output module of the U-Net; Configure the network parameters of the U-Net.
7. A fetal heart rate monitoring data entry device, characterized in that, include: The photo acquisition module is used to acquire fetal heart rate monitoring photos of the user to be entered. The scale recognition module is used to input the fetal heart rate monitoring photos into the text detection and recognition model to perform scale recognition operations and obtain scale data. The table line recognition module is used to input the fetal heart rate monitoring photo into the table line detection model to perform table line recognition operation and obtain a binary image of the scale line, wherein the binary image of the scale line includes vertical scale lines and horizontal scale lines. The first combining module is used to combine the scale data, the vertical scale line, and the horizontal scale line to obtain the scale line coordinates, wherein the scale line coordinates include the vertical scale line coordinates and the horizontal scale line coordinates. The fetal heart rate curve recognition module is used to input the fetal heart rate monitoring photos into the fetal heart rate curve detection model to perform fetal heart rate curve recognition operations and obtain a binary image of the fetal heart rate curve. The second combining module is used to combine the coordinates of the vertical scale line, the coordinates of the horizontal scale line, and the binary image of the fetal heart rate curve to obtain fetal heart rate monitoring curve coordinate data, wherein, for each pixel point on the binary image of the fetal heart rate curve ( Search for the nearest upper vertical line in the horizontal pixels of the ruler. ,in, Indicates the position of a pixel in the image. express The corresponding ruler value; search for the nearest upper horizontal line in the vertical ruler pixels. ,in, Indicates the position of a pixel in the image. express The corresponding scale values; the fetal heart rate monitoring curve coordinate data are represented as follows: The data entry module is used to store the fetal heart rate monitoring curve coordinate data into a database corresponding to the user to be entered.
8. The fetal heart rate monitoring data entry device according to claim 7, characterized in that, The device further includes: The corner marker acquisition module is used to acquire the four corner markers of the fetal heart rate monitoring image; The bounding rectangle calculation module is used to perform bounding rectangle calculation operations on the four corner point markers according to the OpenCV minAreaRect function to obtain the minimum bounding rectangle. The rotation transformation calculation module is used to perform rotation transformation calculation operations on the minimum bounding rectangle according to the matrix correction algorithm to obtain the rotation transformation matrix. The matrix correction module is used to perform matrix correction processing on the fetal heart rate monitoring images according to the rotation transformation matrix.
9. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the fetal heart rate monitoring data entry method as described in any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the fetal heart rate monitoring data entry method as described in any one of claims 1 to 6.