Patient anomaly recognition method and device based on shelter hospital, server and storage medium

Abstract

This invention discloses a method, device, server, and storage medium for patient anomaly identification in makeshift hospitals. By establishing positional relationships between cameras, it fully acquires images of patients' abnormal behavior characteristics. Images from multiple cameras are synthesized into a feature tensor that can be recognized by a neural network, effectively reflecting the image features of patients' abnormal behavior. It effectively extracts the patient's location features and the correlation features between these locations, and uses changes in the angular orientation of these features to determine whether the actions are characteristic of an epidemic. Furthermore, it can combine abnormal characteristic actions with the time, frequency, and time interval of these actions to determine whether a patient is abnormal, reducing computational effort and providing relatively accurate anomaly identification results.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image recognition technology, and in particular to a method, device, server and storage medium for identifying abnormal patients in makeshift hospitals. Background Technology

[0002] Currently, makeshift hospitals are mostly converted from large stadiums or entertainment venues, resulting in high population density and a large number of medical devices, placing significant pressure on medical staff. In particular, real-time monitoring and management of patient conditions consumes a large portion of their energy. While patient service apps can assist in monitoring patient conditions, this method requires patients to accurately understand their own health status and actively fill out forms, which has a certain lag and low accuracy. Therefore, it does not fundamentally solve the aforementioned problems.

[0003] Currently, cameras can be used to capture images of patients' activities, and image features can be used to identify abnormal behaviors and help identify abnormal epidemic characteristics in patients. However, in makeshift hospitals, the large number of people and the limited space mean that camera positions cannot be set according to actual needs. This leads to inaccuracies in the captured image information, resulting in a significant possibility of misjudgment. Consequently, the efficiency and accuracy of this approach cannot effectively assist medical staff in monitoring patients' epidemic characteristics in makeshift hospitals. Summary of the Invention

[0004] This invention provides a method, device, server, and storage medium for abnormal patient identification in makeshift hospitals, in order to solve the technical problem that the efficiency and accuracy of image-assisted medical monitoring of patients' epidemic characteristics are low in the prior art.

[0005] In a first aspect, embodiments of the present invention provide a method for identifying abnormal patients in makeshift hospitals, including:

[0006] Obtain the BIM information of the makeshift hospital, and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information;

[0007] The shooting area and effective shooting depth of each image acquisition device are determined based on the position coordinates of the image acquisition device and the obstruction.

[0008] The image association region is determined based on the shooting area and effective shooting depth of each image acquisition device;

[0009] Based on the associated image region, identify the associated image acquisition device and establish an associated image acquisition device association table;

[0010] The system acquires multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device. The acquired associated images are then input into a trained semantic segmentation neural network and an object detection neural network. The trained semantic segmentation neural network is used to identify patient sub-images, and the object detection neural network outputs the position coordinates of the patient sub-images in the associated images.

[0011] The main image is selected from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is the patient's frontal view and is the patient sub-image that occupies the largest proportion in the image.

[0012] An image matrix is ​​established based on the main image, and the patient sub-image is fused with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image to form a high-dimensional image tensor.

[0013] The high-dimensional image tensor is input into the trained human feature neural network model, and the patient's condition is determined based on the output.

[0014] Secondly, embodiments of the present invention also provide a patient anomaly identification device based on a makeshift hospital, comprising:

[0015] The acquisition module is used to acquire the BIM information of the makeshift hospital and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information.

[0016] The determination module is used to determine the shooting area and effective shooting depth of each image acquisition device based on the position coordinates of the image acquisition device and the obstruction.

[0017] The image association region determination module is used to determine the image association region based on the shooting area and effective shooting depth of each image acquisition device;

[0018] A module is established to determine the associated image acquisition devices based on the associated image regions and to create an association table for the associated image acquisition devices.

[0019] The recognition module is used to acquire multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device, input the real-time acquired associated images into the trained semantic segmentation neural network and the target detection neural network respectively, use the trained semantic segmentation neural network to identify the patient sub-image, and output the position coordinates of the patient sub-image in the associated image through the target detection neural network.

[0020] The selection module is used to select a main image from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is the patient's frontal view and is the patient sub-image that occupies the largest proportion in the image.

[0021] The fusion module is used to establish an image matrix based on the main image, and to fuse the patient sub-image with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image to form a high-dimensional image tensor.

[0022] The input module is used to input the high-dimensional image tensor into the trained human feature neural network model, and determine whether the patient is abnormal based on the output result.

[0023] Thirdly, embodiments of the present invention also provide a server, comprising:

[0024] One or more processors;

[0025] Storage device for storing one or more programs;

[0026] When the one or more programs are executed by the one or more processors, the one or more processors implement the patient anomaly identification method based on makeshift hospitals provided in the above embodiments.

[0027] Fourthly, embodiments of the present invention also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the patient anomaly identification method based on a makeshift hospital as provided in the above embodiments.

[0028] The present invention provides a method, device, server, and storage medium for patient anomaly identification based on makeshift hospitals. This involves acquiring BIM information of the makeshift hospital and determining the position coordinates of image acquisition devices and obstructions based on the BIM information; determining the shooting area and effective shooting depth of each image acquisition device based on the position coordinates of the image acquisition devices and obstructions; determining image association regions based on the shooting area and effective shooting depth of each image acquisition device; identifying associated image acquisition devices based on the image association regions and establishing an association table for associated image acquisition devices; acquiring multiple frames of simultaneously acquired associated images in real time by the associated image acquisition devices; and inputting the real-time acquired associated images into a trained semantic segmentation neural network. A target detection neural network is used to identify patient sub-images using a trained semantic segmentation neural network. The target detection neural network then outputs the position coordinates of the patient sub-images in associated images. Based on the patient sub-images and their position coordinates in associated images, a main image is selected from the associated images. The main image is the patient's frontal view and the sub-image occupying the largest proportion of the image. An image matrix is ​​established based on the main image. According to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-images in the associated images, the patient sub-images are fused with the main image to form a high-dimensional image tensor. This high-dimensional image tensor is input into a trained human feature neural network model, and the output result determines whether the patient is abnormal. The method can utilize the established positional correlation between cameras to fully acquire images of abnormal patient behavior. Images from multiple cameras are synthesized into a feature tensor that can be recognized by the neural network, fully reflecting the image features of abnormal patient behavior. It effectively extracts the patient's location features and the correlation features between locations, and uses the changes in the azimuth angle between features to determine whether there are abnormal behavior characteristics. It can also combine abnormal characteristic movements with the time, frequency and time interval of the movement to determine whether the patient is abnormal, which can reduce computing power and provide more accurate abnormal judgment results. Attached Figure Description

[0029] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0030] Figure 1 This is a flowchart illustrating the patient anomaly identification method based on a makeshift hospital provided in Embodiment 1 of the present invention.

[0031] Figure 2 This is a flowchart illustrating the patient anomaly identification method based on a makeshift hospital provided in Embodiment 2 of the present invention.

[0032] Figure 3 This is a flowchart illustrating the patient anomaly identification method based on a makeshift hospital provided in Embodiment 3 of the present invention.

[0033] Figure 4 This is a flowchart illustrating the patient anomaly identification method based on a makeshift hospital provided in Embodiment 4 of the present invention.

[0034] Figure 4 This is a schematic diagram of the patient abnormality identification device based on a makeshift hospital provided in Embodiment 4 of the present invention;

[0035] Figure 5 This is a schematic diagram of the structure of a server provided in Embodiment 5 of the present invention. Detailed Implementation

[0036] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0037] Example 1

[0038] Figure 1 This is a flowchart illustrating the patient anomaly identification method based on a makeshift hospital provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where abnormal patient conditions are effectively identified based on images captured by cameras already installed in the makeshift hospital. The method can be executed using a patient anomaly identification device in the makeshift hospital and specifically includes the following steps:

[0039] Step 110: Obtain the BIM information of the makeshift hospital, and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information.

[0040] Fangcang hospitals are typically converted from large venues. During the construction of these venues, corresponding Building Information Modeling (BIM) models are retained. Based on the BIM model, basic construction information such as the length, width, and height of the venue can be determined. It can also reveal the specific locations and orientations of cameras (image acquisition devices) inside the venue, as well as the building components. For example, the size and coordinates of objects that may obstruct the view, such as supporting columns.

[0041] Step 120: Determine the shooting area and effective shooting depth of each image acquisition device based on the position coordinates of the image acquisition device and the obstruction.

[0042] Since each image acquisition device corresponds to a specific shooting angle, and due to the presence of obstructions, it is necessary to determine the shooting area and effective shooting depth for each camera. The effective depth refers to the furthest distance at which the captured image is obtained, assuming the presence of obstacles.

[0043] Step 130: Determine the image association region based on the shooting area and effective shooting depth of each image acquisition device.

[0044] Because each image acquisition device operates at a different angle, their captured images may overlap. That is, objects in one image may also appear in the image captured by another image acquisition device. Therefore, image association regions can be determined based on the shooting area and effective shooting depth of each image acquisition device. The image association region refers to the same object or area in the image.

[0045] Step 140: Determine the associated image acquisition device based on the associated image region, and establish an associated image acquisition device association table.

[0046] Image acquisition devices with associated image regions are designated as associated image acquisition devices, and an association table for associated image acquisition devices is established based on the association relationship. The association table records the shooting angle differences corresponding to the associated image regions.

[0047] Step 150: Acquire multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device. Input the real-time acquired associated images into the trained semantic segmentation neural network and object detection neural network respectively. Use the trained semantic segmentation neural network to identify the patient sub-image. Output the position coordinates of the patient sub-image in the associated image through the object detection neural network.

[0048] This application requires detecting a patient's condition using images; therefore, the patient image must first be extracted from complex background elements. Furthermore, since images are acquired from multiple image acquisition devices, it is also necessary to determine the patient's coordinate position within the images.

[0049] In this embodiment, the above objectives can be achieved using pre-trained semantic segmentation neural networks and object detection neural networks. Image semantic segmentation involves separating elements within an image that share the same semantic meaning. The trained semantic segmentation neural network can then identify the corresponding patient images. The object detection neural network's task is to locate objects in the image and their corresponding positions. The object detection neural network is used to obtain the coordinates of these objects.

[0050] Step 160: Select a main image from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is the patient's frontal view and is the patient sub-image that occupies the largest proportion in the image.

[0051] Image acquisition devices at different locations, due to their varying positions and angles, produce patient images with varying clarity and orientation, thus contributing differently to the identification of abnormal signs. Therefore, it is necessary to identify the most important image as the master image. For example, the master image can be determined based on the size of the patient image as defined in the above steps, and whether the target being identified is a frontal view of the patient. The master image can be a sub-image of the patient output by the semantic segmentation neural network described in the above steps.

[0052] Step 170: Establish an image matrix based on the main image, and fuse the patient sub-image with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image to form a high-dimensional image tensor.

[0053] In this embodiment, in order to obtain patient image features as accurately as possible, it is necessary to fuse and encode multiple images to obtain a more rich encoding of image feature information, which facilitates the convolutional neural network to obtain accurate patient feature recognition results.

[0054] For example, establishing an image matrix based on the main image may include: setting the most important original image as R. D×H×W H and W represent the width and height of the image, respectively, and D represents the grayscale value.

[0055] Accordingly, the step of fusing the patient sub-image with the main image based on the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated images may include: generating a feature tensor from other images together with the main original image based on the correlation degree, wherein the correlation degree is determined according to the angle size in the association table; and forming a multidimensional feature tensor R based on the number of frames at different angles. D×H×W×N N represents the number of frames. Using the above method, images of the same patient taken from different angles can be fused to form higher-dimensional image information. This facilitates feature extraction later and improves the accuracy of diagnosis.

[0056] Specifically, the matrix of each image is multiplied by the corresponding angle coefficient and the coefficient representing the size of the region corresponding to the position coordinates in the overall image, and then fused with the matrix elements of the main image by adding or juxtaposing the elements. This forms a multidimensional feature tensor R. D×H×W×N .

[0057] Step 180: Input the high-dimensional image tensor into the trained human feature neural network model, and determine whether the patient is abnormal based on the output result.

[0058] Human feature neural network models are used to extract features of important parts of the human body from images, particularly the facial features such as the mouth, eyes, and nose, as well as the main trunk and various joints. By utilizing changes in these features, abnormal states and behaviors of patients can be effectively identified, leading to accurate anomaly detection results.

[0059] This embodiment acquires the BIM information of the makeshift hospital and determines the position coordinates of image acquisition devices and obstructions based on the BIM information; it determines the shooting area and effective shooting depth corresponding to each image acquisition device based on the position coordinates of the image acquisition devices and obstructions; it determines the image association region based on the shooting area and effective shooting depth of each image acquisition device; it determines the associated image acquisition devices based on the image association region and establishes an association table of associated image acquisition devices; it acquires multiple frames of simultaneously associated images acquired in real time by the associated image acquisition devices, and inputs the real-time acquired associated images into a trained semantic segmentation neural network and an object detection neural network, respectively, and utilizes the trained... A semantic segmentation neural network identifies patient sub-images; a target detection neural network outputs the position coordinates of the patient sub-images in associated images; based on the patient sub-images and their position coordinates in associated images, a main image is selected from the associated images, the main image being the patient's frontal view and the patient sub-image occupying the largest proportion in the image; an image matrix is ​​established based on the main image, and the patient sub-images are fused with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-images in the associated images to form a high-dimensional image tensor; the high-dimensional image tensor is input into a trained human feature neural network model, and the patient's abnormality is determined based on the output result. The system can utilize the established positional correlation between cameras to fully acquire images of the patient's abnormal behavior features. Images acquired by multiple cameras are synthesized into a feature tensor that can be recognized by the neural network, fully reflecting the image features of the patient's abnormal behavior. It effectively extracts the patient's location features and the correlation features between locations, and uses the changes in the azimuth angle between features to determine whether there are abnormal behavior characteristics. Furthermore, it can combine abnormal behavior features with the time, frequency, and time interval of the behavior to determine whether the patient is abnormal, reducing computational effort and providing more accurate abnormality judgment results.

[0060] Example 2

[0061] Figure 2 This is a flowchart illustrating the patient anomaly identification method based on a makeshift hospital provided in Embodiment 2 of the present invention. This embodiment is an optimization based on the above embodiment. In this embodiment, the step of determining whether a patient is abnormal based on the output result is specifically optimized as follows: obtaining a simplified feature map of key human body parts; calculating the orientation and distance between features based on the simplified feature map; and determining whether a patient is abnormal based on the orientation and distance.

[0062] Accordingly, the patient anomaly identification method based on makeshift hospitals provided in this embodiment specifically includes:

[0063] Step 210: Obtain the BIM information of the makeshift hospital, and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information.

[0064] Step 220: Determine the shooting area and effective shooting depth of each image acquisition device based on the position coordinates of the image acquisition device and the obstruction.

[0065] Step 230: Determine the image association region based on the shooting area and effective shooting depth of each image acquisition device.

[0066] Step 240: Determine the associated image acquisition device based on the associated image region, and establish an associated image acquisition device association table.

[0067] Step 250: Acquire multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device. Input the real-time acquired associated images into the trained semantic segmentation neural network and object detection neural network respectively. Use the trained semantic segmentation neural network to identify the patient sub-image. Output the position coordinates of the patient sub-image in the associated image through the object detection neural network.

[0068] Step 260: Select a main image from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is the patient's frontal view and is the patient sub-image that occupies the largest proportion in the image.

[0069] Step 270: Establish an image matrix based on the main image, and fuse the patient sub-image with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image to form a high-dimensional image tensor.

[0070] Step 280: Input the high-dimensional image tensor into the trained human feature neural network model to obtain the output simplified feature map of key human body parts.

[0071] In this embodiment, the human feature neural network model can output a simplified feature map of key human body parts. This simplified feature map can be a line drawing depicting the connections between various important organs and parts. In the feature map, important organs and parts can be considered as points, with the emphasis on the lines connecting them. Under normal conditions, the features of each part vary within a reasonable range, and their orientation and distance are within a certain threshold range. Therefore, orientation and distance can be used to determine whether a patient is abnormal. Compared with traditional methods, this approach not only requires less computation but also avoids the problem of inaccurate judgments due to insufficient sample size.

[0072] Step 290: Calculate the orientation and distance between the features based on the simplified diagram of key human body parts, and determine whether the patient is abnormal based on the orientation and distance.

[0073] Based on the extracted epidemic-related characteristic actions and the time of their occurrence, the duration, frequency, and time interval of abnormal characteristic actions are recorded. Furthermore, based on the relationship between epidemic-related characteristic actions and their occurrence time, it is determined whether a patient exhibits epidemic-related characteristics, and a notification is sent to medical staff. For example, if a patient's main characteristics are diarrhea, elevated body temperature, decreased activity, and increased sneezing frequency, corresponding epidemic-related characteristic actions can be identified. Combining these with sneezing frequency, activity frequency, time intervals between toilet visits, and duration of toilet visits, it is determined whether epidemic-related characteristics are present.

[0074] This embodiment optimizes the process of determining whether a patient is abnormal based on the output results to: acquiring a simplified feature map of key human body parts, calculating the orientation and distance between features based on the simplified feature map, and determining whether the patient is abnormal based on the orientation and distance. This not only has the advantage of lower computational load but also avoids the problem of inaccurate judgments due to insufficient sample size.

[0075] Example 3

[0076] Figure 3 This is a flowchart illustrating the patient anomaly identification method based on a makeshift hospital provided in Embodiment 3 of the present invention. This embodiment is an optimization based on the above embodiment. In this embodiment, after selecting the main image from the associated images based on the patient sub-image and its position coordinates in the associated images, and before establishing the image matrix based on the main image, the method may further include the following step: obtaining the original image from the sub-image through deformable convolution. Figure 1 Feature maps of sizes 2, 1 / 3, and 1 / 5 are then used to adjust the size of the feature maps to the same preset size through dilated convolution.

[0077] Accordingly, the patient anomaly identification method based on makeshift hospitals provided in this embodiment specifically includes:

[0078] Step 310: Obtain the BIM information of the makeshift hospital, and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information.

[0079] Step 320: Determine the shooting area and effective shooting depth of each image acquisition device based on the position coordinates of the image acquisition device and the obstruction.

[0080] Step 330: Determine the image association region based on the shooting area and effective shooting depth of each image acquisition device.

[0081] Step 340: Determine the associated image acquisition device based on the associated image region, and establish an associated image acquisition device association table.

[0082] Step 350: Acquire multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device. Input the real-time acquired associated images into the trained semantic segmentation neural network and object detection neural network respectively. Use the trained semantic segmentation neural network to identify the patient sub-image. Output the position coordinates of the patient sub-image in the associated image through the object detection neural network.

[0083] Step 360: Select a main image from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is the patient's frontal view and is the patient sub-image that occupies the largest proportion in the image.

[0084] Step 370: Obtain the original image from the sub-image through deformable convolution. Figure 1 Feature maps of sizes 2, 1 / 3, and 1 / 5 are then used to adjust the size of the feature maps to the same preset size through dilated convolution.

[0085] The varying sizes of each patient's sub-image pose significant challenges for subsequent image fusion. Due to these differences, the sub-images need to be pre-processed to ensure consistent size while preserving the features of the original images, facilitating later processing. Deformable convolution essentially involves adding an offset to the sampling position in standard convolution operations, allowing the convolution kernel to expand over a large area during training. Furthermore, dilated convolution can effectively enhance the expanded receptive field of view, preventing the omission of associated image features.

[0086] Step 380: Based on the main image, establish an image matrix, and according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image, fuse the patient sub-image with the main image to form a high-dimensional image tensor.

[0087] Step 390: Input the high-dimensional image tensor into the trained human feature neural network model, and determine whether the patient is abnormal based on the output result.

[0088] Specifically, the human feature neural network model includes: a first convolutional branch for determining the credibility of the collected human features; a second convolutional branch for calculating the correlation vector between connected features; and a fusion layer for fusing the convolutional results of the first and second convolutional branches. Specifically, the human feature neural network model then determines the credibility of the collected human features, including the credibility of facial and body features, through the first branch. This includes concatenated 1×1, 1×1, 3×3, 3×3, and 3×3 convolutional kernels. The second branch is used to calculate the correlation vector between connected features. It also uses concatenated 3×3, 3×3, 3×3, 1×1, and 1×1 convolutional kernels. The first and second branches are fused to obtain the fusion result, thus achieving a multi-dimensional output of a simplified patient feature map.

[0089] 3×3, 3×3, 3×3, 1×1, and 1×1 convolutional kernels were used. This is mainly because the characteristics of different parts of the human body will produce correlated changes when abnormal. For example, when breathing is rapid, changes will occur in the mouth, nose, eyes, and head. A 1×1 convolutional kernel can extract changes in a single part, while a 3×3 convolutional kernel is more likely to extract changes in related parts together.

[0090] In this embodiment, after selecting the main image from the associated images based on the patient's sub-image and its position coordinates in the associated images, and before establishing the image matrix based on the main image, the method may further include the following step: obtaining the original image from the sub-image through deformable convolution. Figure 1 Feature maps of sizes 2, 1 / 3, and 1 / 5 are generated, and then their sizes are adjusted to the same preset size using dilated convolution. This method facilitates subsequent image fusion. Furthermore, by optimizing the structure of the human feature neural network model, it is possible to better extract patient features and related features, enabling accurate determination of whether a patient has an abnormality.

[0091] Example 4

[0092] Figure 4 This is a schematic diagram of the patient abnormality identification device based on a makeshift hospital provided in Embodiment 4 of the present invention, as shown below. Figure 4 As shown, the device includes:

[0093] The acquisition module 410 is used to acquire the BIM information of the makeshift hospital and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information.

[0094] The determining module 420 is used to determine the shooting area and effective shooting depth of each image acquisition device based on the position coordinates of the image acquisition device and the obstruction.

[0095] The image association region determination module 430 is used to determine the image association region based on the shooting area and effective shooting depth of each image acquisition device;

[0096] Module 440 is established to determine the associated image acquisition device based on the associated image region and to establish an associated table of associated image acquisition devices.

[0097] The recognition module 450 is used to acquire multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device, input the real-time acquired associated images into the trained semantic segmentation neural network and the target detection neural network respectively, use the trained semantic segmentation neural network to identify the patient sub-image, and output the position coordinates of the patient sub-image in the associated image through the target detection neural network.

[0098] The selection module 460 is used to select a main image from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is a frontal view of the patient and is the patient sub-image that occupies the largest proportion in the image.

[0099] The fusion module 470 is used to establish an image matrix based on the main image, and to fuse the patient sub-image with the main image according to the angle correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image to form a high-dimensional image tensor.

[0100] The input module 480 is used to input the high-dimensional image tensor into the trained human feature neural network model and determine whether the patient is abnormal based on the output result.

[0101] The patient anomaly recognition device based on a makeshift hospital provided in this embodiment acquires the BIM information of the makeshift hospital and determines the position coordinates of the image acquisition device and obstructions based on the BIM information; determines the shooting area and effective shooting depth corresponding to each image acquisition device based on the position coordinates of the image acquisition device and obstructions; determines the image association region based on the shooting area and effective shooting depth of each image acquisition device; determines the associated image acquisition devices based on the image association region and establishes an association table of associated image acquisition devices; acquires multiple frames of simultaneously associated images acquired in real time by the associated image acquisition devices, and inputs the real-time acquired associated images into a trained semantic segmentation neural network and an object detection neural network respectively. The network uses a trained semantic segmentation neural network to identify patient sub-images; and outputs the position coordinates of the patient sub-images in associated images through a target detection neural network. Based on the patient sub-images and their position coordinates in associated images, a main image is selected from the associated images. The main image is the patient's frontal view and the sub-image with the largest proportion in the image. An image matrix is ​​established based on the main image, and the patient sub-images are fused with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-images in the associated images to form a high-dimensional image tensor. The high-dimensional image tensor is input into a trained human feature neural network model, and the patient's abnormality is determined based on the output result. The network can utilize the positional correlation between cameras to fully acquire images of abnormal patient behavior. Images from multiple cameras are synthesized into a feature tensor that can be recognized by the neural network, fully reflecting the image features of abnormal patient behavior. The network effectively extracts the patient's location features and the correlation features between locations, and uses the changes in the angular orientation between features to determine whether the patient's actions are abnormal. It can also combine abnormal characteristic movements with the time, frequency and time interval of the movement to determine whether the patient is abnormal, which can reduce computing power and provide more accurate abnormal judgment results.

[0102] Based on the above embodiments, the fusion module includes:

[0103] The setting unit is used to set the most important original image to R. D×H×W H and W represent the width and height of the image, respectively, and D represents the grayscale value.

[0104] Based on the above embodiments, the fusion module includes:

[0105] Tensor generation unit is used to generate feature tensors from other images together with the main original image based on their correlation degree. The correlation degree is determined sequentially according to the angle size in the correlation table.

[0106] Forming units are used to form a multidimensional feature tensor R based on the number of frames at different angles. D×H×W×N N is the number of frames.

[0107] Based on the above embodiments, the input module includes:

[0108] An anomaly detection unit is used to acquire a simplified feature map of key human body parts, calculate the orientation and distance between features based on the simplified feature map, and determine whether the patient is abnormal based on the orientation and distance.

[0109] Based on the above embodiments, the anomaly detection unit includes:

[0110] The action determination subunit is used to determine whether a patient has any epidemic-related actions based on the orientation and distance between the human body part node image features and other part features, according to the angle.

[0111] The recording sub-unit is used to record the duration, frequency, and time interval of epidemic-characteristic actions, and to determine whether a patient is abnormal based on the relationship between the epidemic-characteristic actions and the time of occurrence.

[0112] Based on the above embodiments, the human feature neural network model includes:

[0113] The first convolutional branch is used to determine the reliability of the collected human features;

[0114] The second convolution branch is used to calculate the correlation vector between connected features;

[0115] A fusion layer used to combine the convolution results of the first and second convolution branches.

[0116] Based on the above embodiments, the device further includes:

[0117] The adjustment module is used to obtain the original image from the sub-image through deformable convolution. Figure 1 Feature maps of sizes 2, 1 / 3, and 1 / 5 are then used to adjust the size of the feature maps to the same preset size through dilated convolution.

[0118] The patient anomaly identification device based on makeshift hospitals provided in this embodiment of the invention can execute the patient anomaly identification method based on makeshift hospitals provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0119] Example 5

[0120] Figure 5 This is a schematic diagram of the structure of a server provided in Embodiment 5 of the present invention. Figure 5 A block diagram of an exemplary server 12 suitable for implementing embodiments of the present invention is shown. Figure 5The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0121] like Figure 5 As shown, server 12 is presented as a general-purpose computing device. The components of server 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0122] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0123] Server 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by server 12, including volatile and non-volatile media, removable and non-removable media.

[0124] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. Server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0125] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0126] Server 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable users to interact with server 12, and / or with any device that enables server 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, server 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of server 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with server 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0127] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the patient anomaly identification method based on a makeshift hospital provided in the embodiments of the present invention.

[0128] Example 6

[0129] Embodiment 6 of the present invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform any of the patient anomaly identification methods based on makeshift hospitals provided in the above embodiments.

[0130] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0131] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0132] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0133] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for identifying abnormal patients in makeshift hospitals, characterized in that, include: Obtain the BIM information of the makeshift hospital, and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information; The shooting area and effective shooting depth of each image acquisition device are determined based on the position coordinates of the image acquisition device and the obstruction. The image association region is determined based on the shooting area and effective shooting depth of each image acquisition device. The image association region refers to the same object or region in the image. Based on the associated image region, identify the associated image acquisition device and establish an associated image acquisition device association table; The system acquires multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device. The real-time acquired associated images are then input into the trained semantic segmentation neural network and the target detection neural network, respectively. The trained semantic segmentation neural network is then used to identify patient sub-images. The system then outputs the coordinates of the patient's sub-image within the associated image using a target detection neural network. The main image is selected from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is the patient's frontal view and is the patient sub-image that occupies the largest proportion in the image. An image matrix is ​​established based on the main image, and the patient sub-image is fused with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image to form a high-dimensional image tensor. The high-dimensional image tensor is input into the trained human feature neural network model, and the patient's condition is determined based on the output results. The step of fusing the patient sub-image with the main image based on the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image includes: Other images are used together with the main original image to generate a feature tensor based on their correlation degree. The correlation degree is determined sequentially according to the angle size in the correlation table. A multidimensional feature tensor R is formed based on the number of frames from different angles. D×H×W×N N is the number of frames.

2. The method according to claim 1, characterized in that, The step of establishing an image matrix based on the main image includes: Set the most important original image to R D×H×W H and W represent the width and height of the image, respectively, and D represents the grayscale value.

3. The method according to claim 1, characterized in that, The process of determining whether a patient is abnormal based on the output results includes: Obtain the simplified feature map of key human body parts, calculate the orientation and distance between features based on the simplified feature map, and determine whether the patient is abnormal based on the orientation and distance.

4. The method according to claim 3, characterized in that, The step of calculating the orientation and distance between features based on the simplified diagram of key human body parts, and determining whether the patient is abnormal based on the orientation and distance, includes: Based on the orientation and distance between the human body part node image features and other part features, the presence of epidemic-characteristic actions in the patient is determined according to the angle. Record the duration, frequency, and time interval of characteristic actions during the epidemic, and determine whether the patient is abnormal based on the relationship between the characteristic actions and the time of occurrence.

5. The method according to claim 1, characterized in that, The human feature neural network model includes: The first convolutional branch is used to determine the reliability of the collected human features; The second convolution branch is used to calculate the correlation vector between connected features; A fusion layer used to combine the convolution results of the first and second convolution branches.

6. The method according to claim 1, characterized in that, After selecting a master image from the associated images based on the patient sub-image and its position coordinates in the associated images, and before establishing an image matrix based on the master image, the method further includes: The sub-images are processed by deformable convolution to obtain feature maps of 1 / 2, 1 / 3, and 1 / 5 the size of the original image, and then the feature maps are adjusted to the same preset size by dilated convolution.

7. A patient anomaly identification device based on a makeshift hospital, characterized in that, include: The acquisition module is used to acquire the BIM information of the makeshift hospital and determine the position coordinates of the image acquisition device and the obstruction based on the BIM information. The determination module is used to determine the shooting area and effective shooting depth of each image acquisition device based on the position coordinates of the image acquisition device and the obstruction. The image association region determination module is used to determine the image association region based on the shooting area and effective shooting depth of each image acquisition device; A module is established to determine the associated image acquisition devices based on the associated image regions and to create an association table for the associated image acquisition devices. The recognition module is used to acquire multiple frames of simultaneously associated images acquired in real time by the associated image acquisition device, and input the real-time acquired associated images into the trained semantic segmentation neural network and the target detection neural network respectively, and use the trained semantic segmentation neural network to identify the patient sub-images. The system then outputs the coordinates of the patient's sub-image within the associated image using a target detection neural network. The selection module is used to select a main image from the associated images based on the patient sub-image and its position coordinates in the associated images. The main image is the patient's frontal view and is the patient sub-image that occupies the largest proportion in the image. The fusion module is used to establish an image matrix based on the main image, and to fuse the patient sub-image with the main image according to the angular correspondence of the associated image acquisition devices in the association table and the position coordinates of the patient sub-image in the associated image to form a high-dimensional image tensor. The input module is used to input the high-dimensional image tensor into the trained human feature neural network model, and determine whether the patient is abnormal based on the output result. The fusion module includes: Tensor generation unit is used to generate feature tensors from other images together with the main original image based on their correlation degree. The correlation degree is determined sequentially according to the angle size in the correlation table. Forming units are used to form a multidimensional feature tensor R based on the number of frames at different angles. D×H×W×N N is the number of frames.

8. A server, characterized in that, The server includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the patient anomaly identification method based on makeshift hospitals as described in any one of claims 1-6.

9. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the patient anomaly identification method based on a makeshift hospital as described in any one of claims 1-6.

Images