Intelligent medical care interaction platform system based on image processing technology

Abstract

The invention relates to the technical field of image processing, in particular to an intelligent medical care interaction platform system based on an image processing technology. The method comprises the following modules: a data acquisition module, a target identification module, a nursing behavior identification module and an auxiliary prompt module. The method specifically comprises the following steps: respectively acquiring local camera data and infrared sensing data of a ward area; according to the local camera data, a to-be-nursed target and a nursing execution target are identified, and the effective behavior interaction time of the nursing execution target and the to-be-nursed target is determined in combination with the infrared sensing data; comparing the interaction behavior characteristics in the effective behavior interaction time with a preset nursing behavior label, and determining the suitability of the current nursing behavior type; and screening a connection nursing behavior tag from the nursing behavior tags according to the suitability of the current nursing behavior type. Under high-frequency nursing scenes such as intensive care, omission risk and operation error rate can be reduced, and nursing safety and nursing quality are integrally improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a smart medical care interactive platform system based on image processing technology. Background Technology

[0002] In traditional nursing models, healthcare workers primarily rely on paper records, manual rounds, and manual operations to monitor and manage patients' health. This not only leads to low efficiency, delayed information transmission, and uneven quality of care, but also easily results in wasted healthcare resources and potential safety hazards. Especially in scenarios such as intensive care, geriatric care, and chronic disease management, where nursing procedures are frequent and real-time requirements are high, traditional methods struggle to meet the demands for real-time monitoring and rapid response.

[0003] In recent years, with the continuous development of image processing technology and computer vision algorithms, image-based intelligent monitoring and analysis have been increasingly applied in the field of medical care. For example, with the patient's authorization, image data of patient behavior, facial expressions, and physiological indicators can be acquired through cameras. Combined with image recognition and deep learning models, patient status monitoring, abnormal behavior detection, and decision support can be achieved. However, existing image processing methods are easily affected by changes in ambient light, occlusion interference, and the diversity of patient movements in practical applications, resulting in unstable recognition accuracy and making it difficult to meet the high-precision requirements of clinical nursing. Summary of the Invention

[0004] Therefore, it is necessary for the present invention to provide a smart medical care interaction platform system based on image processing technology to solve at least one of the above-mentioned technical problems.

[0005] To achieve the above objectives, a smart medical care interaction platform system based on image processing technology includes the following modules:

[0006] The data acquisition module includes a camera group and an infrared sensor group pre-deployed in the ward area; the camera group and the infrared sensor group respectively collect local video data and infrared sensor data of the ward area.

[0007] The target recognition module identifies the target to be cared for and the target to be cared for based on the edge contour information in the local camera data, and determines the effective behavioral interaction time between the target to be cared for and the target to be cared for by combining infrared sensor data;

[0008] The nursing behavior recognition module compares the interactive behavior characteristics within the effective interaction time with preset nursing behavior tags to identify the current nursing behavior type and determine the suitability of the current nursing behavior type.

[0009] The auxiliary prompt module filters connecting nursing behavior tags from nursing behavior tags based on the adaptability of the current nursing behavior type, and sends the connecting nursing behavior tags to the medical and nursing terminals corresponding to the nursing execution goals to prompt the nursing execution goals.

[0010] This application constructs a multi-view camera acquisition and infrared sensing collaborative perception architecture in the ward area to achieve continuous, non-contact monitoring of nursing execution goals and nursing care objectives. It can acquire behavioral information and changes in body surface status during the nursing process in real time without increasing the operational burden on medical staff. By temporally aligning and jointly analyzing image grayscale change features, edge contour features, and infrared temperature features, the impact of ambient light fluctuations, partial occlusion, and posture changes on the recognition results can be effectively reduced, significantly improving the stability and robustness of nursing behavior recognition. Simultaneously, by constructing a nursing behavior label library and introducing an adaptation assessment and connection prompting mechanism, the application further enhances the nursing behavior recognition capabilities. This system not only identifies the current nursing behavior type but also dynamically judges whether the nursing effect meets expectations based on changes in the patient's physical condition. It automatically pushes the next recommended nursing behavior to the medical and nursing terminals, thereby reducing reliance on experience and human judgment bias, and improving the standardization and continuity of the nursing process. In addition, through a closed-loop control method of automated data collection, intelligent analysis, and real-time prompts, the system effectively reduces the frequency of manual inspections, improves nursing response speed and resource utilization efficiency. In high-frequency nursing scenarios such as intensive care, geriatric care, and chronic disease management, it helps reduce the risk of omissions and operational error rates, and improves the overall nursing safety and quality. Attached Figure Description

[0011] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0012] Figure 1 This is a schematic diagram of the modules of the intelligent medical care interactive platform system based on image processing technology of the present invention;

[0013] Figure 2 This is a block diagram of the task execution architecture of the camera group and infrared sensor group in an embodiment of the present invention;

[0014] Figure 3 This is a schematic diagram showing the distribution of the camera group and the infrared sensor group in an embodiment of the present invention;

[0015] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0017] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.

[0018] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0019] To achieve the above objectives, please refer to Figures 1 to 3 This invention provides a smart medical care interactive platform system based on image processing technology, the system comprising the following modules:

[0020] The data acquisition module 101 includes a camera group and an infrared sensor group pre-deployed in the ward area; the camera group and the infrared sensor group respectively collect local video data and infrared sensor data of the ward area.

[0021] In this embodiment, two sets of fixed-focus cameras, pre-deployed on both sides of each bed within the ward area, synchronously acquire local video data of the bed area. The resolution of the cameras is set to 1280×720, and the frame rate is set to 25 frames per second. Simultaneously, an infrared temperature matrix data of the corresponding area is acquired by an infrared sensor group pre-deployed directly above the geometric center of the ward area. The spatial resolution of the infrared sensor is set to 64×64, and the sampling period is set to 0.2 seconds. Subsequently, grayscale conversion and median filtering are performed on the local video data, and temperature drift correction and noise suppression are performed on the infrared sensor data to obtain a time-aligned local video sequence and infrared temperature sequence.

[0022] The target recognition module 102 identifies the target to be cared for and the target to be cared for based on the edge contour information in the local camera data, and determines the effective behavioral interaction time between the target to be cared for and the target to be cared for by combining infrared sensor data;

[0023] In a further embodiment, Canny edge maps are extracted from each frame of the preprocessed local camera data, and several closed contour regions are formed by marking 8 connected regions. The area parameter A and aspect ratio parameter R of each closed contour region are calculated, and these parameters are compared with the patient body shape feature intervals [A1,A2] and [R1,R2] bound to the hospital beds in the medical management platform to select similar human body contours as nursing targets. Subsequently, barcode grayscale periodic regions are detected in the same frame image, and adaptive binarization and stripe width normalization are performed on the barcode regions to decode identity information. The decoding results are matched with the nursing staff identity code table to determine the nursing execution target. Furthermore, the infrared temperature sub-region is aligned with the two target boxes, and the temperature synchronization coefficient K and spatial proximity D are calculated. If K ≥ 0.7 and D falls within the 0.3m to 0.8m range, the corresponding time period is determined as the effective behavioral interaction time.

[0024] The nursing behavior recognition module 103 compares the interactive behavior characteristics within the effective interaction time with the preset nursing behavior tags to identify the current nursing behavior type and determine the suitability of the current nursing behavior type.

[0025] In a further embodiment, during the effective behavioral interaction time Calculate the gray-level difference matrix between adjacent frames. And extract local grayscale change sub-matrices within the target bounding box corresponding to the nursing execution goal and the nursing goal to be cared for. Calculate the values ​​of the sub-matrices in the following cases. Cumulative change within Continuous variation duration and the proportion of areas with concentrated changes and the parameter sequence Interactive behavior feature vectors are generated by combining them in chronological order. Then, the cosine similarity of these interactive behavior feature vectors is calculated one by one with the reference feature vectors corresponding to each nursing behavior in the nursing behavior tag library. Select Nursing behaviors with a maximum value greater than 0.6 were identified as the current nursing behavior type. Further analysis was performed by extracting the mean difference in body surface temperature between the start and end times of the interaction. And compare it with the corresponding expected change range to obtain the fit value.

[0026] The auxiliary prompt module 104 filters connecting nursing behavior tags from nursing behavior tags based on the adaptability of the current nursing behavior type, and sends the connecting nursing behavior tags to the medical and nursing terminals corresponding to the nursing execution goals to prompt the nursing execution goals.

[0027] In a further embodiment, a set of connectable nursing behaviors corresponding to the nursing behavior type is retrieved from the nursing behavior tag library, and the applicable object category and trigger condition parameter range of each connectable nursing behavior are read respectively. If the temperature exceeds a preset threshold of 0.65, the average body surface temperature of the target patient at the current moment will be extracted. With temperature stability And calculate the trigger matching degree by matching with each trigger condition interval. Select The most significant nursing action serves as a connecting nursing action. If If the value is not greater than 0.65, the same matching calculation is performed from the set of corrective nursing behaviors, the corrective nursing behavior is selected as the connecting nursing behavior, and the corresponding nursing behavior code is sent to the medical and nursing terminal of the nursing execution target to form the next nursing prompt information.

[0028] It is worth noting that the nursing behavior tag library is derived from standard nursing operation procedure documents and historical nursing record data retrieved from the medical and nursing management platform. Nursing operations are categorized and organized according to nursing project type to form a basic set of nursing behaviors. For each type of nursing behavior, the corresponding operation stage is manually labeled and divided into intervals. Grayscale change parameters, action duration parameters, and body surface state change parameters within the corresponding stage are extracted from historical nursing video samples to construct action change feature vectors for the nursing execution target side and body surface state change feature vectors for the target side. Subsequently, the feature vectors, expected state change intervals, applicable object categories, and connectable nursing behavior identifiers corresponding to each nursing behavior are uniformly encapsulated into nursing behavior tag units. Using the nursing behavior type code as the index key, multiple nursing behavior tag units are stored in a structured tag table. This tag table includes behavior feature fields, expected change interval fields, object category fields, and connection (correction) relationship fields, thus forming the nursing behavior tag library.

[0029] Optionally, the data acquisition module may collect local camera data and infrared sensor data of the ward area, including:

[0030] Local camera data from both sides of the bed is acquired by pre-deployed camera groups on both sides of the bed in the ward area; infrared sensor data from the bed area is acquired by pre-deployed infrared sensor groups directly above the geometric center point of the ward area.

[0031] Data preprocessing is performed on the local camera data and the infrared sensor data respectively.

[0032] In this embodiment, if the ward enters nursing monitoring mode, a fixed-focus camera is activated on each side of the bed for synchronous image acquisition. The camera is installed at a height of 1.4 to 2.3 meters above the ground, with its optical axis pointing towards the center line of the bed, a horizontal field of view of 70°, a single frame resolution of 1280×720, and a frame rate of 25 frames per second. The system aligns the image frames acquired by the cameras on both sides according to a unified timestamp and combines the left and right view images corresponding to each moment to form a local multi-view image frame group, which serves as the local camera data input for the bed area and is used for subsequent contour extraction and target localization. If the system completes the camera data synchronization, an infrared sensor group deployed directly above the geometric center of the ward is simultaneously activated for thermal imaging acquisition. The infrared sensor group has a spatial resolution of 64×64, a temperature measurement range of 20℃ to 45℃, a temperature resolution of 0.1℃, and a sampling period of 0.2 seconds. The system projects the two-dimensional temperature matrix collected by the infrared sensor onto a preset coordinate plane according to the bed area, and establishes a mapping relationship between the infrared temperature matrix and each bed spatial area based on the bed layout parameters, thereby forming infrared temperature time-series data segmented by bed area. After obtaining the original local camera data and infrared temperature matrix data, the camera images are first processed by grayscale and noise is suppressed using a 3×3 median filter window. At the same time, temperature drift correction and spatial smoothing are performed on the infrared temperature matrix, with the smoothing window size set to 5×5. Subsequently, the two types of data are resampled in time according to a unified timestamp, so that the time interval is uniformly 0.2 seconds, and a frame-level index relationship is established between the image frame and the temperature matrix corresponding to each time moment, thereby obtaining the time-aligned local camera sequence and infrared temperature sequence, which serve as the standard input data for subsequent target recognition.

[0033] It is worth noting that, to avoid interference from ambient lighting on the image contrast of the camera assembly and the temperature measurement accuracy of the infrared sensor assembly, the main lighting fixtures in the ward area are positioned in the strip-shaped area along the edge of the ceiling on both sides above the hospital bed. The horizontal offset distance between the center line of the lighting fixtures and the center line of the hospital bed is set to 0.6 to 1.2 meters, and the light output angle of the lighting fixtures is tilted outward by 15° to 25° to avoid direct illumination on the surface of the hospital bed and the center of the infrared sensor's field of view. At the same time, auxiliary diffuse reflection ceiling lights are set in the four corners of the ward, with their height level with the ceiling, and the illuminance of a single light is controlled within the range of 150 to 250 lux, so that the hospital bed area forms uniform diffuse illumination without generating strong reflective hot spots. For the area directly above the hospital bed and directly below the infrared sensor, a zone without direct lighting fixtures is set up, keeping the illuminance in this area below 100 lux, thereby reducing the false high error of the body surface temperature caused by visible light radiation and ensuring that the grayscale distribution of the camera image is stable and the measurement accuracy of the infrared temperature matrix is ​​controlled synchronously.

[0034] It is worth noting that the nursing monitoring status is automatically activated when the nursing time is reached (based on statistical classifications of medical records and diagnostic results). At this time, the patient is placed in a suitable bed area and their body posture is adjusted according to different diagnostic result types. Furthermore, when the nursing monitoring status is not activated, only the infrared sensor group operates in the ward area; the camera group is not operational. Human contour comparison only retains the contours of the patient (the target of nursing care) or medical staff (the target of nursing execution); other contours are automatically deleted.

[0035] Figure 2 This is a block diagram of the task execution architecture of the camera group and infrared sensor group in an embodiment of the present invention; the camera group and infrared sensor group 201 are connected to the intelligent medical care interaction platform system 203 based on image processing technology through network 202. Network 202 can be a wide area network or a local area network, or a combination of both.

[0036] In some embodiments, the intelligent medical care interaction platform system 203 based on image processing technology receives instructions that include the start and stop of nursing monitoring status, and sends them to the camera group and infrared sensor group 201 via the network 202 through the communication component. The camera group and infrared sensor group 201 then perform corresponding data acquisition tasks according to the instructions.

[0037] Optionally, the target recognition module identifies the target to be cared for, including:

[0038] Detect the grayscale distribution and edge gradient of each frame in local camera data to identify the outlines of candidate bodies and beds within the ward area;

[0039] The closed area and aspect ratio of each candidate body contour are calculated separately to serve as the geometric features of the candidate body contour. The geometric features of each candidate body contour are then compared with the body shape features of patients in the electronic medical records of the corresponding beds in the ward area of ​​the medical management platform to screen for similar human body contours of patients.

[0040] Based on the spatial relationship between the patient's similar human body contour and the contour of the hospital bed, nursing targets are selected.

[0041] In this embodiment, upon receiving preprocessed local camera data, adaptive histogram equalization is performed on each frame to enhance grayscale contrast, and the Sobel edge gradient magnitude and direction for each pixel are calculated. Subsequently, an 8-connected region detection algorithm is used to extract closed contours from the edge images, forming several closed contour regions. Preliminary screening is performed using the perimeter-to-area ratio of the contours to remove noisy contours with an area less than 200 pixels or a perimeter less than 50 pixels. Simultaneously, the bed contour is identified using the projection contour method, ensuring its area and aspect ratio match the actual dimensions of the bed (approximately 2 meters long and 1 meter wide). If candidate body contours are obtained, the closed area A and aspect ratio R are calculated for each closed contour, and a geometric feature vector is constructed. Then, patient body shape feature data, including body length and shoulder width ranges, is obtained from the electronic medical records in the medical management platform. The similarity S between the contour geometric features and the patient body shape features is calculated using Euclidean distance. When the similarity threshold S ≥ 0.75, the corresponding contour is marked as a similar human body contour to the patient and used as a candidate for subsequent nursing care. A two-dimensional bed coordinate system is established based on the bed contour, with the bed centerline as the Y-axis and the head of the bed as the origin. The projection distance D and contour height H of the center point of the patient's similar human contour in the bed coordinate system are calculated, and D and H are compared with the preset threshold range of the resting area (D≤0.3 meters, H≤0.5 meters). If both fall within the threshold range, the bounding box of the corresponding contour is marked as the target to be cared for, and the target box coordinates and contour feature parameters are output.

[0042] Optionally, identifying candidate body contours within the ward area includes:

[0043] The grayscale distribution of each frame image is equalized and corrected, and several grayscale regions are divided according to the average grayscale value of adjacent pixels in each frame image after equalization and correction.

[0044] In this embodiment, adaptive histogram equalization is performed on each frame of the local camera data to enhance grayscale details in low-contrast areas, while the number of segments in the equalization histogram is set to 256. Subsequently, the grayscale mean of each pixel and its horizontal and vertical 3×3 neighboring pixels is calculated, and pixels with similar consecutive grayscale mean values ​​are grouped into the same grayscale region. The number of grayscale regions is set to 6 to 10 to represent the grayscale distribution characteristics of different parts of the human body.

[0045] The gradient magnitude and gradient direction of each pixel are extracted based on the edge gradient in each frame image, and several closed edge contour regions are divided according to the similarity of the gradient magnitude and the consistency of the gradient direction of adjacent pixels.

[0046] In a further embodiment, the Sobel operator edge gradient of each frame image is calculated to obtain the gradient magnitude and gradient direction of each pixel. Based on the continuity of adjacent pixel gradient magnitude differences being less than 15% and directional deviations being less than 10°, pixels are clustered into several closed edge contour regions, and closure detection is performed using the 8-connectivity algorithm. The area and perimeter of each contour are calculated, and contours with an area less than 200 pixels or aspect ratios that do not conform to human features are discarded to improve the accuracy of candidate contours.

[0047] Overlay closed edge contour areas and grayscale areas. If the number of grayscale areas contained within the closed edge contour area falls within the preset human body grayscale distribution range, and the area of ​​the largest grayscale area is higher than 50%, then the corresponding closed edge contour area will be used as the candidate human body contour.

[0048] In a further embodiment, the grayscale regions within the contour region are statistically analyzed to calculate the number N of grayscale regions and the proportion of the largest grayscale region to the total contour area. .like Falling into the preset human body grayscale distribution range (3-8 grayscale areas), and If the value is greater than 0.5, the closed contour region is marked as the candidate body contour, and the contour bounding box and grayscale region distribution parameters are output.

[0049] Optionally, screening for care targets includes:

[0050] Establish a bed coordinate system based on the bed outline, and calculate the projection distance and height distribution parameters of the patient's similar human body outline relative to the center line of the bed based on the spatial position relationship of the patient's similar human body outline in the bed coordinate system.

[0051] The target box corresponding to the patient's similar human body contour, whose projection distance and height distribution parameters both fall within the preset threshold range of the patient's resting area, is used as the nursing target.

[0052] In this embodiment, a two-dimensional bed coordinate system is established based on the bed contour, with the bed centerline as the Y-axis and the head of the bed as the origin, and the bed width direction is set as the X-axis. The center point of the bounding box of the patient's similar human contour is mapped to this coordinate system, and its projection distance D along the X-axis and its height distribution H along the Y-axis are calculated. Simultaneously, the longitudinal position and horizontal offset of the contour relative to the head of the bed are recorded. The sampling precision for the projection distance and height distribution is set to 0.01 meters. The spatial parameters of the patient's similar human contour in the bed coordinate system are then compared with a preset resting area threshold, where the threshold range is set to D≤0.3 meters and H≤0.5 meters. If both the projection distance and height distribution fall within the threshold range, the bounding box corresponding to the contour is marked as the target to be cared for, and the coordinate information and geometric features of the target box are output.

[0053] Optionally, the target recognition module identifies nursing execution targets including:

[0054] Based on the grayscale distribution of each frame in the local camera data, the barcode region in the image is identified, and the pixels within the barcode region are subjected to binarization enhancement and stripe width normalization processing to determine the barcode information;

[0055] In this embodiment, each frame of the local camera data is converted to grayscale, and the grayscale mean change curve of each row and column of pixels is calculated. Extreme point detection and periodic stability analysis are performed on the grayscale mean curve to identify areas with periodic grayscale changes as candidate barcode regions. Subsequently, binarization enhancement is performed on the pixels in the barcode region, with the threshold set to Otsu's automatic threshold, and the stripe width normalization range set to 5–20 pixels. This allows barcodes with different stripe widths to be uniformly parsed, thereby generating a decodeable barcode information matrix.

[0056] The barcode information is matched with the nursing staff identification codes in the medical and nursing management platform to obtain the physical characteristics of the nursing staff;

[0057] In a further embodiment, the barcode information matrix is ​​decoded into a nursing staff identification code string, and the corresponding nursing staff record is retrieved in the medical and nursing management platform to obtain the nursing staff's physical characteristics, including height range (1.55-1.90 meters), shoulder width range (0.35-0.55 meters), and nursing staff type (such as nurse or nursing assistant).

[0058] Based on the physical characteristics of nursing staff, target boxes corresponding to similar human body contours of nursing staff are selected from the candidate body contours and used as nursing execution targets.

[0059] In a further embodiment, the similarity between the geometric feature vector of each contour and the body shape vector of the nursing staff is calculated from the candidate body contour set using weighted Euclidean distance, with a threshold set to 0.8. Contours with similarity exceeding the threshold are marked as similar human body contours to the nursing staff, and their target box coordinates and contour features are output.

[0060] Of particular importance is the identification of barcode regions in the image, including:

[0061] Detect the grayscale projection of each frame of local camera data in the row and column directions after grayscale processing, and calculate the grayscale mean change curve of each row of pixels and each column of pixels respectively.

[0062] In this embodiment, each frame of the local camera data is converted to grayscale, and the mean grayscale value of each row and column pixel is calculated along the row and column directions respectively, resulting in row grayscale projection curves and column grayscale projection curves. The mean grayscale value is calculated using 3×3 pixel neighborhood smoothing, and the sampling precision of the projection curve is 0.01 pixel grayscale unit.

[0063] Based on the grayscale mean change curves of pixels in each row and column, extreme point detection and periodic stability analysis are performed on the grayscale mean change curves to detect grayscale periodic change regions.

[0064] In a further embodiment, local extremum point detection is performed on the grayscale mean change curves of each row of pixels and each column of pixels. The threshold is set to ±15% of the curve mean to identify the location of grayscale peaks and valleys. Subsequently, periodic stability analysis is performed on the distance between adjacent extremum points, with the periodic deviation set to no more than 5%. If the distance between consecutive peaks and valleys meets the stability condition, the corresponding region is marked as a grayscale periodic change region.

[0065] Extract the first-order difference sequence of pixel gray levels within the periodically changing gray level region, and calculate the continuity of gray level change amplitude and black-and-white alternation frequency of adjacent pixels to form stripe structure feature parameters.

[0066] In a further embodiment, the first-order difference sequence of pixel gray levels within the periodically changing gray level region is extracted, and the continuity of gray level change amplitude and the black-and-white alternation frequency of adjacent pixels are calculated. The continuity parameter is set as the proportion of pixels with adjacent difference changes of less than 10 gray level units, and the black-and-white alternation frequency parameter is the number of black and white stripes per millimeter, typically 10 to 30, used to describe the barcode stripe characteristics.

[0067] The stripe structure feature parameters are compared with the preset barcode stripe structure judgment threshold to filter barcode regions.

[0068] In a further embodiment, parameters such as continuity and alternation frequency are compared with preset barcode stripe structure determination thresholds. The threshold range is: continuity ≥ 0.85, black-and-white alternation frequency between 10 and 30 stripes / mm. Gray-scale periodic areas that meet the determination criteria are marked as barcode areas, and the coordinates of the area bounding box and stripe parameters are output.

[0069] Optionally, the target recognition module, in conjunction with infrared sensor data, determines the effective behavioral interaction time between the nursing execution target and the target to be cared for, including:

[0070] Align the surface temperature value of each pixel in the infrared sensor data with the target box corresponding to the nursing execution target and the target box corresponding to the target to be cared for, and determine the infrared temperature sub-region of the target box corresponding to each target.

[0071] In this embodiment, the nursing execution target bounding box and the target bounding box to be cared for, already located in the visible light frame, are used as spatial constraints to perform affine mapping on the infrared image at the same scale. During the mapping process, a boundary compensation area of ​​no more than 8 pixels is extended outward from the center point of the target bounding box as a reference to form an infrared temperature sub-region. Subsequently, the temperature value of each pixel in this sub-region is screened for validity, and abnormal points below 32℃ or above 42℃ are removed. A two-dimensional temperature matrix is ​​then constructed using row and column indexing.

[0072] The temperature change amplitude and temperature gradient distribution of the infrared temperature sub-region corresponding to the nursing execution target are used as the thermal motion characteristics of the nursing execution target; at the same time, the temperature stability parameters and local temperature rise rate in the infrared temperature sub-region corresponding to the nursing target are calculated as the surface state characteristics of the nursing target.

[0073] In a further embodiment, the sampling interval between two consecutive infrared sub-regions is fixed at 0.1s. The temperature difference of each pixel in the target sub-region of the nursing care is calculated, the average temperature rise amplitude per unit time is calculated, and the distribution ratio of temperature gradient direction is statistically analyzed in a 4×4 local grid to form a thermal motion feature vector containing the temperature rise amplitude and the gradient direction ratio. At the same time, the variance of a 5-frame sliding window is calculated for the pixel temperature sequence in the target sub-region of the nursing care to characterize the temperature stability, and the ratio of the maximum temperature rise within the window to the time interval is further calculated to obtain the local temperature rise change rate.

[0074] Calculate the synchronization coefficient and spatial proximity of temperature changes between the thermal motion characteristics of the nursing execution target and the surface state characteristics of the target at adjacent time points;

[0075] In a further embodiment, Pearson correlation calculation is performed on the thermal motion feature sequence of the nursing execution target and the surface state feature sequence of the target to be cared for, with a minimum of 10 effective sample points, and the obtained correlation coefficient is used as the temperature change synchronization coefficient; at the same time, the spatial proximity parameter is obtained by converting the Euclidean distance between the center points of the two target frames in the infrared coordinate system and the actual distance in combination with the infrared resolution.

[0076] If the temperature change synchronization coefficient is higher than the preset interaction threshold and the spatial proximity falls within the preset nursing operation distance range, then it is determined that there is an effective behavioral interaction between the nursing execution goal and the nursing goal to be cared for, so as to determine the effective behavioral interaction time.

[0077] In a further embodiment, if the temperature change synchronization coefficient at the current moment is greater than 0.65 after filtering, and the corresponding spatial proximity falls within the preset nursing operation distance range of 0.3m to 0.8m, then the moment is marked as a candidate interaction moment; further, it is required that this condition remains true for no less than 5 consecutive frames to exclude instantaneous proximity or noise interference; when the number of consecutively satisfied frames reaches a threshold, the moment of the first frame is recorded as the start time of the effective behavior interaction, and the frame before the condition fails is recorded as the end time, thereby outputting a continuous effective behavior interaction time interval.

[0078] Optionally, the nursing behavior recognition module identifies the current nursing behavior type, including:

[0079] By utilizing the grayscale changes within the effective behavioral interaction time in local camera data, the interactive behavioral characteristics between nursing execution goals and nursing objectives can be determined.

[0080] In this embodiment, at least 50 grayscale images are continuously extracted from the effective interaction time interval. For each frame, the grayscale mean and grayscale variance are calculated in the overlapping area between the nursing execution target box and the target box to be cared for. The grayscale mean difference of adjacent frames is further used to obtain a grayscale change amplitude sequence. A time sliding window is constructed in groups of 5 frames, and the mean and standard deviation of grayscale changes within the window are statistically analyzed to characterize the intensity and stability of the action. At the same time, the interaction area is divided into 4×4 grids, and the consistency ratio of grayscale change direction within each grid is statistically analyzed. Finally, an interaction behavior feature vector containing change amplitude, fluctuation and spatial distribution ratio is formed.

[0081] The interaction behavior features are matched one by one with the nursing behavior reference standard features corresponding to each nursing behavior in the preset nursing behavior label library. The similarity value between the current interaction behavior feature and each nursing behavior reference standard feature is calculated to determine the current nursing behavior type.

[0082] In a further embodiment, each type of nursing behavior in the nursing behavior label library is represented by a reference feature vector of fixed length. Each vector is composed of amplitude features, stability features, and spatial distribution features concatenated with weights of 0.4, 0.3, and 0.3, respectively. The cosine similarity between the current interaction behavior feature and each reference feature is calculated, with a minimum number of participating dimensions of no less than 12. When the highest similarity value is greater than 0.75 and the difference between it and the second highest similarity value is not less than 0.1, the corresponding label is determined as the current nursing behavior type. If the difference constraint is not met, the time period is marked as pending confirmation, and the top two similarity values ​​are output for subsequent review.

[0083] Of particular importance is identifying the behavioral characteristics of the interaction between nursing performance goals and the goals to be cared for, including:

[0084] Using local camera data, calculate the gray-level difference matrix between adjacent frames within the effective behavioral interaction time period;

[0085] Extract the local gray-level change sub-matrices corresponding to the spatial range of the target boxes corresponding to the nursing execution target and the nursing target from the gray-level difference matrix respectively, and calculate the cumulative change, change duration and change concentration area distribution parameters of the local gray-level change sub-matrices during the effective behavior interaction time;

[0086] By integrating the cumulative change, the duration of change, and the distribution parameters of the concentrated change area in chronological order, the interactive behavioral characteristics between nursing execution goals and nursing goals within the effective behavioral interaction time can be obtained.

[0087] In this embodiment, within the effective interaction time interval, consecutive frame images are read from local camera data in chronological order. A pixel-by-pixel difference operation is performed on the pixel matrix of adjacent frames after 8-bit grayscale processing to obtain an inter-frame grayscale difference matrix. Simultaneously, a noise suppression threshold of 5 grayscale levels is set, and pixels with absolute differences less than the threshold are set to zero to weaken the influence of illumination perturbations on the difference results. This ultimately forms a set of time-series grayscale difference matrices that only reflect significant motion changes. If the pixel size of the nursing execution target box and the target box to be cared for in the current frame is not less than 60×120 pixels, then a local region of the corresponding target box is extracted from the grayscale difference matrix of each frame as a grayscale change sub-matrix. The absolute differences of non-zero pixels within the sub-matrix are accumulated to obtain the change per unit time. Simultaneously, the number of consecutive non-zero change frames is counted as the change duration parameter, and the sub-matrix is ​​divided into 8×8 grids. The proportion of changed pixels in each grid is calculated to construct a distribution parameter vector for the concentrated change region. If the effective interaction time contains no less than 40 differential frames, the cumulative change sequence, duration parameter and regional distribution vector of each moment are spliced ​​together in chronological order, and the mean and peak value within each time slice are calculated in 10-frame slices. The entire time series is then normalized so that each feature component falls into the [0,1] interval, and finally an interaction behavior feature vector containing information on intensity change, duration change and spatial distribution evolution is formed.

[0088] Optionally, the adaptation of the current nursing behavior type determined in the nursing behavior recognition module includes:

[0089] Extract the initial body surface state parameters corresponding to the start time of the effective behavior interaction time and the end body surface state parameters corresponding to the end time of the effective behavior interaction time from the body surface state features.

[0090] In this embodiment, the starting moment of the effective behavioral interaction time is captured. Calculate the average body surface temperature within the infrared temperature sub-region corresponding to the target being cared for. Temperature standard deviation and temperature gradient mean , as the initial body surface state parameters; at the end time Extract corresponding temperature sub-regions within the same spatial range and calculate the average body surface temperature. Temperature standard deviation and temperature gradient mean This forms the final body surface state parameters, thereby ensuring the consistency of the two sets of parameters in terms of spatial scale and statistical method.

[0091] Calculate the difference between the initial and final body surface state parameters to obtain the actual state change parameters corresponding to the current nursing behavior;

[0092] In a further embodiment, calculations are performed based on the initial body surface state parameters and the final body surface state parameters, respectively. , and The difference is normalized by absolute value so that each change component falls into the [0,1] interval; at the same time, the minimum effective change threshold is set to 0.02, and the change components below the threshold are marked as having no significant change. Finally, a parameter vector of actual state change is formed to describe the temperature rise of the body surface, the change in stability and the trend of local force change.

[0093] The expected state change range corresponding to the current nursing behavior type is retrieved from the nursing behavior tag library. The matching degree value of each actual state change parameter falling into the expected state change range is calculated to obtain the suitability of the current nursing behavior type.

[0094] In a further embodiment, based on the pre-stored expected state change ranges for the current nursing behavior type in the nursing behavior tag library, such as temperature change range [0.3℃, 1.2℃], dispersion change range [-0.1, 0.2], and gradient change range [0.05, 0.4], each actual state change parameter is compared with the corresponding range one by one, and the position ratio within the range is calculated as the single-item matching degree; further, a weighted sum is performed with a temperature change weight of 0.5, a dispersion weight of 0.3, and a gradient weight of 0.2 to obtain the overall suitability value of the current nursing behavior type.

[0095] Optionally, the auxiliary prompt module can filter the following nursing behavior tags:

[0096] Retrieve the connectable nursing behavior tags for the current nursing behavior type from the nursing behavior tag library, and retrieve the applicable object categories and corresponding trigger condition parameters for each connectable nursing behavior tag;

[0097] In this embodiment, the connectable set of the current nursing behavior type is retrieved from the connectability field in the nursing behavior tag library, and the object category code (such as bedridden patient, postoperative patient) and trigger condition parameter vector corresponding to each tag are read simultaneously. The trigger condition parameter vector includes at least a body surface temperature threshold. Stability threshold and local temperature rise rate threshold This is to form the structured parameter set required for subsequent matching.

[0098] If the adaptability of the current nursing behavior type is greater than the preset adaptability threshold, the latest body surface status feature parameters of the target to be cared for at the end of the current nursing behavior are obtained, and the latest body surface status feature parameters are compared with the trigger condition parameters of the applicable object category corresponding to the target to be cared for in order to calculate the trigger matching degree value of each connectable nursing behavior label, and then select the connectable nursing behavior label with the highest trigger matching degree value as the connectable nursing behavior label.

[0099] In a further embodiment, the fit value of the current nursing behavior type The threshold is set to 0.75, and the result is calculated in real time. Then at the end of the current nursing care activity Read the latest body surface temperature from the body surface state feature cache. Stability and temperature rise rate It then compares each of these parameters with the trigger condition parameters of each connectable nursing behavior label to calculate... , and The normalized distance is calculated and weighted by temperature (0.4), stability (0.35), and temperature rise rate (0.25) to form a trigger matching value. The label with the highest matching value is selected as the label for connecting nursing behavior.

[0100] If the suitability of the current nursing behavior type is less than or equal to the suitability threshold, the latest body surface state feature parameters of the target to be cared for at the current moment are obtained, and the latest body surface state feature parameters are compared with the corrective nursing behavior tags corresponding to the applicable object category of the target to be cared for in the nursing behavior tag library to calculate the trigger matching degree value of each corrective nursing behavior tag, and then the corrective nursing behavior tag with the highest trigger matching degree value is selected as the connecting nursing behavior tag.

[0101] In another embodiment, if the fit value of the current nursing behavior type ,For example Then, retrieve the set of corrective nursing behavior tags corresponding to the category of the target object to be cared for from the nursing behavior tag library, and at the current moment... Read the latest body surface status characteristic parameters and the latest body surface temperature. Stability and temperature rise rate For each corrective nursing behavior label, the same normalized distance calculation method is used to obtain the corrective trigger matching degree value, and an additional safety weight coefficient of 0.2 is introduced to amplify the impact of abnormal physical signs deviation. Finally, the corrective nursing behavior label with the highest trigger matching degree value is selected as the connecting nursing behavior label to guide the corrective execution of subsequent nursing processes.

[0102] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0103] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A smart medical care interactive platform system based on image processing technology, characterized in that, Includes the following modules: The data acquisition module includes a camera group and an infrared sensor group pre-deployed in the ward area; the camera group and the infrared sensor group respectively collect local video data and infrared sensor data of the ward area. The target recognition module identifies the target to be cared for and the target to be cared for based on the edge contour information in the local camera data, and determines the effective behavioral interaction time between the target to be cared for and the target to be cared for by combining infrared sensor data; The nursing behavior recognition module compares the interactive behavior characteristics within the effective interaction time with preset nursing behavior tags to identify the current nursing behavior type and determine the suitability of the current nursing behavior type. The auxiliary prompt module filters connecting nursing behavior tags from nursing behavior tags based on the adaptability of the current nursing behavior type, and sends the connecting nursing behavior tags to the medical and nursing terminals corresponding to the nursing execution goals to prompt the nursing execution goals.

2. The intelligent medical care interactive platform system based on image processing technology according to claim 1, characterized in that, The data acquisition module collects local camera data and infrared sensor data of the ward area, including: Local camera data from both sides of the bed is acquired by pre-deployed camera groups on both sides of the bed in the ward area; infrared sensor data from the bed area is acquired by pre-deployed infrared sensor groups directly above the geometric center point of the ward area. Data preprocessing is performed on the local camera data and the infrared sensor data respectively.

3. The intelligent medical care interactive platform system based on image processing technology according to claim 1, characterized in that, The target recognition module identifies the following targets requiring care: Detect the grayscale distribution and edge gradient of each frame in local camera data to identify the outlines of candidate bodies and beds within the ward area; The closed area and aspect ratio of each candidate body contour are calculated separately to serve as the geometric features of the candidate body contour. The geometric features of each candidate body contour are then compared with the body shape features of patients in the electronic medical records of the corresponding beds in the ward area of ​​the medical management platform to screen for similar human body contours of patients. Based on the spatial relationship between the patient's similar human body contour and the contour of the hospital bed, nursing targets are selected.

4. The intelligent medical care interactive platform system based on image processing technology according to claim 3, characterized in that, Identifying candidate body contours within the ward area includes: The grayscale distribution of each frame image is equalized and corrected, and several grayscale regions are divided according to the average grayscale value of adjacent pixels in each frame image after equalization and correction. The gradient magnitude and gradient direction of each pixel are extracted based on the edge gradient in each frame image, and several closed edge contour regions are divided according to the similarity of the gradient magnitude and the consistency of the gradient direction of adjacent pixels. Overlay closed edge contour areas and grayscale areas. If the number of grayscale areas contained within the closed edge contour area falls within the preset human body grayscale distribution range, and the area of ​​the largest grayscale area is higher than 50%, then the corresponding closed edge contour area will be used as the candidate human body contour.

5. The intelligent medical care interactive platform system based on image processing technology according to claim 3, characterized in that, Screening for care targets includes: Establish a bed coordinate system based on the bed outline, and calculate the projection distance and height distribution parameters of the patient's similar human body outline relative to the center line of the bed based on the spatial position relationship of the patient's similar human body outline in the bed coordinate system. The target box corresponding to the patient's similar human body contour, whose projection distance and height distribution parameters both fall within the preset threshold range of the patient's resting area, is used as the nursing target.

6. The intelligent medical care interactive platform system based on image processing technology according to claim 1, characterized in that, The target identification module identifies nursing execution targets including: Based on the grayscale distribution of each frame in the local camera data, the barcode region in the image is identified, and the pixels within the barcode region are subjected to binarization enhancement and stripe width normalization processing to determine the barcode information; The barcode information is matched with the nursing staff identification codes in the medical and nursing management platform to obtain the physical characteristics of the nursing staff; Based on the physical characteristics of nursing staff, target boxes corresponding to similar human body contours of nursing staff are selected from the candidate body contours and used as nursing execution targets.

7. The intelligent medical care interactive platform system based on image processing technology according to claim 1, characterized in that, The target recognition module, by combining infrared sensor data, determines the effective behavioral interaction time between the nursing execution target and the target to be cared for, including: Align the surface temperature value of each pixel in the infrared sensor data with the target box corresponding to the nursing execution target and the target box corresponding to the target to be cared for, and determine the infrared temperature sub-region of the target box corresponding to each target. The temperature change amplitude and temperature gradient distribution of the infrared temperature sub-region corresponding to the nursing execution target are used as the thermal motion characteristics of the nursing execution target; at the same time, the temperature stability parameters and local temperature rise rate in the infrared temperature sub-region corresponding to the nursing target are calculated as the surface state characteristics of the nursing target. Calculate the synchronization coefficient and spatial proximity of temperature changes between the thermal motion characteristics of the nursing execution target and the surface state characteristics of the target at adjacent time points; If the temperature change synchronization coefficient is higher than the preset interaction threshold and the spatial proximity falls within the preset nursing operation distance range, then it is determined that there is an effective behavioral interaction between the nursing execution goal and the nursing goal to be cared for, so as to determine the effective behavioral interaction time.

8. The intelligent medical care interactive platform system based on image processing technology according to claim 1, characterized in that, The nursing behavior recognition module identifies the current nursing behavior type, including: By utilizing the grayscale changes within the effective behavioral interaction time in local camera data, the interactive behavioral characteristics between nursing execution goals and nursing objectives can be determined. The interaction behavior features are matched one by one with the nursing behavior reference standard features corresponding to each nursing behavior in the preset nursing behavior label library. The similarity value between the current interaction behavior feature and each nursing behavior reference standard feature is calculated to determine the current nursing behavior type.

9. The intelligent medical care interactive platform system based on image processing technology according to claim 1, characterized in that, The nursing behavior recognition module determines the suitability of the current nursing behavior type, including: Extract the initial body surface state parameters corresponding to the start time of the effective behavior interaction time and the end body surface state parameters corresponding to the end time of the effective behavior interaction time from the body surface state features. Calculate the difference between the initial and final body surface state parameters to obtain the actual state change parameters corresponding to the current nursing behavior; The expected state change range corresponding to the current nursing behavior type is retrieved from the nursing behavior tag library. The matching degree value of each actual state change parameter falling into the expected state change range is calculated to obtain the suitability of the current nursing behavior type.

10. The intelligent medical care interactive platform system based on image processing technology according to claim 1, characterized in that, The auxiliary prompt module includes the following tags for filtering transitional nursing behaviors: Retrieve the connectable nursing behavior tags for the current nursing behavior type from the nursing behavior tag library, and retrieve the applicable object categories and corresponding trigger condition parameters for each connectable nursing behavior tag; If the adaptability of the current nursing behavior type is greater than the preset adaptability threshold, the latest body surface status feature parameters of the target to be cared for at the end of the current nursing behavior are obtained, and the latest body surface status feature parameters are compared with the trigger condition parameters of the applicable object category corresponding to the target to be cared for in order to calculate the trigger matching degree value of each connectable nursing behavior label, and then select the connectable nursing behavior label with the highest trigger matching degree value as the connectable nursing behavior label. If the suitability of the current nursing behavior type is less than or equal to the suitability threshold, the latest body surface state feature parameters of the target to be cared for at the current moment are obtained, and the latest body surface state feature parameters are compared with the corrective nursing behavior tags corresponding to the applicable object category of the target to be cared for in the nursing behavior tag library to calculate the trigger matching degree value of each corrective nursing behavior tag, and then the corrective nursing behavior tag with the highest trigger matching degree value is selected as the connecting nursing behavior tag.

Images