Newborn health assessment method and system based on visual analysis

Abstract

The invention relates to the technical field of health assessment, in particular to a newborn health assessment method and system based on visual analysis, and the method comprises the following steps: obtaining a newborn image frame sequence, extracting a hue curve, recognizing a variation region to generate a layer, and extracting a mutation region bitmap group in combination with a heart rate RR interval difference value; and analyzing an included angle mapping grid between the center of gravity of the pigment and the heart rate slope, superposing a respiratory rate curvature, performing co-occurrence clustering to form a trend graph block, and evaluating a risk distribution interface formed by joint mutation. According to the method, a dynamic coupling relation between an image and a physiological signal is established through slope direction angle change and included angle deviation, cross-dimension mapping of heart rate trend change and visual feature change is achieved, the heart rate, skin color tracks and curvature change of respiratory frequency are fused, a trend gathering map is constructed, and the dynamic coupling relation between the image and the physiological signal is obtained. The risk judgment and display are completed according to the joint grading of the block density, the time span and the parameter quantity in the atlas, and the health level fluctuation trend is visually presented in the risk evolution sorting.

Description

Technical Field

[0001] The present invention relates to the technical field of health assessment, and in particular to a method and system for neonatal health assessment based on visual analysis. Background Art

[0002] The field of health assessment technology involves methods and tools for systematically analyzing and evaluating multiple health states, including physiological, psychological, and developmental. This field integrates biomedical engineering, physiological signal processing, data analysis, artificial intelligence, and other means to achieve a quantitative assessment of human health levels. The assessment content includes multiple dimensions such as vital signs, growth and development indicators, neurobehavioral responses, physiological signal characteristics, and biochemical indicators. It aims to provide early warning of individual health status, risk identification, and subsequent intervention recommendations through standardized and objective testing and analysis. This technical field can be applied in scenarios such as clinical diagnostic support, public health monitoring, and family health management, and has key characteristics such as high precision, repeatability, and real-time feedback.

[0003] The newborn health assessment method aims to comprehensively assess a newborn's health status by collecting and analyzing key physiological and behavioral parameters within a specific time period after birth. This method is used to screen for potential health abnormalities, identify individuals whose development deviates from the normal range, and provide data support for clinical intervention, nursing decision-making, and health management. Application scenarios include medical service systems such as obstetric wards, pediatric outpatient clinics, and neonatal intensive care units, as well as the continuous assessment of newborn health status in subsequent maternal and child health tracking systems.

[0004] Traditional assessment methods are weak in constructing dynamic correlations between indicators. They usually use static feature comparisons or statistical analysis based on time period averages, which makes it difficult to capture high-frequency change features or instantaneous linkage changes between indicators. In particular, there is a lack of clustering mechanism in the identification of multi-parameter intersection periods, resulting in the inability to effectively present trend aggregation phenomena before health risks occur. For example, when facial skin color variations and heart rate fluctuations occur at the same time but at different times, existing analysis methods find it difficult to establish cross-signal linkage judgments, affecting the sensitive capture of early abnormal signals, and easily causing health status assessment deviations or risk identification delays. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the existing technology and propose a newborn health assessment method and system based on visual analysis.

[0006] To achieve the above-mentioned purpose, the present invention adopts the following technical solution: a method for evaluating the health of newborns based on visual analysis, comprising the following steps: S1: Obtain a sequence of image frames from the newborn's face and chest, mark the pixel areas where the pixel change rate is greater than the HSV saturation change threshold as variation areas, and superimpose the variation areas of the image frames in each time period to generate a layer of significant pigment variation segments; S2: calling the pigment variation significant segment layer, extracting the ECG sampling sequence of the corresponding time, calculating the RR interval difference of the heart rate at consecutive time points, extracting the time period where the change rate exceeds the heart rate variability threshold, and obtaining the heart rate trend mutation area map group; S3: According to the start and end time of the mutation segment in the heart rate trend mutation area map group, select image frames of the corresponding time period in the pigment variation significant segment layer, mark the angle difference between the two as the angle offset, and map them into grid nodes in the two-dimensional coordinates to generate a synchronous drift interlaced mapping grid; S4: Based on the synchronized drift interlaced mapping grid, the curvature change points of the respiratory rate per minute are extracted, the time marks of the intersection of the heart rate and skin color trajectories are aligned, and the time periods where significant trend changes occur in the three indicators are co-occurred and clustered. The co-occurrence areas are marked as trend fusion blocks to generate a cross-parameter trend cluster map.

[0007] As a further solution of the present invention, the pigment variation significant segment layer includes the significant saturation change area in the image frame, the variation pixel distribution boundary, the color level change interval label, the change area time index point and the layer overlay transparency parameter; the heart rate trend mutation area map group includes the mutation start and end time period, the heart rate slope change mark, the RR interval change annotation point, the direction transition position node and the trend map group index order; the synchronous drift interlaced mapping grid is specifically the heart rate change direction angle mapping point, the skin color area center of gravity displacement trajectory node, the angle offset distribution hot zone, the grid level division rule and the heat weight density factor; the cross-parameter trend aggregation map is specifically the respiratory frequency curvature fluctuation segment, the three-parameter overlapping time block, the trend same-direction aggregation area, the map transparency mapping layer and the aggregation segment classification identification information.

[0008] As a further solution of the present invention, the steps for obtaining the layer of the region with significant pigment variation are specifically as follows: S101: Acquire a sequence of image frames in the facial and chest acquisition areas of the newborn, extract the pixel values of each pixel in the image frame under the three RGB channels, and record the brightness change value and saturation change value of the pixel in the time series according to the pixel index. Call the time index of the image frame to perform inter-frame difference calculation on the brightness value and saturation value at the same pixel position to generate brightness and saturation difference information; S102: Based on the brightness and saturation difference information, according to the change rate data of each pixel point, an HSV saturation change threshold is set, pixels whose change rate exceeds the threshold are screened, the spatial coordinate positions of the pixels are extracted, and the pixels are integrated into a pixel change layer within a continuous time period according to the image frame index to generate a high-variability pixel area labeling group; S103: Call the high-variability pixel area marking map group, superimpose the pixel positions of the layers in each time period, perform statistical calculations on the number of repeated markings at the same spatial coordinate position, draw a variation frequency superposition map, output the image area layer with marking frequency information, and generate a pigment variation significant segment layer.

[0009] As a further solution of the present invention, the steps for obtaining the heart rate trend mutation area map group are specifically as follows: S201: calling the pigment variation significant segment layer, extracting the ECG sampling sequence corresponding to each time point, calculating the time interval between adjacent R waves, recording a continuous RR interval sequence, and performing a derivative operation on the time point difference of the sequence to obtain the RR interval change rate per unit time, screening the time period where the change rate exceeds the standard heart rate variability threshold, and generating a heart rate mutation interval sequence; S202: Based on the heart rate mutation interval sequence, according to the RR interval value in each mutation time period, the sign changes of the consecutive RR differences are extracted in chronological order, and the time periods where the direction of the rate of change changes are changed are selected as inflection points. A position marker structure is established according to the time label, and the corresponding slope sign change position is recorded to generate a slope direction transition segment set; S203: Call the slope direction transition segment set, arrange the RR interval change values of each segment in an overlapping manner, establish a trend segment group graph structure based on the time axis, count the repeated direction transition positions and mark the intensity density areas, and generate a heart rate trend mutation area map group.

[0010] As a further solution of the present invention, the step of acquiring the synchronous drift staggered mapping grid is specifically as follows: S301: Based on the start and end times of the mutation segment in the heart rate trend mutation area map group, select image frames of the corresponding time period in the pigment variation significant segment layer, extract a set of pixels in the variation area in each frame, calculate the spatial center of gravity position of the variation area based on the two-dimensional coordinates of each pixel position in the set, obtain the vector path of the center of gravity coordinates in the time dimension in consecutive frames, and generate center of gravity displacement vector trajectory information; S302: Retrieving the centroid displacement vector trajectory information, extracting the RR interval difference within the corresponding mutation time period in the heart rate trend mutation area map group based on the spatial change path of each time period, calculating the average change rate of the RR interval per unit time in the mutation time period, and constructing the heart rate change direction angle vector. Obtaining the angle difference between the centroid path vector and the heart rate direction vector, calculating the angle offset, screening nodes with angle offset values greater than the offset tolerance threshold, and generating a set of significant angle offset points. S303: Based on the angle offset significant point concentration, the image frame number and coordinate point corresponding to each frame are extracted, the significant angle position is mapped to a two-dimensional plane grid, the number of times each grid node is mapped is counted, the color heat level is generated according to the cumulative number of nodes, and a synchronous drift interlaced mapping grid is established.

[0011] As a further solution of the present invention, the formula for obtaining the angle offset is specifically: ; in, Indicates the angle offset between the center of gravity path and the heart rate direction vector, Represents the normalized value of the displacement vector of the centroid of the mutation area, Represents the normalized value of the RR interval change direction vector, represents the heart rate change rate gain parameter, Indicates the offset of the center of gravity change value between image frames in the current time period.

[0012] As a further solution of the present invention, the steps for obtaining the cross-parameter trend cluster map are specifically as follows: S401: Based on the synchronous drift interleaved mapping grid, according to the node area whose heat value is higher than the heat threshold, extract the time index range covered by the node, call the original sampling curve of the respiratory rate, align the corresponding segments according to time consistency, discretize the respiratory rate value per minute in the time period, calculate the first-order derivative difference of adjacent change points, extract the extreme value point as the curvature change node, and generate a respiratory rate curvature change point set; S402: calling the respiratory rate curvature change point set, matching the overlapping time points of the heart rate mutation segment and the pigment trajectory offset segment on the same time axis, aligning the trend mutation nodes of the three indicators by time to form a co-occurrence structure, performing aggregation processing by time window in the co-occurrence structure, counting the number of simultaneous mutations of the three parameters in each time window and marking the position nodes, calculating to obtain the trend fusion density, marking the segment where the trend fusion density exceeds the density benchmark value as a trend fusion block area, and generating a trend fusion block identification set; S403: Based on the trend fusion tile identification set, according to the time axis distribution of each identification segment, it is mapped to a unified two-dimensional atlas matrix structure, the trend fusion density corresponding to each tile is assigned as the opacity mapping factor to the layer rendering channel, the tile boundaries are marked and the atlas coordinate axis structure is drawn to establish a cross-parameter trend aggregation atlas.

[0013] As a further solution of the present invention, the formula for calculating the trend fusion density is specifically: ; in, It represents the trend fusion density, which is used to evaluate the degree of aggregation of multi-parameter trend covariation within the time window. Indicates the The normalized value of the heart rate variation in a time window, Indicates the The normalized value of the change in respiratory frequency curvature within a time window, Indicates the The sensitivity factor for responding to overlapping trend events within a time window, Indicates the The normalized value of the displacement amplitude of the pigment trajectory within a time window, Indicates the Normalized value of the pigment distribution diffusion range within a time window, Indicates the The number of overlapping trend mutation events in a time window, is the total number of time windows.

[0014] As a further embodiment of the present invention, the method further comprises the following steps: S5: Based on the cross-parameter trend cluster map, the blocks whose joint mutation index exceeds the risk ladder division threshold are set as high-risk units, and the display frames corresponding to the risk levels are filled with segmented color ribbons. The risk evolution trends are sorted and plotted according to the timeline sequence to generate visual diagnostic display information of the risk ladder; The risk ladder visual diagnosis display information includes a risk level strip structure, a risk unit color band overlay diagram, risk evolution sequence nodes, graphic area weight marks and a risk ladder horizontal grading axis.

[0015] As a further solution of the present invention, the steps for obtaining the risk ladder visual diagnosis display information are specifically as follows: S501: Based on the cross-parameter trend cluster map, extract the time span value, trend fusion density level, and number of participating physiological indicators of each block according to the start and end positions of the time axis of each block, set the trend density threshold and the lower limit of the number of participating parameters, filter each block, mark the blocks where the change of combined indicators exceeds the filtering conditions as high-risk sections, and generate a high-risk block identification mark set; S502: The high-risk tile identification mark set is called, and the tiles are sequentially numbered in chronological order according to the time index sequence and risk level mark of each tile. Color band codes are assigned according to each risk level, and the tiles corresponding to each risk level are drawn as a color segment structure to form a continuous horizontal axis color band combination area, thereby generating risk color band coding structure information; S503: According to the risk color band coding structure information, call the time index and color level of each color band segment, build a two-dimensional visualization area layer structure, define the color band transparency mapping relationship, use the risk level value as the transparency input, adjust the tile area opacity coefficient, overlay the risk tiles to form a display area in the coordinate plane, and establish risk ladder visual diagnostic display information.

[0016] A neonatal health assessment system based on visual analysis, wherein the neonatal health assessment system based on visual analysis is used to implement the above-mentioned neonatal health assessment method based on visual analysis, and the system comprises: The pigment variation recognition module obtains a sequence of image frames from the newborn's face and chest, marks pixel areas where the pixel change rate is greater than the HSV saturation change threshold as variation areas, and superimposes the variation areas of the image frames in each time period to generate a layer of sections with significant pigment variation. The heart rate mutation identification module calls the pigment variation significant segment layer, extracts the electrocardiogram sampling sequence of the corresponding time, calculates the RR interval difference of the heart rate at consecutive time points, extracts the time period where the change rate exceeds the heart rate variability threshold, and obtains the heart rate trend mutation area map group; The drift interleaving mapping module selects image frames of corresponding time periods in the pigment variation significant segment layer according to the start and end times of the mutation segment in the heart rate trend mutation segment map group, marks the angle difference between the two as an angle offset, and maps them into grid nodes in two-dimensional coordinates to generate a synchronous drift interleaving mapping grid; The parameter trend aggregation module extracts the curvature change points of the respiratory rate per minute based on the synchronized drift interlaced mapping grid, aligns the time marks of the intersection of the heart rate and skin color trajectories, and performs co-occurrence clustering on the time periods where significant trend changes occur in the three indicators. The co-occurrence areas are marked as trend fusion blocks to generate a cross-parameter trend aggregation map; The risk ladder visualization module sets the blocks whose joint mutation indicators exceed the risk ladder division threshold as high-risk units based on the cross-parameter trend aggregation map, uses segmented color ribbons to fill the display frame corresponding to the risk level, sorts and draws the risk evolution trend in timeline order, and generates risk ladder visual diagnostic display information.

[0017] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by extracting and comparing the pixel hue change curve and brightness saturation gradient in the newborn's facial and chest image frame sequence, the local variation area can be accurately calibrated, and its synchronous variation with the heart rate RR interval difference is used as the trend mutation positioning benchmark. The dynamic coupling relationship between the image and the physiological signal is established through the slope direction angle change and the angle offset, and the cross-dimensional mapping of the heart rate trend change and the visual feature change is realized. By introducing the heat statistics mechanism, the covariation time period of multiple indicators is aggregated in the grid node, and the curvature changes of the heart rate, skin color trajectory and respiratory rate are integrated to construct a trend aggregation map. The risk judgment and display are completed based on the joint classification of the block density, time span and number of parameters in the map, and the health level fluctuation trend is intuitively presented in the risk evolution ranking. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a schematic diagram of the main steps of the present invention; Figure 2 This is a schematic diagram of the refinement of S1 of the present invention; Figure 3 This is a schematic diagram of the refinement of S2 of the present invention; Figure 4 This is a schematic diagram of the refinement of S3 of the present invention; Figure 5 This is a schematic diagram of the refinement of S4 of the present invention; Figure 6 This is a detailed schematic diagram of S5 of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0021] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0022] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0023] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0024] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0025] See also Figure 1 The present invention provides a technical solution: a method for evaluating the health of newborns based on visual analysis, comprising the following steps: S1: Obtain a sequence of image frames from the newborn's face and chest, extract the hue change curve of each pixel in the RGB channel on the time axis, calculate the gradient change of each pixel's brightness and saturation values between consecutive frames, and use the change rate as the division basis. Pixel areas with a change rate greater than the HSV saturation change threshold are marked as variation areas. The variation areas of the image frames in each time period are superimposed to generate a layer of sections with significant pigment variation. The saturation value in the HSV color space represents the purity of the color, and the change threshold can be defined as the rate of change of the saturation value component exceeding the set standard per unit time; S2: Call the pigment variation significant segment layer, extract the ECG sampling sequence of the corresponding time, calculate the RR interval difference of the heart rate at consecutive time points, extract the time period where the change rate exceeds the heart rate variability threshold, establish the mutation segment range, and perform direction judgment on the heart rate curve segment within the mutation time period. Extract the slope segment where the change rate sign changes, superimpose it to form a graph group structure, and obtain the heart rate trend mutation area map group; Heart rate variability can be expressed by indicators such as standard deviation and root mean square deviation. A sudden change is defined as a change of two consecutive RR intervals exceeding ±20ms; S3: Based on the start and end times of the mutation segments in the heart rate trend mutation area map group, select image frames of the corresponding time period in the pigment variation significant segment layer, calculate the change vector of the centroid coordinate position of the mutation area on the time axis, and extract the corresponding slope direction angle within the heart rate variation segment. The angle difference between the two is marked as the angle offset and mapped into grid nodes in the two-dimensional coordinate system. The number of angle nodes is counted and the color intensity is plotted to generate a synchronized drift interlaced mapping grid. The centroid coordinate position refers to the geometric center point after weighted average of the pixels in the annotated area in the image, and the angle between the two is the physical representation of the dynamic offset between the parameters; S4: Based on a synchronized drift interlaced mapping grid, the respiratory rate curve's change path within the corresponding time period is superimposed based on the grid's high-heat areas. The curvature change points of the respiratory rate per minute are extracted, and the time stamps of the intersections of the heart rate and skin color trajectories are aligned. Time periods with significant trend changes in the three indicators are co-occurrence clustered. The co-occurrence regions are marked as trend fusion tiles, and the density is distinguished by opacity. This constructs a map framework and generates a cross-parameter trend cluster map. Respiratory rate is a routine clinical monitoring indicator, and the atlas structure is formed based on the spatiotemporal coupling of the three covariant parameters; S5: Based on the cross-parameter trend aggregation map, the time span, density classification, and number of participating parameters of each block are called. Blocks whose joint mutation indicators exceed the risk ladder division threshold are set as high-risk units. The display frame corresponding to the risk level is filled with segmented color ribbons. The risk evolution trend is sorted and mapped according to the time axis sequence. The risk distribution interface under multi-parameter fusion is constructed to generate visual diagnostic display information of the risk ladder. The risk tiering threshold can be set based on any two of the three indicators changing by more than 20% simultaneously within the same time period and for more than 60 seconds. The display chart uses a color gradient (e.g., green → yellow → orange → red) to express the risk level hierarchy. The pigment variation significant segment layer includes the significant saturation change area in the image frame, the variation pixel distribution boundary, the color level change interval label, the change area time index point and the layer overlay transparency parameter. The heart rate trend mutation area map group includes the mutation start and end time period, the heart rate slope change mark, the RR interval change annotation point, the direction transition position node and the trend map group index order. The synchronous drift interlaced mapping grid specifically includes the heart rate change direction angle mapping point, the skin color area center of gravity displacement trajectory node, the angle offset distribution hot zone, the grid level division rule and the heat weight density factor. The cross-parameter trend aggregation map specifically includes the respiratory rate curvature fluctuation segment, the three-parameter overlapping time block, the trend same-direction aggregation area, the map transparency mapping layer and the aggregation segment classification identification information. The risk ladder visual diagnosis display information includes the risk level strip structure, the risk unit color band overlay map, the risk evolution sequence node, the graphic area weight mark and the risk ladder horizontal grading axis.

[0026] See also Figure 2 The specific steps for obtaining the layer of the segment with significant pigment variation are as follows: S101: Acquire a sequence of image frames in the facial and chest acquisition areas of the newborn, extract the pixel values of each pixel in the image frame under the three RGB channels, and record the brightness change value and saturation change value of the pixel in the time series according to the pixel index. Call the time index of the image frame to perform inter-frame difference calculation on the brightness value and saturation value at the same pixel position to generate brightness and saturation difference information; When acquiring the image frame sequence in the acquisition area of the newborn's face and chest, 30 consecutive frames of images can be used as a time window. In the continuous monitoring device for newborn vital signs, the RGB video signal collected at 10 frames per second is extracted separately with a resolution of 1920×1080. A three-channel structure is established for each pixel to record its R, G, and B values. In the time dimension, for the coordinate point position The R channel values of the pixels are 130, 135, 145, and 150, respectively, and the corresponding time points are , , , , first calculate the brightness difference of adjacent frames, brightness use Brightness sequence is obtained by conversion , respectively 139.6, 143.2, 148.9, 150.3, and then further calculate the brightness difference 、 、 , and similarly for saturation After HSV transformation, extract Component values are used to calculate the inter-frame difference, assuming that the pixel is Saturation 0.36, 0.42, 0.38, 0.40, calculate the inter-frame difference 、 、 , the brightness difference and saturation difference are recorded as the channel difference time series information of the current pixel point, and after completing the calculation of the time series brightness difference and saturation difference of 2073600 pixels in the entire image frame, the inter-frame brightness and saturation differences of all pixels are aggregated into a two-dimensional array, and each pixel records the brightness difference sequence in the array Saturation difference sequence , the corresponding index time point is used as its horizontal dimension index, spatial coordinate As a vertical index, after completion, the array information is exported as the basis for subsequent mutation rate judgment to generate brightness and saturation difference information.

[0027] S102: Based on the brightness and saturation difference information and the change rate data of each pixel, an HSV saturation change threshold is set, pixels whose change rate exceeds the threshold are screened, the spatial coordinate positions of the pixels are extracted, and the pixel change layers within the continuous time period are integrated according to the image frame index to generate a high-variability pixel area labeling group; Based on the change rate data of each pixel in the brightness and saturation difference information, the image frame time interval is As a benchmark, the rate calculation is performed on any pixel difference information, for example, when Between two frames When , compare this rate with the set HSV saturation change threshold, which is set to This value is determined based on the statistical distribution of the saturation change rate of 95% of the pixels outside the variation area in the image frame. Specifically, 30 frames of image data are randomly extracted from the newborn in a stable state, and the saturation rate of the non-face area pixels in the 200×200 area in each frame is calculated. The average value is approximately , the standard deviation is approximately , taking the mean plus 1.5 times the standard deviation as the cutoff point, we can calculate , select all pixels whose change rate is greater than the threshold, extract the spatial coordinate positions of these pixels and record their frame index numbers, and construct the set of mutated pixels in the frame. Continuous marks in the frame are identified as stable variation areas. The layer records in all consecutive frames are formed into variation tracks, which are superimposed and stored in the form of layer indexes. The variation intensity spatial structure map group is output and the high variation pixel area mark map group is generated.

[0028] S103: Calling the high-variability pixel region marking map group, superimposing the pixel positions of the layers within each time period, performing statistical calculations on the number of repeated markings at the same spatial coordinate position, drawing a variation frequency overlay map, outputting an image region layer with marking frequency information, and generating a pigment variation significant segment layer; Call each frame layer in the high-variability pixel area marking group, count the mutated pixel sets contained in each frame, perform coordinate matching processing on the pixels at the same spatial position in each frame, and calculate the pixel Set the number of times it appears in all frames ,like It is recorded as a repeated variation point, and the threshold is set as the frame overlap statistical threshold According to the pixel variation frequency distribution of the target area in the 30-frame sliding window, the distribution of the number of occurrences in consecutive frames is skewed, of which more than 70% of repeated pixels fall more than 3 times. Therefore, 3 times is selected as the basis for the minimum marking intensity. The sliding window mechanism is used for statistics. Each group of 30 frames of data is used as the processing unit. The variation frequency matrix is constructed according to the spatial dimension within the period. The matrix elements are , represents the number of variation frames of the corresponding pixel in this time period, all The value is mapped to the image grayscale image, and the grayscale value is set to The layer generated after mapping is an overlay map, which converts the grayscale values of different areas into different color channels for visual expression. Finally, a spatial image area with fused statistical counts is established to indicate high-frequency areas with repeated mutations, and generate a layer of significant pigment variation segments.

[0029] See also Figure 3 The specific steps for obtaining the heart rate trend mutation area map group are as follows: S201: Call the pigment variation significant segment layer, extract the ECG sampling sequence corresponding to each time point, calculate the time interval between adjacent R waves, record the continuous RR interval sequence, and perform derivative operation on the time point difference of the sequence to obtain the RR interval change rate per unit time. Filter the time period where the change rate exceeds the standard heart rate variability threshold to generate the heart rate mutation interval sequence; Call the corresponding time index range in the pigment variation significant segment layer, and extract the corresponding time window according to the frame number mark recorded in the layer. Generally, the corresponding ECG sampling sequence is extracted in units of 5 seconds, and the sampling frequency is , collect 2500 data points in 5 seconds. First locate the peak of the R wave, and use the threshold judgment method to extract the local maximum position. Set the R wave recognition threshold to the amplitude of the original voltage signal exceeding The position of the R wave is extracted, and the time interval between adjacent R waves is calculated to obtain the RR interval sequence. For example, to extract a sequence of RR intervals: , , , , then the first-order difference is calculated as , , , further perform derivative operation on the difference sequence to obtain the RR interval change rate of each segment. Divide the difference by the unit sampling period , get the RR change rate sequence. Set the heart rate variability threshold to The threshold is set based on the statistical results of the average RR interval derivative values of 200 newborns in the quiet period. The derivative value corresponding to the 95th percentile is taken as the judgment standard. The time period is screened and the start and end index ranges are recorded to form a mutation interval. For example, if If the threshold is exceeded, it is recorded as the starting point of the mutation segment and a heart rate mutation interval sequence is generated.

[0030] S202: Based on the heart rate mutation interval sequence, according to the RR interval value within each mutation time period, the sign changes of the consecutive RR differences are extracted in chronological order. The time periods where the direction of the rate of change changes are selected as inflection points are selected. A position marker structure is established according to the time label, and the corresponding slope sign change position is recorded to generate a slope direction transition segment set. Based on the RR interval data of each mutation time period in the heart rate mutation interval sequence, the difference sequence is arranged in chronological order to determine whether the positive and negative signs of two consecutive differences change. The change of positive and negative signs indicates that the heart rate change changes from acceleration to deceleration or vice versa, forming a directional jump point. Taking the RR sequence as an example, if , , , then from arrive When a sign change occurs, mark the time point as the directional inflection point. Repeat this operation for all mutation time periods, extract each positive and negative change point, record its time index and corresponding RR value, number the directional transition segments in sequence, and establish their relative position relationship in the continuous curve. Set an identification line on the time axis to indicate the location of the transition event. After extracting all directional conversion segments, merge them to generate a slope direction transition segment set.

[0031] S203: Calling the slope direction transition segment set, overlapping and arranging the RR interval change values of each segment, establishing a trend segment group graph structure based on the time axis, counting the repeated direction transition positions and marking the intensity density areas, and generating a heart rate trend mutation area map group; Call all time periods in the slope direction transition segment set, standardize the RR interval sequence of each time period, normalize the original RR sequence to the [0,1] interval and then superimpose and arrange it, align each segment along the time axis and construct a two-dimensional matrix diagram, with the horizontal axis as the time point index and the vertical axis as the RR interval value. Mark all transition events as discrete points in the matrix, count the number of direction transitions at the same time index as the repeated occurrence count item, and form an event occurrence density vector. Set the direction transition intensity density threshold to According to the density distribution of 95% of the frame positions in the RR direction switching event statistics, it can be observed that most densities are less than 3. If the cumulative number of direction transition events at a certain time point reaches or exceeds the threshold, the position is defined as a high-frequency transition position. The density peak position is plotted in the trend graph using color heat. The generated graphic structure displays the transition aggregation trend in the RR curve in the form of heat blocks, generating a heart rate trend mutation area map group.

[0032] See also Figure 4 ,The specific steps for obtaining the synchronous drift staggered mapping grid are: S301: Based on the start and end times of the mutation segment in the heart rate trend mutation area map group, select image frames of the corresponding time period in the pigment mutation significant segment layer, extract the pixel set of the mutation area in each frame, calculate the spatial center of gravity position of the mutation area based on the two-dimensional coordinates of each pixel position in the set, obtain the vector path of the center of gravity coordinates in the time dimension in consecutive frames, and generate center of gravity displacement vector trajectory information; According to the start and end time of each mutation segment recorded in the heart rate trend mutation area map group, the image frame sequence with the same time index in the pigment mutation significant segment layer is called. The acquisition frame rate is 10 frames per second. The duration of each mutation segment is set to 4 seconds, which corresponds to a total of 40 image frames. The pixel set of the mutation area is extracted in each frame and set in the frame In the , the set of identified variant pixels is 1325, and their pixel coordinates are recorded as , calculate the two-dimensional center of gravity position of all mutated pixels , using weighted average method , in is the number of mutated pixels in the current frame. For example, , and after substituting its coordinates into , , get each frame in turn on the entire 40-frame timeline , forming the vector trajectory of the center of gravity path of the variation area in time sequence, and constructing the vector sequence The trajectory vector is expressed as the displacement between two frames, namely: ,This trajectory is used for subsequent angle offset determination processing to generate center of gravity displacement vector trajectory information.

[0033] S302: Retrieving the centroid displacement vector trajectory information, extracting the RR interval difference within the corresponding mutation time period from the heart rate trend mutation area map group based on the spatial change path of each time period, calculating the average change rate of the RR interval per unit time in the mutation time period, and constructing the heart rate change direction angle vector. Obtaining the angle difference between the centroid path vector and the heart rate direction vector, calculating the angle offset, filtering out nodes with angle offset values greater than the offset tolerance threshold, and generating a set of significant angle offset points. The formula for obtaining the angle offset is: ; in, Indicates the angle offset between the center of gravity path and the heart rate direction vector, Represents the normalized value of the displacement vector of the centroid of the mutation area, Represents the normalized value of the RR interval change direction vector, Represents the heart rate change rate gain parameter, which is taken from the normalized value of the RR difference increment per unit time. Indicates the offset of the center of gravity change value between image frames in the current time period, expressed as the sum of the absolute differences of the consecutive center of gravity coordinate change values. represents the dot product, represents the vector modulus, represents the inverse cosine function, represents the natural logarithm function, Represents the square root operation; Call the center of gravity path vector of the continuous frame in the center of gravity displacement vector trajectory information , extract the heart rate mutation segment corresponding to its time period, and set the RR interval sequence as Calculate the difference in RR change rate per unit time as ,in , construct the direction vector Used to express the direction of heart rate change, the displacement vector of the frame in the same time period and RR direction vector To calculate the angle deviation, use the following formula: ; The structural design of this formula has three levels of logical relationships: Molecular part represents the directional consistency correction value, where It is the dot product of two normalized vectors, reflecting the cosine value of the angle between them. If the directions are consistent, the dot product is close to 1, and if the directions are opposite, it is negative. Multiply by Apply a dynamic gain of the heart rate change speed to the direction relationship, The larger the value, the more intense the RR change, and thus the weight of its influence on the angle offset increases.

[0034] Denominator represents the amplitude coordination and disturbance compensation terms, Represents the product of the modulus length of two vectors, used to standardize the dot product part, Indicates the compensation factor for the center of gravity change disturbance. If the visual change is drastic, it will offset part of the angle change to avoid overreaction.

[0035] The overall structure is an arccosine function, and the output is an angle with a value range of arrive , which essentially represents the degree of directional deviation between two changing trends (visual center of gravity and heart rate change direction) in the current time period, in degrees.

[0036] Calculation results The numerical meaning of is the degree of inconsistency between the change of the newborn's skin color center of gravity and the heart rate fluctuation direction in the current frame. If the angle is close to , indicating that the two directions are highly consistent. If they are close There is no correlation, if close , then there may be a completely opposite direction, that is, the physiological response is disconnected from the visual abnormal variation.

[0037] Example import: , , , ; ; ; ; ; ; ; The results show that in the current period, the direction of the change of the visual center of gravity is almost perpendicular to the RR change trend, which is a trend decoupling state. In this case, it can be identified as a potential abnormal response. and setting the deviation tolerance threshold By comparison, it is concluded that it is an abnormal offset frame and should be marked into the set of significant angle offset points.

[0038] The benefit of the formula is that it introduces as a dynamic adjustment term based on the RR gain, and As a disturbance compensation item, the offset angle calculation result not only considers directional consistency, but also combines multiple information of physiological fluctuation intensity and visual disturbance, which improves the sensitivity and robustness of offset judgment and helps the system identify risk state frames of trend separation.

[0039] S303: Based on the angle offset salient point concentration, the image frame number and coordinate point corresponding to each frame are extracted, the significant angle position is mapped to a two-dimensional plane grid, the number of times each grid node is mapped is counted, and a color heat level is generated based on the cumulative number of nodes, and a synchronous drift interlaced mapping grid is established; Based on the frame number and corresponding spatial coordinates of each salient point in the angle-shifted salient point set, these are mapped to a two-dimensional image region grid. Assuming the image space is divided into a 100×100 grid, the coordinates of each salient point fall into a corresponding grid cell, and the cumulative number of times the point falls into each cell is counted. For example, the cumulative number of times the point falls into the 43rd grid cell is 6, and the cumulative number of times the point falls into the 87th grid cell is 2. The cumulative number of times for all nodes is normalized to the range of 0.1 and then colored according to a color mapping table. Nodes with a cumulative number of 5 or more are defined as high-heat nodes and assigned a red color mapping level. Nodes with a cumulative number of 34 are defined as medium-heat nodes and assigned an orange-yellow color. The rest are defined as low-heat nodes. A two-dimensional matrix structure is constructed, and the color values are filled into the corresponding grid cells to form a complete heat distribution map. The layer structure is aligned to the temporal order based on the frame index overlay rule, creating a synchronized drift interlaced mapping grid.

[0040] Table 1 Synchronous drift interleaved heat mapping data table

[0041] As shown in Table 1, significant points of synchronous drift fall within multiple fixed spatial grids, constructing a spatial heat map that provides a spatial structural foundation for subsequent cross-indicator trend aggregation. The table shows the grid mapping and cumulative counts of representative points. This result indicates that areas of significant centroid drift are concentrated in a fixed number of grid areas, which are assigned higher risk weights in subsequent assessments. Ultimately, this numerical statistical structure constitutes the building block of the synchronous drift interlaced mapping grid.

[0042] See also Figure 5 , the specific steps for obtaining the cross-parameter trend cluster map are: S401: Based on the synchronized drift interleaved mapping grid, extract the time index range covered by the node based on the node area with heat values higher than the heat threshold, call the original respiratory rate sampling curve, align the corresponding segments according to time consistency, discretize the respiratory rate value per minute in the time period, calculate the first-order derivative difference of adjacent change points, and extract the extreme value points as curvature change nodes to generate a respiratory rate curvature change point set; Based on the node area with heat value higher than the heat threshold in the synchronous drift interleaving mapping grid, first read the heat count value corresponding to each grid node in the two-dimensional heat map matrix, set the heat threshold to the number of times the node is mapped more than 5 times, that is, when the heat value is greater than 5, it is marked as a high heat node, and then extract the image frame time index range of all nodes that meet the conditions, such as grid node The frame indexes between 10 and 25 are continuously mapped as high-heat areas, and the time period range is the corresponding time series to , call the original respiratory frequency sampling sequence corresponding to the time period, set the sampling frequency to 1Hz, then each minute corresponds to 60 sampling points, extract the local sequence according to 60 points as a sampling window and perform discretization processing, that is, perform differential operation on the frequency value within each minute to obtain the fluctuation point sequence of trend change, and perform first-order derivative operation on the respiratory frequency change value within each group of sampling windows, extract the extreme point of the derivative sequence and record it as the respiratory curvature mutation point. For example, in a certain minute, the respiratory frequency sampling sequence is , then the first-order derivative is , the change direction of the derivative value from positive to negative indicates the appearance of a local maximum point, and its corresponding position is marked as the curvature change node, thereby generating a set of respiratory frequency curvature change points.

[0043] S402: Recalling the respiratory rate curvature change point set, matching the overlapping time points of the heart rate mutation segment and the pigment trajectory offset segment on the same time axis, aligning the trend mutation nodes of the three indicators by time to form a co-occurrence structure, performing aggregation processing on the co-occurrence structure by time window, counting the number of simultaneous mutations of the three parameters in each time window and marking the position nodes, calculating to obtain the trend fusion density, marking the segments where the trend fusion density exceeds the density benchmark value as trend fusion block areas, and generating a trend fusion block identification set; The specific formula for calculating the trend fusion density is: ; in, It represents the trend fusion density, which is used to evaluate the degree of aggregation of multi-parameter trend covariation within the time window. Indicates the The normalized value of the heart rate variation in a time window, Indicates the The normalized value of the change in respiratory frequency curvature within a time window, Indicates the The sensitivity factor for overlapping trend events in a time window is determined by the mean of the mutation amplitudes of the three types of trend events. Indicates the The normalized value of the displacement amplitude of the pigment trajectory within a time window, Indicates the Normalized value of the pigment distribution diffusion range within a time window, Indicates the The number of overlapping trend mutation events in a time window, Indicates the absolute value, represents the logarithmic function with natural logarithm as base, represents the square root operation, is the total number of time windows; After calling each time node of the respiratory frequency curvature change point set, the corresponding mutation time point sequence extracted from the heart rate mutation segment and the pigment trajectory offset segment under the same time axis is matched, and the time alignment tolerance window is set to ±1s. By judging whether there is a mutation record in the same window of the three types of indicators, a co-occurrence event structure is formed. For example, if the respiratory curvature point appears in a certain time window, , while the mutations in heart rate and pigment trajectory appeared in and , then the three meet the time co-occurrence condition, and the number of mutation items in the window is counted as 3, which is set as the aggregation node. For each time window, the heart rate change amplitude is obtained separately , respiratory curvature change value , pigment displacement amplitude and diffusion range , calculate the number of overlapping events And obtain the mean of three changes to calculate the overlap sensitivity factor , based on this, the formula is called: ; The operational logic of this formula is as follows: The numerator part It is used to comprehensively measure the difference in the trend fluctuation of heart rate and respiratory curvature, and the mutation response sensitivity factor Adjust the sensitivity. The larger the value, the stronger the weight of the trend co-occurrence. Then the trend quantity suppression mechanism for scoring is introduced, where the number of trend overlaps The bigger the The higher the value, the more spatial deviation of pigment displacement and diffusion degree is added. , the two together constitute the correction factor for trend inconsistency or spatial discreteness. The overall formula shows that: the more consistent the trend, the more sensitive the response, and the more stable the spatial change, the The higher the value, the more obvious the multi-parameter trend clustering phenomenon is in this period. On the contrary, if the trend response amplitude difference is small, the number of events is small, or the spatial direction is inconsistent, then The value is suppressed, indicating that its trend fusion feature is not significant. The value is used to determine whether the window should be included in the trend fusion block area. If the density is greater than a certain benchmark value, the period is marked as a fusion area.

[0044] The calculation result of this formula is a comprehensive quantitative reflection of the consistency, intensity and spatial coordination of the three types of parameter mutations in a single time window, which is used to determine whether a trend co-occurrence aggregation phenomenon is formed. The value sequence reflects the density level of co-occurring events under cross-modal and multi-parameter conditions. It is an important indicator for subsequent trend fusion block screening and a basic variable for transparency setting in subsequent map drawing.

[0045] For example: In the first time window, if , , , , then: ; The results show that: the trend fusion density The value is approximately 0.01898. If the density baseline is set to 0.015, this time window is marked as a trend fusion block. This value indicates that the trend difference during this period is significant, with high sensitivity, good spatial stability, and trend aggregation characteristics. It is a high-density cluster area that requires special attention when constructing the map.

[0046] Table 2 lists the parameter values within the example time window.

[0047] Table 2 Trend fusion parameter sample table

[0048] As shown in Table 2, the values of each parameter are all within a reasonable range after normalization, which is convenient for cross-modal analysis.

[0049] S403: Based on the trend fusion tile identification set, the time axis distribution of each identification segment is mapped to a unified two-dimensional atlas matrix structure, the trend fusion density corresponding to each tile is assigned as an opacity mapping factor to the layer rendering channel, the tile boundaries are marked and the atlas coordinate axis structure is drawn to establish a cross-parameter trend aggregation atlas; According to the trend fusion block identification, the time start and end points of each block segment are recorded in the centralized data, and the sequence number is mapped to the two-dimensional atlas coordinate structure. The horizontal axis of the atlas represents time and the vertical axis represents parameter type coding number. Each fusion block is fused according to its trend fusion density. The transparency level is calculated by normalizing the value and setting the mapping function as ,in The maximum value in the current tile Value, the transparency weight of each tile in the layer drawing is based on its Proportional distribution is performed, and the boundaries of the blocks are positioned to draw a rectangular structure, which is presented with a unified numbering of the respiration, heart rate and pigment parameter types. A two-dimensional map structure is drawn with the horizontal axis as time and the vertical axis as the degree of parameter trend aggregation, thereby establishing a cross-parameter trend aggregation map.

[0050] See also Figure 6 The specific steps for obtaining the risk ladder visual diagnosis display information are as follows: S501: Based on the cross-parameter trend cluster map, extract the time span value, trend fusion density level, and number of participating physiological indicators of each block according to the start and end positions of the time axis of each block. Set the trend density threshold and the lower limit of the number of participating parameters respectively, filter each block, mark the blocks where the change of combined indicators exceeds the screening conditions as high-risk sections, and generate a high-risk block identification tag set; Based on the start and end positions of the time axis of each block in the cross-parameter trend cluster map, the start time and end time are extracted respectively to calculate the time span value of the block. For example, if a block starts at , ending at , then the time span is 60 seconds. At the same time, the trend fusion density level value and the number of physiological indicators marked in the atlas matrix of the block are extracted. For example, the density level value is 0.032, and the participating indicators include three categories of heart rate, respiratory rate and skin color trajectory, totaling 3 items. The trend fusion density value and the number of parameters are compared with the preset screening criteria respectively. The trend density threshold is set to 0.025, which comes from the 85th percentile of the previous mean fusion density of all blocks. After counting, the 85th percentile density of 30 blocks is 0.025. In addition, the lower limit of the number of participating parameters is set to 3 items as the lower bound of multi-parameter fusion. Since the actual total number of participating parameters is 4, the standard is set as its lower limit. The conditional screening mechanism is used to perform joint conditional judgment and risk marking operations on blocks that meet the density value ≥ 0.025 and the number of parameters ≥ 3. The marking results include the block number, time start and end values, density level and risk level. All blocks that meet the conditions are classified into the high-risk set, and finally a high-risk block identification label set is generated; S502: Calling the high-risk tile identification tag set, sequentially numbering the tiles in chronological order according to the time index sequence and risk level tag of each tile, assigning color band codes according to each risk level, and drawing the tiles corresponding to each risk level into a color segment structure to form a continuous horizontal axis color band combination area, thereby generating risk color band coding structure information; The time index and corresponding risk level of each block in the high-risk block identification mark set are called, and the blocks are sorted in order from small to large according to the starting time and assigned sequential numbers. For example, the blocks are numbered G1 to G12. The risk level is set according to the trend fusion density value. If the density value is between 0.025-0.030, it is set as level one risk, between 0.030-0.035, it is set as level two risk, and greater than 0.035 is set as level three risk. The classification is based on the quantile statistical structure in the high-risk block set, and risk color band codes are assigned respectively: level one is yellow, level two is orange, and level three is red. Each risk level block is drawn as a different color segment using a continuous color band construction method. The color segment structure area is generated by horizontally arranging the time axis continuously. The starting position corresponds to the start time of the block, and the end position corresponds to the end time of the block. A horizontal color band bar is formed, and finally the risk color band coding structure information is generated; S503: Based on the risk color band coding structure information, the time index and color level of each color band segment are called to construct a two-dimensional visualization area layer structure, the color band transparency mapping relationship is defined, the risk level value is used as the transparency input, the opacity coefficient of the tile area is adjusted, and the risk tiles are superimposed on the coordinate plane to form a display area, thereby establishing the risk ladder visual diagnosis display information; According to the time index and corresponding color level of each color band segment in the risk color band coding structure information, each block is drawn as a rectangular area in the two-dimensional atlas coordinate matrix, with its horizontal axis as the time dimension and the vertical axis as the parameter combination index dimension. An opacity adjustment mechanism is set for each block, and the risk level is mapped to a transparency control factor. The transparency conversion standard is set as follows: level 1 risk is 0.3, level 2 risk is 0.6, and level 3 risk is 0.9, corresponding to the highest opacity value of the red block area. The opacity coefficient of the RGB image channel is adjusted according to this ratio. The layer is rendered together with the color channel and the transparency channel, and all blocks are superimposed to form a complete atlas display structure. Axis and time labels are added to the atlas boundaries, and the coordinate position and number identification of the high-risk area are located. Finally, a risk ladder visual diagnosis display information is established; A neonatal health assessment system based on visual analysis is used to implement the above-mentioned neonatal health assessment method based on visual analysis. The system includes: The pigment variation recognition module obtains a sequence of image frames from the newborn's face and chest, marks pixel areas where the pixel change rate is greater than the HSV saturation change threshold as variation areas, and superimposes the variation areas of the image frames in each time period to generate a layer of sections with significant pigment variation. The heart rate mutation identification module calls the pigment variation significant segment layer, extracts the ECG sampling sequence of the corresponding time, calculates the RR interval difference of the heart rate at consecutive time points, extracts the time period where the change rate exceeds the heart rate variability threshold, and obtains the heart rate trend mutation area map group; The drift interleaving mapping module selects image frames of the corresponding time period from the pigment variation significant segment layer based on the start and end times of the mutation segment in the heart rate trend mutation segment map group, marks the angle difference between the two as the angle offset, and maps them into grid nodes in the two-dimensional coordinate system to generate a synchronized drift interleaving mapping grid. The parameter trend aggregation module uses a synchronized drift interlaced mapping grid to extract the curvature change points of the respiratory rate per minute, align the time stamps of the intersection of the heart rate and skin color trajectories, and perform co-occurrence clustering on the time periods where significant trend changes occur in the three indicators. The co-occurrence areas are marked as trend fusion blocks to generate a cross-parameter trend aggregation map. The risk ladder visualization module sets the blocks whose joint mutation indicators exceed the risk ladder division threshold as high-risk units based on the cross-parameter trend aggregation map, uses segmented color ribbons to fill the display frame corresponding to the risk level, sorts and draws the risk evolution trend in timeline order, and generates risk ladder visual diagnostic display information.

[0051] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0052] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0053] It should be understood that in various embodiments of the present invention, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0054] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0055] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0056] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0057] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0058] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0059] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0060] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A newborn health assessment method based on visual analysis, characterized in that: The following steps are involved: S1: Obtain a sequence of image frames from the newborn's face and chest, mark the pixel areas where the pixel change rate is greater than the HSV saturation change threshold as variation areas, and superimpose the variation areas of the image frames in each time period to generate a layer of significant pigment variation segments; S2: calling the pigment variation significant segment layer, extracting the ECG sampling sequence of the corresponding time, calculating the RR interval difference of the heart rate at consecutive time points, extracting the time period where the change rate exceeds the heart rate variability threshold, and obtaining the heart rate trend mutation area map group; S3: According to the start and end time of the mutation segment in the heart rate trend mutation area map group, select image frames of the corresponding time period in the pigment variation significant segment layer, mark the angle difference between the two as the angle offset, and map them into grid nodes in the two-dimensional coordinates to generate a synchronous drift interlaced mapping grid; S4: Based on the synchronized drift interlaced mapping grid, the curvature change points of the respiratory rate per minute are extracted, the time marks of the intersection of the heart rate and skin color trajectories are aligned, and the time periods where significant trend changes occur in the three indicators are co-occurred and clustered. The co-occurrence areas are marked as trend fusion blocks to generate a cross-parameter trend cluster map.

2. The method for neonatal health assessment based on visual analysis according to claim 1, characterized in that: The pigment variation significant segment layer includes the significant saturation change area in the image frame, the variation pixel distribution boundary, the color level change interval label, the change area time index point and the layer overlay transparency parameter; the heart rate trend mutation area map group includes the mutation start and end time period, the heart rate slope change mark, the RR interval change mark point, the direction transition position node and the trend map group index order; the synchronous drift interlaced mapping grid is specifically the heart rate change direction angle mapping point, the skin color area center of gravity displacement trajectory node, the angle offset distribution hot zone, the grid level division rule and the heat weight density factor; the cross-parameter trend aggregation map is specifically the respiratory frequency curvature fluctuation segment, the three-parameter overlapping time block, the trend same-direction aggregation area, the map transparency mapping layer and the aggregation segment classification identification information.

3. The method for neonatal health assessment based on visual analysis according to claim 2, characterized in that: The steps for obtaining the layer of the significant pigment variation section are specifically as follows: S101: Acquire a sequence of image frames in the facial and chest acquisition areas of the newborn, extract the pixel values of each pixel in the image frame under the three RGB channels, and record the brightness change value and saturation change value of the pixel in the time series according to the pixel index. Call the time index of the image frame to perform inter-frame difference calculation on the brightness value and saturation value at the same pixel position to generate brightness and saturation difference information; S102: Based on the brightness and saturation difference information, according to the change rate data of each pixel point, an HSV saturation change threshold is set, pixels whose change rate exceeds the threshold are screened, the spatial coordinate positions of the pixels are extracted, and the pixels are integrated into a pixel change layer within a continuous time period according to the image frame index to generate a high-variability pixel area labeling group; S103: Call the high-variability pixel area marking map group, superimpose the pixel positions of the layers in each time period, perform statistical calculations on the number of repeated markings at the same spatial coordinate position, draw a variation frequency superposition map, output the image area layer with marking frequency information, and generate a pigment variation significant segment layer.

4. The method for neonatal health assessment based on visual analysis according to claim 3, characterized in that: The steps for obtaining the heart rate trend mutation area map group are specifically as follows: S201: calling the pigment variation significant segment layer, extracting the ECG sampling sequence corresponding to each time point, calculating the time interval between adjacent R waves, recording a continuous RR interval sequence, and performing a derivative operation on the time point difference of the sequence to obtain the RR interval change rate per unit time, screening the time period where the change rate exceeds the standard heart rate variability threshold, and generating a heart rate mutation interval sequence; S202: Based on the heart rate mutation interval sequence, according to the RR interval value in each mutation time period, the sign changes of the consecutive RR differences are extracted in chronological order, and the time periods where the direction of the rate of change changes are changed are selected as inflection points. A position marker structure is established according to the time label, and the corresponding slope sign change position is recorded to generate a slope direction transition segment set; S203: Call the slope direction transition segment set, arrange the RR interval change values of each segment in an overlapping manner, establish a trend segment group graph structure based on the time axis, count the repeated direction transition positions and mark the intensity density areas, and generate a heart rate trend mutation area map group.

5. The method for neonatal health assessment based on visual analysis according to claim 4, characterized in that: The steps for obtaining the synchronous drift staggered mapping grid are specifically as follows: S301: Based on the start and end times of the mutation segment in the heart rate trend mutation area map group, select image frames of the corresponding time period in the pigment variation significant segment layer, extract a set of pixels in the variation area in each frame, calculate the spatial center of gravity position of the variation area based on the two-dimensional coordinates of each pixel position in the set, obtain the vector path of the center of gravity coordinates in the time dimension in consecutive frames, and generate center of gravity displacement vector trajectory information; S302: Retrieving the centroid displacement vector trajectory information, extracting the RR interval difference within the corresponding mutation time period in the heart rate trend mutation area map group based on the spatial change path of each time period, calculating the average change rate of the RR interval per unit time in the mutation time period, and constructing the heart rate change direction angle vector. Obtaining the angle difference between the centroid path vector and the heart rate direction vector, calculating the angle offset, screening nodes with angle offset values greater than the offset tolerance threshold, and generating a set of significant angle offset points. S303: Based on the angle offset significant point concentration, the image frame number and coordinate point corresponding to each frame are extracted, the significant angle position is mapped to a two-dimensional plane grid, the number of times each grid node is mapped is counted, the color heat level is generated according to the cumulative number of nodes, and a synchronous drift interlaced mapping grid is established.

6. The method for neonatal health assessment based on visual analysis according to claim 5, characterized in that: The formula for obtaining the angle offset is specifically: ； in, Indicates the angle offset between the center of gravity path and the heart rate direction vector, Represents the normalized value of the displacement vector of the centroid of the mutation area, Represents the normalized value of the RR interval change direction vector, represents the heart rate change rate gain parameter, Indicates the offset of the center of gravity change value between image frames in the current time period.

7. The method for neonatal health assessment based on visual analysis according to claim 6, characterized in that: The steps for obtaining the cross-parameter trend cluster map are specifically as follows: S401: Based on the synchronous drift interleaved mapping grid, according to the node area whose heat value is higher than the heat threshold, extract the time index range covered by the node, call the original sampling curve of the respiratory rate, align the corresponding segments according to time consistency, discretize the respiratory rate value per minute in the time period, calculate the first-order derivative difference of adjacent change points, extract the extreme value point as the curvature change node, and generate a respiratory rate curvature change point set; S402: calling the respiratory rate curvature change point set, matching the overlapping time points of the heart rate mutation segment and the pigment trajectory offset segment on the same time axis, aligning the trend mutation nodes of the three indicators by time to form a co-occurrence structure, performing aggregation processing by time window in the co-occurrence structure, counting the number of simultaneous mutations of the three parameters in each time window and marking the position nodes, calculating to obtain the trend fusion density, marking the segment where the trend fusion density exceeds the density benchmark value as a trend fusion block area, and generating a trend fusion block identification set; The formula for calculating the trend fusion density is as follows: ； in, It represents the trend fusion density, which is used to evaluate the degree of aggregation of multi-parameter trend covariation within the time window. Indicates the The normalized value of the heart rate variation in a time window, Indicates the The normalized value of the change in respiratory frequency curvature within a time window, Indicates the The sensitivity factor for responding to overlapping trend events within a time window, Indicates the The normalized value of the displacement amplitude of the pigment trajectory within a time window, Indicates the Normalized value of the pigment distribution diffusion range within a time window, Indicates the The number of overlapping trend mutation events in a time window, is the total number of time windows; S403: Based on the trend fusion tile identification set, according to the time axis distribution of each identification segment, it is mapped to a unified two-dimensional atlas matrix structure, the trend fusion density corresponding to each tile is assigned as the opacity mapping factor to the layer rendering channel, the tile boundaries are marked and the atlas coordinate axis structure is drawn to establish a cross-parameter trend aggregation atlas.

8. The method for neonatal health assessment based on visual analysis according to claim 7, characterized in that: The method further comprises the following steps: S5: Based on the cross-parameter trend cluster map, the blocks whose joint mutation index exceeds the risk ladder division threshold are set as high-risk units, and the display frames corresponding to the risk levels are filled with segmented color ribbons. The risk evolution trends are sorted and plotted according to the timeline sequence to generate visual diagnostic display information of the risk ladder; The risk ladder visual diagnosis display information includes a risk level strip structure, a risk unit color band overlay diagram, risk evolution sequence nodes, graphic area weight marks and a risk ladder horizontal grading axis.

9. The method for neonatal health assessment based on visual analysis according to claim 8, characterized in that: The steps for obtaining the risk ladder visual diagnosis display information are as follows: S501: Based on the cross-parameter trend cluster map, extract the time span value, trend fusion density level, and number of participating physiological indicators of each block according to the start and end positions of the time axis of each block, set the trend density threshold and the lower limit of the number of participating parameters, filter each block, mark the blocks where the change of combined indicators exceeds the filtering conditions as high-risk sections, and generate a high-risk block identification mark set; S502: The high-risk tile identification mark set is called, and the tiles are sequentially numbered in chronological order according to the time index sequence and risk level mark of each tile. Color band codes are assigned according to each risk level, and the tiles corresponding to each risk level are drawn as a color segment structure to form a continuous horizontal axis color band combination area, thereby generating risk color band coding structure information; S503: According to the risk color band coding structure information, call the time index and color level of each color band segment, build a two-dimensional visualization area layer structure, define the color band transparency mapping relationship, use the risk level value as the transparency input, adjust the tile area opacity coefficient, overlay the risk tiles to form a display area in the coordinate plane, and establish risk ladder visual diagnostic display information.

10. A newborn health assessment system based on visual analysis, characterized in that: The system is used to implement the neonatal health assessment method based on visual analysis according to any one of claims 1 to 9, and the system includes: The pigment variation recognition module obtains a sequence of image frames from the newborn's face and chest, marks pixel areas where the pixel change rate is greater than the HSV saturation change threshold as variation areas, and superimposes the variation areas of the image frames in each time period to generate a layer of sections with significant pigment variation. The heart rate mutation identification module calls the pigment variation significant segment layer, extracts the electrocardiogram sampling sequence of the corresponding time, calculates the RR interval difference of the heart rate at consecutive time points, extracts the time period where the change rate exceeds the heart rate variability threshold, and obtains the heart rate trend mutation area map group; The drift interleaving mapping module selects image frames of corresponding time periods in the pigment variation significant segment layer according to the start and end times of the mutation segment in the heart rate trend mutation segment map group, marks the angle difference between the two as an angle offset, and maps them into grid nodes in two-dimensional coordinates to generate a synchronous drift interleaving mapping grid; The parameter trend aggregation module extracts the curvature change points of the respiratory rate per minute based on the synchronized drift interlaced mapping grid, aligns the time marks of the intersection of the heart rate and skin color trajectories, and performs co-occurrence clustering on the time periods where significant trend changes occur in the three indicators. The co-occurrence areas are marked as trend fusion blocks to generate a cross-parameter trend aggregation map; The risk ladder visualization module sets the blocks whose joint mutation indicators exceed the risk ladder division threshold as high-risk units based on the cross-parameter trend aggregation map, uses segmented color ribbons to fill the display frame corresponding to the risk level, sorts and draws the risk evolution trend in timeline order, and generates risk ladder visual diagnostic display information.

Images