Bone age evaluation method, device, electronic device and storage medium

Abstract

The embodiment of the present invention provides a bone age assessment method, comprising: obtaining a skeletal map and a voiceprint sequence of a target person; extracting features from the skeletal map through a preset image feature extraction network to obtain skeletal features of the skeletal map; extracting features from the voiceprint sequence through a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence; fusing skeletal features with voiceprint features through a preset first feature fusion network to obtain fusion features of skeletal features and voiceprint features; classifying the fused features for bone age through a preset classification network, and assessing the target person's bone age based on the classification results of the bone age classification to obtain a bone age assessment result for the target person. The method can detect the age corresponding to the bones in the skeletal map, and because voiceprints are added as auxiliary detection, the accuracy of bone age assessment is improved, and there is no need for doctors to manually observe the skeletal map and interpret the skeletal image, thereby improving the efficiency of bone age assessment.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and smart medicine, and in particular to a bone age assessment method, device, electronic device and storage medium. Background Art

[0002] Human skeletal development changes are generally similar, with each bone developing in a continuous and staged process. Bones at different stages exhibit distinct morphological characteristics, making bone age assessment a relatively accurate reflection of an individual's growth and maturity. Bone age, short for skeletal age, is determined using specific bone X-ray images. Bone age, also known as biological age, measures skeletal maturity expressed in years. It reflects the degree of physical and physiological development by measuring changes in bone morphology, structure, size, and interrelationships. Traditional bone age assessment typically involves X-rays of the subject's hand and wrist, which are then interpreted by a physician. This interpretation process is time-consuming, highly dependent on the physician's experience, and prone to error. Consequently, traditional bone age assessments suffer from poor accuracy. Summary of the Invention

[0003] An embodiment of the present invention provides a bone age assessment method, which can use the skeletal map of a target person and the voiceprint sequence when the target person speaks according to specified content to process the skeletal map, thereby detecting the age corresponding to the bones in the skeletal map. Since the voiceprint is added as an auxiliary detection, the accuracy of bone age assessment is improved, and there is no need for doctors to manually observe the skeletal map and interpret the bone image, thereby improving the assessment efficiency of bone age assessment.

[0004] In a first aspect, an embodiment of the present invention provides a bone age assessment method, the method comprising:

[0005] Obtaining a skeleton diagram and voiceprint sequence of the target person, wherein the skeleton diagram is a skeleton diagram of a specified body part, and the voiceprint sequence is collected when the target person speaks according to the specified content;

[0006] Performing feature extraction on the skeleton graph through a preset image feature extraction network to obtain skeleton features of the skeleton graph;

[0007] Extracting features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence;

[0008] Performing feature fusion on the skeleton feature and the voiceprint feature through a preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature;

[0009] The fusion features are classified into bone age categories using a preset classification network, and the bone age of the target person is assessed based on the classification results of the bone age classification to obtain the bone age assessment results of the target person.

[0010] Optionally, the preset image feature extraction network includes a preset global feature extraction network, a preset local feature extraction network, and a preset second feature fusion network. The feature extraction of the skeleton graph by the preset image feature extraction network to obtain the skeleton features of the skeleton graph includes:

[0011] Performing feature extraction processing on the skeleton graph through the global feature extraction network to obtain global features of the skeleton graph;

[0012] Segmenting the skeleton map according to a preset segmentation strategy to obtain a corresponding number of skeleton local maps, wherein each of the skeleton local maps has an overlapping area with an adjacent skeleton local map, and the resolution scale of the skeleton map is greater than the resolution scale of the skeleton local map;

[0013] Performing feature extraction processing on the local skeleton map through the preset local feature extraction network to obtain local features of the skeleton map;

[0014] The global features and the local features are fused through the preset second feature fusion network to obtain the skeleton features of the skeleton graph.

[0015] Optionally, the preset voiceprint feature extraction network includes a time feature extraction network, a frequency feature extraction network, and a third feature fusion network. The feature extraction of the voiceprint sequence by the preset voiceprint feature extraction network to obtain the voiceprint features of the voiceprint sequence includes:

[0016] Performing feature extraction processing on the preset voiceprint sequence through the time feature extraction network to obtain the time feature of the voiceprint sequence;

[0017] The voiceprint sequence is transformed from the time domain to the frequency domain by fast Fourier transform to obtain a spectrum of the voiceprint sequence;

[0018] Performing feature extraction processing on the spectrum graph through the frequency feature extraction network to obtain the frequency features of the voiceprint sequence;

[0019] The time feature and the frequency feature are fused by the preset third feature fusion network to obtain the voiceprint feature of the voiceprint sequence.

[0020] Optionally, the transforming the voiceprint sequence from the time domain to the frequency domain by fast Fourier transform to obtain a spectrum of the voiceprint sequence includes:

[0021] performing frame processing on the voiceprint sequence to obtain a frame-processed voiceprint sequence;

[0022] Performing windowing processing on the voiceprint sequence after the frame processing to obtain a windowed voiceprint sequence;

[0023] Performing fast Fourier transform point processing on the windowed voiceprint sequence, converting the voiceprint sequence from the time domain to the frequency domain, and obtaining a spectrum diagram of the voiceprint sequence.

[0024] Optionally, before extracting features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence, the method further includes:

[0025] Preprocessing the voiceprint sequence to obtain a preprocessed voiceprint sequence;

[0026] performing semantic recognition on the preprocessed voiceprint sequence;

[0027] When the semantic recognition result is the same as the specified content, the pre-processed voiceprint sequence is determined to be the voiceprint sequence of the target person.

[0028] Optionally, the preset first feature fusion network includes a first linear transformation layer, a second linear transformation layer, and a first full convolution layer. The skeleton feature and the voiceprint feature are fused by the preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature, including:

[0029] Performing linear transformation on the bone features through the first linear transformation layer to obtain linearly transformed bone features;

[0030] performing a linear transformation on the voiceprint feature through the second linear transformation layer to obtain a linearly transformed voiceprint feature, wherein the linearly transformed skeleton feature and the linearly transformed voiceprint feature have the same dimension;

[0031] The skeleton features after the linear transformation and the voiceprint features after the linear transformation are convolved by the first full convolution layer to obtain a fusion feature of the skeleton features and the voiceprint features.

[0032] Optionally, the image feature extraction network, the voiceprint feature extraction network, the first feature fusion network, and the classification network are trained using the same data set, and the training steps include:

[0033] Constructing a data set, the data set including a sample skeleton diagram, a sample voiceprint sequence, and corresponding age-labeled data, wherein the sample skeleton diagram is the skeleton diagram of the sample person, the age-labeled data is the real age of the sample person, and the sample voiceprint sequence is a voiceprint sequence collected when the sample person speaks according to specified content;

[0034] The image feature extraction network, the voiceprint feature extraction network, the first feature fusion network and the classification network are jointly trained using the data set.

[0035] In a second aspect, an embodiment of the present invention further provides a bone age assessment device, comprising:

[0036] An acquisition module is used to acquire a skeleton diagram and a voiceprint sequence of a target person, wherein the skeleton diagram is a skeleton diagram of a specified body part, and the voiceprint sequence is collected when the target person speaks according to the specified content;

[0037] A first extraction module is used to extract features from the skeleton graph using a preset image feature extraction network to obtain skeleton features of the skeleton graph;

[0038] a second extraction module, configured to extract features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence;

[0039] a fusion module, configured to fuse the skeleton feature with the voiceprint feature through a preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature;

[0040] The evaluation module is used to classify the fusion features into bone age through a preset classification network, and to evaluate the bone age of the target person based on the classification results of the bone age classification to obtain the bone age evaluation results of the target person.

[0041] In a third aspect, an embodiment of the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the bone age assessment method provided in the embodiment of the present invention are implemented.

[0042] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the bone age assessment method provided in the embodiment of the invention are implemented.

[0043] In an embodiment of the present invention, a skeleton map and a voiceprint sequence of a target person are obtained, wherein the skeleton map is a skeleton map of a specified body part, and the voiceprint sequence is collected when the target person speaks according to specified content; a preset image feature extraction network is used to extract features from the skeleton map to obtain skeleton features of the skeleton map; a preset voiceprint feature extraction network is used to extract features from the voiceprint sequence to obtain voiceprint features of the voiceprint sequence; a preset first feature fusion network is used to fuse the skeleton features with the voiceprint features to obtain fusion features of the skeleton features and the voiceprint features; a preset classification network is used to classify the fusion features for bone age, and a bone age assessment is performed on the target person based on the classification results of the bone age classification to obtain a bone age assessment result of the target person. The skeleton map of the target person and the voiceprint sequence of the target person speaking according to specified content can be used to process the skeleton map, thereby detecting the age corresponding to the bones in the skeleton map. Since the voiceprint is added as an auxiliary detection, the accuracy of the bone age assessment is improved, and the doctor does not need to manually observe the skeleton map and interpret the bone image, thereby improving the assessment efficiency of the bone age assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 This is a flow chart of a bone age assessment method provided by an embodiment of the present invention;

[0046] Figure 2 1 is a structural diagram of a bone age assessment device provided by an embodiment of the present invention;

[0047] Figure 3 is a structural diagram of a first extraction module provided by an embodiment of the present invention;

[0048] Figure 4 is a structural diagram of a second extraction module provided by an embodiment of the present invention;

[0049] Figure 5 1 is a schematic structural diagram of a conversion submodule provided by an embodiment of the present invention;

[0050] Figure 6 2 is a schematic structural diagram of another bone age assessment device provided by an embodiment of the present invention;

[0051] Figure 7 This is a schematic structural diagram of a fusion module provided by an embodiment of the present invention;

[0052] Figure 8 This is a structural diagram of another bone age assessment device provided by an embodiment of the present invention.

[0053] Figure 9 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] See Figure 1 , Figure 1 This is a flow chart of a bone age assessment method provided by an embodiment of the present invention. Figure 1 As shown, the following steps are included:

[0056] 101. Obtain the target person’s skeleton diagram and voiceprint sequence.

[0057] In an embodiment of the present invention, the above-mentioned skeletal map is a skeletal map of a specified body part. The above-mentioned skeletal map can be an X-ray image of a specified body part of the target person. The above-mentioned specified body part can be an X-ray image of the wrist. The above-mentioned skeletal map can specifically be an X-ray image of the wrist bones, which can be either the left hand or the right hand. The embodiment of the present invention can collect an X-ray image of the wrist bones of the left hand as the skeletal map of the target person.

[0058] In an embodiment of the present invention, the voiceprint sequence is collected when the target person speaks according to the specified content, and the collected voiceprint sequence includes a sequence in the time domain. The voiceprint sequence can be obtained by a sound sensor (such as a microphone), and the voiceprint sequence can be framed according to a preset framing rule. For example, the preset framing rule includes a frame rate n and a duration t, that is, there are n voiceprint frames per second in the voiceprint sequence, the duration of the voiceprint sequence is t seconds, and the total number of voiceprint frames in the voiceprint sequence is N=n×t. The specified content can be text content pronounced with nasal sounds, such as text content with the finals "ang", "en", "eng", "ong", etc., so that the voiceprint sequence includes implicit information when speaking with nasal sounds.

[0059] In one possible embodiment, the specified content can be textual content consisting of the least frequently used initial and final consonant combinations, as determined by statistical analysis. In another possible embodiment, the specified content can be a combination of rare or uncommon characters. This reduces the noise associated with pronunciation habits in the voiceprint sequence, as pronunciation habits are not formed, making the collected voiceprint sequence more similar to the target person's natural physical development.

[0060] It should be noted that the above-mentioned skeleton image can be one frame or multiple consecutive frames. In the embodiment of the present invention, the above-mentioned skeleton image can be selected as one frame, and the above-mentioned voiceprint sequence can be selected as 1 second or 2 seconds.

[0061] In a possible embodiment, the skeleton map and the voiceprint sequence can be collected at the same time. When the skeleton map is a continuous multi-frame image, the above-mentioned voiceprint sequence can have the same preset duration as the skeleton map sequence. For example, if the skeleton map sequence is 3 seconds, the voiceprint sequence is also 3 seconds. It can be further understood that the number of frames of the above-mentioned skeleton map sequence is the same as the number of frames of the voiceprint sequence, and the interval time between two adjacent frames in the skeleton map sequence is the same as the interval time between two adjacent frames in the voiceprint sequence.

[0062] 102. Feature extraction is performed on the skeleton image through a preset image feature extraction network to obtain the skeleton features of the skeleton image.

[0063] In an embodiment of the present invention, the above-mentioned preset image feature extraction network can be a feature extraction network based on a convolutional neural network. Through the image feature extraction network, the bone features in the bone map can be extracted, and the bone features contain the bone age information of the target person.

[0064] Specifically, the above-mentioned preset image feature extraction network includes a preset global feature extraction network, a preset local feature extraction network and a preset second feature fusion network. In the step of extracting features from the skeleton map through the preset image feature extraction network to obtain the skeleton features of the skeleton map, the skeleton map can be feature extracted by the global feature extraction network to obtain the global features of the skeleton map; the skeleton map is segmented according to a preset segmentation strategy to obtain a corresponding number of skeleton local maps, each skeleton local map has an overlapping area with the adjacent skeleton local map, wherein the resolution scale of the skeleton map is greater than the resolution scale of the skeleton local map; the skeleton local map is feature extracted by the preset local feature extraction network to obtain the local features of the skeleton map; the global features and local features are fused by the preset second feature fusion network to obtain the skeleton features of the skeleton map.

[0065] In an embodiment of the present invention, the above-mentioned preset global feature extraction network can be understood as a pre-trained feature extraction network. The above-mentioned global feature extraction network can be constructed based on a deep convolutional network or a residual neural network, and can extract feature maps of different resolution scales at different depths. For example, low-level feature maps with larger resolution scales are extracted in a shallow network, intermediate feature maps with medium resolution scales are extracted in a middle network, and high-level feature maps with smaller resolution scales are extracted in a deep network.

[0066] Specifically, a first global feature map, a second global feature map, and a third global feature map may be sequentially extracted according to different depths of a preset global feature extraction network. The scale resolution of the first global feature map is greater than the resolution scale of the second global feature map, and the resolution scale of the second global feature map is greater than the resolution scale of the third global feature map.

[0067] For example, the resolution scale of the skeleton map is 1024×1024, the resolution scale of the first global feature map may be 512×512, the resolution scale of the second global feature map may be 256×256, and the resolution scale of the third global feature map may be 64×64.

[0068] It should be noted that the first global feature map is a shallow feature map that can retain relatively rich local details. The second global feature map is a mid-level feature map. Compared with the shallow feature map, it is more advanced in semantics, but retains fewer local details than the first global feature. The third global feature is a deep feature map, which is an abstract expression of high-level semantics but has few local details. In one possible embodiment, the above-mentioned global feature extraction network can be constructed based on a residual neural network. The above-mentioned first global feature map, second global feature, and third global feature can all contain the residuals of their previous layer, thereby retaining more local details.

[0069] The above segmentation strategy can be to randomly slice the skeleton graph into Figure 1A local skeletal map with a scale resolution of / 2 is used. For example, the resolution scale of the skeletal map is 1024×1024, and several 512×512 local skeletal maps are obtained by random slicing. Generally speaking, bone age information is often located in the local skeletal maps of the skeletal map, such as the radius, metacarpal bones, proximal phalanges, middle phalanges, distal phalanges, capitate bones, and hamate bones. When performing feature extraction on the skeletal map, some information will be lost during the downsampling process. Therefore, in an embodiment of the present invention, by segmenting the skeletal map, richer local details can be retained based on the information of the original skeletal map, thereby improving the accuracy of skeletal assessment. Moreover, the resolution scale of the local skeletal map is smaller than that of the skeletal map. Multiple local skeletal maps are equivalent to adding computing channels. For convolution calculations, the increase in computing channels does not affect the calculation speed. The size of the resolution scale is the factor that affects the calculation speed. Therefore, the resolution scale of the local skeletal map is smaller than that of the skeletal map, which can also achieve faster calculation speed during the convolution calculation process.

[0070] Through the preset local feature extraction network, a local feature map with the same scale resolution as the first global feature map can be extracted. Taking the above example as an example, the scale resolution of the local feature map can be obtained to be 512×512, and there is no downsampling information loss.

[0071] More specifically, in the step of segmenting the skeleton graph according to a preset segmentation strategy, the skeleton graph can be tiled through a 10*10 grid; the 6*6 area in the upper left corner of the skeleton graph is used as the first segmentation area for segmentation to obtain a first skeleton local graph; the 6*6 area in the lower left corner of the skeleton graph is used as the second segmentation area for segmentation to obtain a second skeleton local graph; the 6*6 area in the upper right corner of the skeleton graph is used as the third segmentation area for segmentation to obtain a third skeleton local graph; and the 6*6 area in the lower right corner of the skeleton graph is used as the fourth segmentation area for segmentation to obtain a fourth skeleton local graph.

[0072] By tiling the skeleton map through a 10*10 grid and dividing the skeleton map into four regions with a 6*6 grid size, each region has an overlapping area with the two adjacent regions. In this way, when performing convolution calculation on each region, the edge information of the adjacent regions can be taken into account, so that the information of each region is not isolated, thereby improving the accuracy of skeleton feature extraction.

[0073] After determining the segmentation area, the first skeletal local image is obtained by segmenting the first segmentation area, the second skeletal local image is obtained by segmenting the second segmentation area, the third skeletal local image is obtained by segmenting the third segmentation area, and the fourth skeletal local image is obtained by segmenting the fourth segmentation area. The first skeletal local image is an image of the upper left corner of the skeletal image, and the left edge information of the third skeletal local image is included on the right side of the first skeletal local image, the upper edge information of the second skeletal local image is included on the lower side of the first skeletal local image, and the upper right corner edge information of the fourth skeletal local image is included in the lower right corner of the first skeletal local image. Similarly, the second skeletal local image is an image of the lower left corner of the skeletal image, and the left edge information of the fourth skeletal local image is included on the right side of the second skeletal local image, the lower edge information of the first skeletal local image is included on the upper side of the second skeletal local image, and the upper left corner edge information of the second skeletal local image is included in the upper right corner of the second skeletal local image. The third skeletal local image is the upper right corner image of the skeletal image. The left side of the third skeletal local image contains the right side edge information of the first skeletal local image, the lower side of the third skeletal local image contains the upper side edge information of the fourth skeletal local image, and the lower left corner of the third skeletal local image contains the upper right corner edge information of the second skeletal local image. The fourth skeletal local image is the lower right corner image of the skeletal image. The left side of the fourth skeletal local image contains the right side edge information of the second skeletal local image, the upper side of the fourth skeletal local image contains the lower side edge information of the third skeletal local image, and the upper left corner of the fourth skeletal local image contains the lower right corner edge information of the first skeletal local image. In this way, when feature extraction is performed on each skeletal local image, the local features of the bones in the edge information can be effectively extracted, thereby improving the accuracy of local feature extraction of the bones.

[0074] In a possible embodiment, in the step of segmenting the skeleton map according to a preset segmentation strategy, the skeleton map can also be tiled using a 10*10 grid; the skeleton map is segmented using the 6*6 area in the upper left corner as a first segmentation area to obtain a first skeleton local map; the skeleton map is segmented using the 6*6 area in the lower left corner as a second segmentation area to obtain a second skeleton local map; the skeleton map is segmented using the 6*6 area in the upper right corner as a third segmentation area to obtain a third skeleton local map; the skeleton map is segmented using the 6*6 area in the lower right corner as a fourth segmentation area to obtain a fourth skeleton local map; and the skeleton map is segmented using the 6*6 area in the center of the skeleton map as a fifth segmentation area to obtain a fifth skeleton local map. The fifth skeleton local map can include most of the wrist bone information, further improving the accuracy of local feature extraction of the skeleton.

[0075] The above-mentioned second feature fusion network may include a third linear transformation layer, a fourth linear transformation layer, and a second full convolution layer. Since the dimensions of global features and local features are different, that is, the resolution scales are different, the global features can be linearly transformed through the third linear transformation layer, and the local features can be linearly transformed through the fourth linear transformation layer. After obtaining global features and local features of the same dimensions, convolution calculations are performed through the second full convolution layer to output the final skeleton features. Of course, when the global features output by the global feature extraction network and the local features output by the local feature extraction network have the same dimensions, the third linear transformation layer and the fourth linear transformation layer are not required, and convolution calculations can be performed directly through the second convolution layer to convolute and integrate the global features and local features to obtain the final skeleton features.

[0076] 103. Perform feature extraction on the voiceprint sequence through a preset voiceprint feature extraction network to obtain the voiceprint features of the voiceprint sequence.

[0077] In the embodiment of the present invention, the voiceprint sequence is time-domain voiceprint information, which can be understood as voiceprint information described on a time axis. The voiceprint features can include the time features and frequency features of the voiceprint sequence. The voiceprint sequence is subjected to feature extraction using the preset voiceprint feature extraction network to obtain voiceprint features including the time-frequency information of the voiceprint sequence.

[0078] The time domain information is the original voiceprint information collected by the sound sensor and can be understood as voiceprint information described on the time axis. The frequency domain information can be obtained by converting the time domain information and can be understood as voiceprint information described on the frequency axis. The dynamic conversion of voiceprint information from the time domain to the frequency domain can be achieved through Fourier series and Fourier transform.

[0079] Specifically, the preset voiceprint feature extraction network includes a time feature extraction network, a frequency feature extraction network and a third feature fusion network. In the step of extracting features of the voiceprint sequence through the preset voiceprint feature extraction network to obtain the voiceprint features of the voiceprint sequence, the preset voiceprint sequence can be feature extracted by the time feature extraction network to obtain the time features of the voiceprint sequence; the voiceprint sequence can be transformed from the time domain to the frequency domain through the fast Fourier transform to obtain the spectrum of the voiceprint sequence; the spectrum can be feature extracted by the frequency feature extraction network to obtain the frequency features of the voiceprint sequence; the time features and the frequency features can be fused by the preset third feature fusion network to obtain the voiceprint features of the voiceprint sequence.

[0080] The temporal feature extraction network can be constructed based on a feature extraction method using filter banks (fbank), Mel-frequency cepstral coefficients (MFCCs), and perceptual linear prediction coefficients (PLPs). This temporal feature extraction network can be used to extract implicit information between the target person's voiceprint timing and age information when they speak according to specified content.

[0081] The above-mentioned frequency feature extraction network can be constructed based on a convolutional neural network. Through the above-mentioned frequency feature extraction network, the implicit information between the voiceprint frequency and age information of the target person when speaking according to the specified content can be extracted.

[0082] The third feature fusion network can include a fifth linear transformation layer, a sixth linear transformation layer, and a third fully convolutional layer. Because the dimensions of time features and frequency features are different, that is, the resolution scales are different, the fifth linear transformation layer can be used to linearly transform the time features, and the sixth linear transformation layer can be used to linearly transform the frequency features. After obtaining time features and frequency features of the same dimension, convolution calculation is performed through the third fully convolutional layer to output the final voiceprint features. The final voiceprint features include the implicit information between the voiceprint time sequence and age information, as well as the implicit information between the voiceprint frequency and age information.

[0083] Specifically, the above-mentioned spectrogram can be understood as the expression of voiceprint information on the frequency axis, which is independent of time. In the above-mentioned step of transforming the voiceprint sequence from the time domain to the frequency domain through fast Fourier transform to obtain the spectrogram of the voiceprint sequence, the voiceprint sequence can be framed to obtain a framed voiceprint sequence; the framed voiceprint sequence can be windowed to obtain a windowed voiceprint sequence; and the windowed voiceprint sequence can be subjected to fast Fourier transform point processing to convert the voiceprint sequence from the time domain to the frequency domain to obtain the spectrogram of the voiceprint sequence.

[0084] In the embodiment of the present invention, the voiceprint sequence may be divided into frames, for example, into n frames per second, where n is greater than or equal to 1.

[0085] Furthermore, the length of the above voiceprint sequence can be expressed by the following formula:

[0086] N=f s ×t

[0087] Where N is the length of the voiceprint sequence, f s is the sampling frequency of the voiceprint sequence (the sampling frequency of the sound sensor), and t is the sampling duration of the voiceprint sequence.

[0088] After framing, the above voiceprint sequence is:

[0089] {y1,y2,…,y m}

[0090] Among them, the above y m is the mth voiceprint frame in the voiceprint sequence after the frame processing, and m is the total number of voiceprint frames in the voiceprint sequence after the frame processing.

[0091] In one possible embodiment, in order to achieve a smooth transition between voiceprint frames in the voiceprint sequence after framing and maintain the continuity of the voiceprint data, an overlapping segmentation method can be used to frame the voiceprint sequence. The overlapping segmentation method can be understood as the overlapping portion between the current voiceprint frame and the previous voiceprint frame, and the overlapping portion between the current voiceprint frame and the next voiceprint frame. In this case, assuming that the length of each voiceprint frame in the voiceprint sequence after framing is nfft, and the overlapping length between two adjacent voiceprint frames in the voiceprint sequence after framing is overlap, the length L of the voiceprint sequence after framing can be expressed by the following formula:

[0092] L=(nfft-overlap)×m

[0093] The initial window type, window length, and sliding step size can be set to determine the window parameters to be added. Further, the voiceprint sequence after frame processing is windowed to obtain the windowed voiceprint sequence. Specifically, the window function y can be calculated according to the adaptive time scale rule. window And the window length, multiply the framed voiceprint data by the window function to get the windowed voiceprint sequence:

[0094]

[0095] Among them, y i is the i-th voiceprint frame data after framing, is the corresponding i-th windowed voiceprint frame.

[0096] After obtaining the windowed voiceprint sequence, it can be subjected to Fast Fourier Transform (FFT) processing. This FFT processing converts the voiceprint sequence from time-domain information to frequency-domain information, obtaining the frequency and amplitude information corresponding to each moment in the voiceprint sequence, thereby obtaining the time-frequency characteristics of the voiceprint sequence in embodiments of the present invention. Specifically, the voiceprint data is first processed using an initial window type, window length, sliding step size, and FFT point number. The window type can be selected to provide good sidelobe suppression, the sliding step size can be 100%, and the FFT point number can be a smaller value. This allows for rapid processing of the voiceprint data, improving processing speed. Next, the window type, window length, sliding step size, and FFT point number are adjusted based on the frequency of the voiceprint data to meet the requirements of time-frequency analysis. Specifically, adjustments can be made by using a window with a narrower main lobe width, a 40% sliding step size, and an increased number of Fast Fourier Transform points. Through the above adaptive time scale adjustments, the window type, window length, sliding step size, and number of Fast Fourier Transform points that meet the requirements of time-frequency analysis are obtained. Then, based on the window type, window length, sliding step size, and number of Fast Fourier Transform points that meet the requirements of time-frequency analysis, the voiceprint data is subjected to a short-time Fourier transform, resulting in a spectrogram of the voiceprint sequence.

[0097] In this embodiment of the present invention, the target person's age is implicitly included in the voiceprint sequence of the specified content. This age can be amplified by the utterance of the specified content and, after being captured by the sound sensor, is implicitly included in the voiceprint sequence. Therefore, the target person's voiceprint sequence can be used as auxiliary information for bone age assessment, thereby improving the accuracy of bone age assessment.

[0098] Optionally, before extracting features of the voiceprint sequence through a preset voiceprint feature extraction network to obtain the voiceprint features of the voiceprint sequence, the voiceprint sequence can be preprocessed to obtain a preprocessed voiceprint sequence; speech recognition is performed on the preprocessed voiceprint sequence; when the speech recognition result is the same as the specified content, the preprocessed voiceprint sequence is determined to be the voiceprint sequence of the target person.

[0099] In an embodiment of the present invention, the above-mentioned preprocessing may be denoising of the voiceprint sequence, and the voiceprint sequence may be denoised by autocorrelation denoising to eliminate the environmental noise of the first voiceprint sequence and obtain a preprocessed voiceprint sequence.

[0100] The above-mentioned speech recognition can be performed through an existing speech recognition model. For example, it can be performed through related applications of speech-to-text, and the text result obtained by speech-to-text conversion is compared with the specified content to determine whether the text result is the same as the specified content. Specifically, it can be determined whether the text result and the specified content are the same in pronunciation. If they are the same, it means that the target person is speaking according to the specified content; if they are not the same, it means that the target person is not speaking according to the specified content. At this time, the target person can be prompted to speak according to the specified content again.

[0101] 104. Perform feature fusion of the skeleton feature and the voiceprint feature through a preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature.

[0102] In an embodiment of the present invention, since the above-mentioned bone features represent the implicit relationship between bones and age, and the voiceprint features represent the implicit relationship between voiceprints and age, the bone features and the voiceprint features are integrated, and the voiceprint features can be used to assist the bone features in bone age assessment, thereby improving the accuracy of bone age assessment.

[0103] Specifically, the above-mentioned preset first feature fusion network includes a first linear transformation layer, a second linear transformation layer and a first full convolution layer. In the step of fusing the skeleton features and the voiceprint features through the preset first feature fusion network to obtain the fusion features of the skeleton features and the voiceprint features, the skeleton features can be linearly transformed through the first linear transformation layer to obtain the linearly transformed skeleton features; the voiceprint features can be linearly transformed through the second linear transformation layer to obtain the linearly transformed voiceprint features, and the linearly transformed skeleton features and the linearly transformed voiceprint features have the same dimension; the linearly transformed skeleton features and the linearly transformed voiceprint features are convolved with the linearly transformed voiceprint features through the first full convolution layer to obtain the fusion features of the skeleton features and the voiceprint features.

[0104] In an embodiment of the present invention, since the dimensions of the skeletal features and the voiceprint features may differ, that is, the resolution scales may differ, the skeletal features may be linearly transformed through a first linear transformation layer, and the voiceprint features may be linearly transformed through a second linear transformation layer. After obtaining skeletal features and voiceprint features of the same dimensions, a convolution calculation is performed through a first fully convolutional layer to output a final fused feature. The final fused feature includes implicit information between the skeletal features and age, as well as implicit information between the voiceprint and age, thereby making skeletal assessment more accurate.

[0105] 105. The fusion features are classified into bone age categories through a preset classification network, and the bone age of the target person is evaluated based on the classification results of the bone age classification to obtain the bone age evaluation results of the target person.

[0106] In an embodiment of the present invention, the above-mentioned classification network can be constructed based on a convolutional neural network. Specifically, the above-mentioned classification network can be the classification part of a neural network with a ResNet structure.

[0107] Through the above classification network, the classification results of the fusion features can be output. The classification results include the bone age assessment results, that is, the bone age results of the target person.

[0108] In an embodiment of the present invention, the image feature extraction network, the voiceprint feature extraction network, the first feature fusion network, and the classification network can be trained using the same data set to construct a data set comprising a sample skeleton diagram, a sample voiceprint sequence, and corresponding age-labeled data. The sample skeleton diagram is a skeleton diagram of a sample person, the age-labeled data is the actual age of the sample person, and the sample voiceprint sequence is a voiceprint sequence obtained by collecting the sample person when speaking according to specified content.

[0109] Through the data set, the image feature extraction network, voiceprint feature extraction network, first feature fusion network and classification network are jointly trained. After the joint training is completed, the trained image feature extraction network, voiceprint feature extraction network, first feature fusion network and classification network are obtained. The image feature extraction network uses its trained parameters to extract the implicit information between bones and age as bone features. The voiceprint feature extraction network uses its trained parameters to extract the implicit information between voiceprint and age as voiceprint features. The first feature fusion network uses its trained parameters to perform linear transformation and convolution fusion. The classification network uses its trained parameters to classify the age information implicit in the fusion features.

[0110] In an embodiment of the present invention, a skeleton map and a voiceprint sequence of a target person are obtained, wherein the skeleton map is a skeleton map of a specified body part, and the voiceprint sequence is collected when the target person speaks according to specified content; a preset image feature extraction network is used to extract features from the skeleton map to obtain skeleton features of the skeleton map; a preset voiceprint feature extraction network is used to extract features from the voiceprint sequence to obtain voiceprint features of the voiceprint sequence; a preset first feature fusion network is used to fuse the skeleton features with the voiceprint features to obtain fusion features of the skeleton features and the voiceprint features; a preset classification network is used to classify the fusion features for bone age, and a bone age assessment is performed on the target person based on the classification results of the bone age classification to obtain a bone age assessment result of the target person. The skeleton map of the target person and the voiceprint sequence of the target person speaking according to specified content can be used to process the skeleton map, thereby detecting the age corresponding to the bones in the skeleton map. Since the voiceprint is added as an auxiliary detection, the accuracy of the bone age assessment is improved, and the doctor does not need to manually observe the skeleton map and interpret the bone image, thereby improving the assessment efficiency of the bone age assessment.

[0111] It should be noted that the bone age assessment method provided in the embodiment of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that can perform bone age assessment.

[0112] See Figure 2 , Figure 2 FIG. 1 is a structural diagram of a bone age assessment device provided by an embodiment of the present invention. Figure 2 As shown, the device includes:

[0113] An acquisition module 201 is used to acquire a skeleton diagram and a voiceprint sequence of a target person, wherein the skeleton diagram is a skeleton diagram of a specified body part, and the voiceprint sequence is collected when the target person speaks according to specified content;

[0114] A first extraction module 202 is configured to extract features from the skeleton graph using a preset image feature extraction network to obtain skeleton features of the skeleton graph;

[0115] The second extraction module 203 is configured to extract features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence;

[0116] A fusion module 204 is configured to fuse the skeleton feature with the voiceprint feature through a preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature;

[0117] The evaluation module 205 is used to classify the fusion features into bone age using a preset classification network, and to evaluate the bone age of the target person based on the classification result of the bone age classification to obtain the bone age evaluation result of the target person.

[0118] Optional, see Figure 3 The preset image feature extraction network includes a preset global feature extraction network, a preset local feature extraction network and a preset second feature fusion network. The first extraction module 202 includes:

[0119] A first extraction submodule 2021 is configured to perform feature extraction processing on the skeleton graph through the global feature extraction network to obtain global features of the skeleton graph;

[0120] A segmentation submodule 2022 is configured to segment the skeleton map according to a preset segmentation strategy to obtain a corresponding number of skeleton local maps, each of which has an overlapping area with an adjacent skeleton local map, and the resolution scale of the skeleton map is greater than the resolution scale of the skeleton local map;

[0121] The second extraction submodule 2023 is configured to perform feature extraction processing on the local skeleton map through the preset local feature extraction network to obtain local features of the skeleton map;

[0122] The first fusion submodule 2024 is configured to fuse the global features with the local features through the preset second feature fusion network to obtain the skeleton features of the skeleton graph.

[0123] Optional, see Figure 4 The preset voiceprint feature extraction network includes a time feature extraction network, a frequency feature extraction network and a third feature fusion network. The second extraction module 203 includes:

[0124] The third extraction submodule 2031 is configured to perform feature extraction processing on the preset voiceprint sequence through the time feature extraction network to obtain the time feature of the voiceprint sequence;

[0125] The transformation submodule 2032 is configured to transform the voiceprint sequence from the time domain to the frequency domain through a fast Fourier transform to obtain a spectrum of the voiceprint sequence;

[0126] The fourth extraction submodule 2033 is configured to perform feature extraction processing on the spectrum graph through the frequency feature extraction network to obtain frequency features of the voiceprint sequence;

[0127] The second fusion submodule 2034 is configured to fuse the time feature and the frequency feature through the preset third feature fusion network to obtain the voiceprint feature of the voiceprint sequence.

[0128] Optional, see Figure 5 , the transformation submodule 2032 includes:

[0129] The framing unit 20321 is configured to perform framing processing on the voiceprint sequence to obtain a voiceprint sequence after framing processing;

[0130] a windowing unit 20322 configured to perform windowing on the voiceprint sequence after the frame processing to obtain a windowed voiceprint sequence;

[0131] The conversion unit 20323 is configured to perform fast Fourier transform point processing on the windowed voiceprint sequence, convert the voiceprint sequence from the time domain to the frequency domain, and obtain a frequency spectrum of the voiceprint sequence.

[0132] Optional, see Figure 6 Before the second extraction module 203, the device further includes:

[0133] The processing module 206 is configured to pre-process the voiceprint sequence to obtain a pre-processed voiceprint sequence;

[0134] Recognition module 207, configured to perform semantic recognition on the pre-processed voiceprint sequence;

[0135] The determination module 208 is configured to determine that the pre-processed voiceprint sequence is the voiceprint sequence of the target person when the semantic recognition result is the same as the specified content.

[0136] Optional, see Figure 7 The preset first feature fusion network includes a first linear transformation layer, a second linear transformation layer and a first full convolution layer. The fusion module 204 includes:

[0137] A first transformation submodule 2041 is configured to perform a linear transformation on the skeleton features through the first linear transformation layer to obtain skeleton features after linear transformation;

[0138] A second transformation submodule 2042 is configured to perform a linear transformation on the voiceprint feature through the second linear transformation layer to obtain a linearly transformed voiceprint feature, wherein the linearly transformed skeleton feature and the linearly transformed voiceprint feature have the same dimension;

[0139] The convolution submodule 2043 is used to perform convolution processing on the skeleton features after the linear transformation and the voiceprint features after the linear transformation through the first full convolution layer to obtain a fusion feature of the skeleton features and the voiceprint features.

[0140] Optional, see Figure 8 The image feature extraction network, the voiceprint feature extraction network, the first feature fusion network, and the classification network are trained using the same data set, and the device further includes:

[0141] A construction module 209 is configured to construct a data set, wherein the data set includes a sample skeleton diagram, a sample voiceprint sequence, and corresponding age-labeled data. The sample skeleton diagram is the skeleton diagram of the sample person, the age-labeled data is the real age of the sample person, and the sample voiceprint sequence is a voiceprint sequence collected when the sample person speaks according to specified content.

[0142] The training module 210 is configured to jointly train the image feature extraction network, the voiceprint feature extraction network, the first feature fusion network, and the classification network using the data set.

[0143] It should be noted that the bone age assessment device provided in the embodiment of the present invention can be applied to mobile phones, monitors, computers, servers and other devices that can perform bone age assessment.

[0144] The bone age assessment device provided in the embodiment of the present invention can implement each process implemented by the bone age assessment method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0145] See also Figure 9 , Figure 9 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 9 As shown, it includes: a memory 902, a processor 901, and a computer program stored in the memory 902 and executable on the processor 901, wherein:

[0146] The processor 901 is configured to call the computer program stored in the memory 902 and execute the following steps:

[0147] Obtaining a skeleton diagram and voiceprint sequence of the target person, wherein the skeleton diagram is a skeleton diagram of a specified body part, and the voiceprint sequence is collected when the target person speaks according to the specified content;

[0148] Performing feature extraction on the skeleton graph through a preset image feature extraction network to obtain skeleton features of the skeleton graph;

[0149] Extracting features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence;

[0150] Performing feature fusion on the skeleton feature and the voiceprint feature through a preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature;

[0151] The fusion features are classified into bone age categories using a preset classification network, and the bone age of the target person is assessed based on the classification results of the bone age classification to obtain the bone age assessment results of the target person.

[0152] Optionally, the preset image feature extraction network executed by the processor 901 includes a preset global feature extraction network, a preset local feature extraction network, and a preset second feature fusion network, and the feature extraction of the skeleton graph by the preset image feature extraction network to obtain the skeleton features of the skeleton graph includes:

[0153] Performing feature extraction processing on the skeleton graph through the global feature extraction network to obtain global features of the skeleton graph;

[0154] Segmenting the skeleton map according to a preset segmentation strategy to obtain a corresponding number of skeleton local maps, wherein each of the skeleton local maps has an overlapping area with an adjacent skeleton local map, and the resolution scale of the skeleton map is greater than the resolution scale of the skeleton local map;

[0155] Performing feature extraction processing on the local skeleton map through the preset local feature extraction network to obtain local features of the skeleton map;

[0156] The global features and the local features are fused through the preset second feature fusion network to obtain the skeleton features of the skeleton graph.

[0157] Optionally, the preset voiceprint feature extraction network executed by the processor 901 includes a time feature extraction network, a frequency feature extraction network, and a third feature fusion network. The feature extraction of the voiceprint sequence by the preset voiceprint feature extraction network to obtain the voiceprint features of the voiceprint sequence includes:

[0158] Performing feature extraction processing on the preset voiceprint sequence through the time feature extraction network to obtain the time feature of the voiceprint sequence;

[0159] The voiceprint sequence is transformed from the time domain to the frequency domain by fast Fourier transform to obtain a spectrum of the voiceprint sequence;

[0160] Performing feature extraction processing on the spectrum graph through the frequency feature extraction network to obtain the frequency features of the voiceprint sequence;

[0161] The time feature and the frequency feature are fused by the preset third feature fusion network to obtain the voiceprint feature of the voiceprint sequence.

[0162] Optionally, the processor 901 performs the transform of the voiceprint sequence from the time domain to the frequency domain through fast Fourier transform to obtain a spectrum of the voiceprint sequence, including:

[0163] performing frame processing on the voiceprint sequence to obtain a frame-processed voiceprint sequence;

[0164] Performing windowing processing on the voiceprint sequence after the frame processing to obtain a windowed voiceprint sequence;

[0165] Performing fast Fourier transform point processing on the windowed voiceprint sequence, converting the voiceprint sequence from the time domain to the frequency domain, and obtaining a spectrum diagram of the voiceprint sequence.

[0166] Optionally, before the processor 901 extracts features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence, the method further includes:

[0167] Preprocessing the voiceprint sequence to obtain a preprocessed voiceprint sequence;

[0168] performing semantic recognition on the preprocessed voiceprint sequence;

[0169] When the semantic recognition result is the same as the specified content, the pre-processed voiceprint sequence is determined to be the voiceprint sequence of the target person.

[0170] Optionally, the preset first feature fusion network executed by the processor 901 includes a first linear transformation layer, a second linear transformation layer, and a first full convolution layer. The skeleton feature and the voiceprint feature are fused by the preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature, including:

[0171] Performing linear transformation on the bone features through the first linear transformation layer to obtain linearly transformed bone features;

[0172] performing a linear transformation on the voiceprint feature through the second linear transformation layer to obtain a linearly transformed voiceprint feature, wherein the linearly transformed skeleton feature and the linearly transformed voiceprint feature have the same dimension;

[0173] The skeleton features after the linear transformation and the voiceprint features after the linear transformation are convolved by the first full convolution layer to obtain a fusion feature of the skeleton features and the voiceprint features.

[0174] Optionally, the image feature extraction network, the voiceprint feature extraction network, the first feature fusion network, and the classification network executed by the processor 901 are trained using the same data set, and the training steps include:

[0175] Constructing a data set, the data set including a sample skeleton diagram, a sample voiceprint sequence, and corresponding age-labeled data, wherein the sample skeleton diagram is the skeleton diagram of the sample person, the age-labeled data is the real age of the sample person, and the sample voiceprint sequence is a voiceprint sequence collected when the sample person speaks according to specified content;

[0176] The image feature extraction network, the voiceprint feature extraction network, the first feature fusion network and the classification network are jointly trained using the data set.

[0177] It should be noted that the above-mentioned electronic devices can be mobile phones, monitors, computers, servers and other devices that can be used for bone age assessment.

[0178] The electronic device provided in the embodiment of the present invention can implement each process implemented by the bone age assessment method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0179] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the bone age assessment method provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0180] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0181] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A bone age assessment method, characterized in that: The following steps are involved: Obtaining a skeleton diagram and voiceprint sequence of the target person, wherein the skeleton diagram is a skeleton diagram of a specified body part, and the voiceprint sequence is collected when the target person speaks according to the specified content, and the skeleton diagram and voiceprint sequence are collected at the same time; Performing feature extraction on the skeleton graph through a preset image feature extraction network to obtain skeleton features of the skeleton graph; Extracting features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence; Performing feature fusion on the skeleton feature and the voiceprint feature through a preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature; The fusion features are classified into bone age categories using a preset classification network, and the bone age of the target person is assessed based on the classification results of the bone age classification to obtain the bone age assessment results of the target person.

2. The method according to claim 1, wherein The preset image feature extraction network includes a preset global feature extraction network, a preset local feature extraction network, and a preset second feature fusion network. The feature extraction of the skeleton graph by the preset image feature extraction network to obtain the skeleton features of the skeleton graph includes: Performing feature extraction processing on the skeleton graph through the global feature extraction network to obtain global features of the skeleton graph; Segmenting the skeleton map according to a preset segmentation strategy to obtain a corresponding number of skeleton local maps, wherein each of the skeleton local maps has an overlapping area with an adjacent skeleton local map, and the resolution scale of the skeleton map is greater than the resolution scale of the skeleton local map; Performing feature extraction processing on the local skeleton map through the preset local feature extraction network to obtain local features of the skeleton map; The global features and the local features are fused through the preset second feature fusion network to obtain the skeleton features of the skeleton graph.

3. The method according to claim 1, wherein The preset voiceprint feature extraction network includes a time feature extraction network, a frequency feature extraction network, and a third feature fusion network. The voiceprint sequence is subjected to feature extraction by the preset voiceprint feature extraction network to obtain the voiceprint features of the voiceprint sequence, including: Performing feature extraction processing on the preset voiceprint sequence through the time feature extraction network to obtain the time feature of the voiceprint sequence; The voiceprint sequence is transformed from the time domain to the frequency domain by fast Fourier transform to obtain a spectrum of the voiceprint sequence; Performing feature extraction processing on the spectrum graph through the frequency feature extraction network to obtain the frequency features of the voiceprint sequence; The time feature and the frequency feature are fused by the preset third feature fusion network to obtain the voiceprint feature of the voiceprint sequence.

4. The method according to claim 3, wherein The method of transforming the voiceprint sequence from the time domain to the frequency domain by fast Fourier transform to obtain a spectrum of the voiceprint sequence includes: performing frame processing on the voiceprint sequence to obtain a frame-processed voiceprint sequence; Performing windowing processing on the voiceprint sequence after the frame processing to obtain a windowed voiceprint sequence; Performing fast Fourier transform point processing on the windowed voiceprint sequence, converting the voiceprint sequence from the time domain to the frequency domain, and obtaining a spectrum diagram of the voiceprint sequence.

5. The method according to claim 1, wherein Before extracting features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence, the method further includes: Preprocessing the voiceprint sequence to obtain a preprocessed voiceprint sequence; performing semantic recognition on the preprocessed voiceprint sequence; When the semantic recognition result is the same as the specified content, the pre-processed voiceprint sequence is determined to be the voiceprint sequence of the target person.

6. The method according to claim 1, wherein The preset first feature fusion network includes a first linear transformation layer, a second linear transformation layer, and a first full convolution layer. The skeleton feature and the voiceprint feature are fused by the preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature, including: Performing linear transformation on the bone features through the first linear transformation layer to obtain linearly transformed bone features; performing a linear transformation on the voiceprint feature through the second linear transformation layer to obtain a linearly transformed voiceprint feature, wherein the linearly transformed skeleton feature and the linearly transformed voiceprint feature have the same dimension; The skeleton features after the linear transformation and the voiceprint features after the linear transformation are convolved by the first full convolution layer to obtain a fusion feature of the skeleton features and the voiceprint features.

7. The method according to any one of claims 1 to 6, characterized in that The image feature extraction network, the voiceprint feature extraction network, the first feature fusion network, and the classification network are trained using the same data set, and the training steps include: Constructing a data set, the data set including a sample skeleton diagram, a sample voiceprint sequence, and corresponding age-labeled data, wherein the sample skeleton diagram is the skeleton diagram of the sample person, the age-labeled data is the real age of the sample person, and the sample voiceprint sequence is a voiceprint sequence collected when the sample person speaks according to specified content; The image feature extraction network, the voiceprint feature extraction network, the first feature fusion network and the classification network are jointly trained using the data set.

8. A bone age assessment device, characterized in that: The device comprises: An acquisition module is used to acquire a skeleton diagram and a voiceprint sequence of a target person, wherein the skeleton diagram is a skeleton diagram of a specified body part, and the voiceprint sequence is collected when the target person speaks according to specified content, and the skeleton diagram and voiceprint sequence are collected at the same time; A first extraction module is used to extract features from the skeleton graph using a preset image feature extraction network to obtain skeleton features of the skeleton graph; a second extraction module, configured to extract features from the voiceprint sequence using a preset voiceprint feature extraction network to obtain voiceprint features of the voiceprint sequence; a fusion module, configured to fuse the skeleton feature with the voiceprint feature through a preset first feature fusion network to obtain a fusion feature of the skeleton feature and the voiceprint feature; The evaluation module is used to classify the fusion features into bone age through a preset classification network, and to evaluate the bone age of the target person based on the classification results of the bone age classification to obtain the bone age evaluation results of the target person.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the bone age assessment method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the bone age assessment method according to any one of claims 1 to 7.

Images