Detection method, readable storage medium, program product and electronic device

Abstract

The present application relates to the field of audio recognition, and in particular to an abnormal sound detection method, a readable storage medium, a program product, and an electronic device. Through the abnormal sound detection method, the abnormal sound audio data generated by the foldable electronic device during the folding process is input into a first model, and the fault type of the abnormal sound audio generated by the foldable electronic device during the folding process is output based on the first model. Therefore, it is not necessary for engineers to manually judge the fault type of the abnormal sound audio generated by the foldable electronic device, thereby improving the accuracy of judging the fault type of the abnormal sound audio generated when the foldable electronic device is folded.

Description

Technical Field

[0001] The present application relates to the field of audio recognition, and in particular to a detection method, a readable storage medium, a program product and an electronic device. Background Art

[0002] At present, audio recognition has been applied in many fields, such as human-computer interaction, industrial production, abnormal sound detection, etc.

[0003] For example, abnormal noise (noise) or vibration of the electroacoustic or mechanical structure of some devices (such as vibration of the hinge structure of folding screen mobile phones) may bring a bad user experience. Therefore, before the equipment leaves the factory, engineers are required to perform abnormal noise detection on the equipment to detect the equipment with abnormal noise, and make corresponding adjustments to the equipment by analyzing the cause of the abnormal noise.

[0004] In some scenarios, abnormal noise detection is performed by engineers through hearing and experience. However, manual detection is affected by the subjective factors and experience of engineers, and the detection results vary greatly. Therefore, the accuracy of manual abnormal noise detection is low. Summary of the invention

[0005] Embodiments of the present application provide an abnormal sound detection method, a readable storage medium, a program product, and an electronic device.

[0006] In a first aspect, an embodiment of the present application provides a method for detecting abnormal noises, including: obtaining first audio data of a first audio generated during a folding process of a first foldable electronic device to be detected; inputting the first audio data into a first model to obtain a fault type of the abnormal noise generated during the folding process of the first foldable electronic device, wherein the first model is trained based on multiple historical abnormal noise audios generated during the historical folding process of at least one second foldable electronic device, and historical fault types corresponding to each historical abnormal noise audio.

[0007] For example, in some embodiments of the present application, when detecting the fault type of the abnormal sound audio during the folding process of the first foldable electronic device, the first audio data during the folding process of the first foldable electronic device can be directly input into the first model, and the fault type that generates the abnormal sound can be output based on the first model. Therefore, it is not necessary for the staff to judge the fault type of the abnormal sound based on hearing, thereby improving the accuracy and efficiency of judging the fault type.

[0008] The first model is trained based on the historical abnormal sound audio generated when the second foldable electronic device is folded and the historical fault type corresponding to the historical abnormal sound audio, that is, the first model is trained based on the existing abnormal sound audio and the fault type corresponding to the abnormal sound audio. Then, the first audio data of the first foldable electronic device to be detected is input into the first model to obtain the fault type of the first foldable electronic device.

[0009] In a possible implementation of the first aspect, the first foldable electronic device and the second foldable electronic device have the same foldable structure.

[0010] It can be understood that, because the same structure produces similar audio when generating abnormal sound under the same fault type, the historical abnormal sound audio used to train the first model and the historical fault type corresponding to the historical abnormal sound audio are the same foldable structure. This makes the inference result of the trained first model more accurate.

[0011] In a possible implementation of the first aspect above, the first model is obtained by training based on first-category features of multiple historical abnormal sound audios and multiple historical fault types, wherein the first-category features are linear predicted Mel cepstral coefficients of historical abnormal sound audios or features corresponding to linear predicted Mel cepstral coefficients of historical abnormal sound audios; and the first audio data is the first-category features corresponding to the first audio.

[0012] Exemplarily, in some embodiments, the feature corresponding to the linear prediction Mel cepstral coefficient of the historical abnormal sound audio may be, for example, feature data obtained by performing a second-order difference on the linear prediction Mel cepstral coefficient of the historical abnormal sound audio.

[0013] In some embodiments, the historical abnormal sound audio may be preprocessed before extracting the linear prediction Mel cepstral coefficients of the historical abnormal sound audio. For example, the historical abnormal sound audio may be framed (divided into multiple audio frames according to a preset duration), pre-emphasized (increased the energy of high-frequency information in the audio), extracted speech segment frames (extracted frames with sound sources in multiple audio frames), windowed (increased the continuity of each audio frame), etc., so as to extract valid data from the abnormal sound audio.

[0014] In a possible implementation of the first aspect above, the training process of the above-mentioned first model includes: training a first intermediate model based on a first training sample, wherein the first training sample includes a first part of historical audios in multiple historical abnormal sound audios, and a historical fault type corresponding to the first part of historical audios; inputting sample data in a second training sample into the first intermediate model to obtain an inference result of the sample data in the second training sample, wherein the sample data in the second training sample includes a second part of historical abnormal sound audios in multiple historical abnormal sound audios except the first part of historical audios, and the sample label of the sample data of the second training sample is the historical fault type corresponding to the second part of historical abnormal sound audio; determining the first sample data in the sample data in the second training sample, whose corresponding inference result is different from the corresponding sample label; training the first intermediate model based on a third training sample in the first training sample and the second training sample, whose corresponding sample label is the same as the sample label of the first sample data, to obtain the first model.

[0015] It can be understood that in some embodiments of the present application, when training the first model, the historical audio data can be divided into a first part of historical audio data and a second part of historical audio data. The first intermediate model is trained based on the first part of historical audio data and the historical fault type corresponding to the first part of historical audio data. Then the second part of historical audio data is input into the first intermediate model to obtain the corresponding reasoning result. Samples whose reasoning results are different from the historical fault type corresponding to the second part of historical audio data are selected as the first sample data. It can be understood that the first sample data is a sample of reasoning error of the first intermediate model. Therefore, the first intermediate model can be trained based on the first part of historical audio data and the second part of historical audio data corresponding to the label of the first sample data, so as to obtain the first model. Through the above training, the accuracy of the first model can be made higher.

[0016] In a possible implementation of the first aspect above, the first intermediate model is trained based on a third training sample in the first training sample and the second training sample, whose corresponding sample label is the same as the sample label of the first sample data, including: when the number of first sample data is greater than a preset number, or the ratio of the number of first sample data to the number of sample data in the second training sample is greater than a preset ratio, training the first intermediate model based on the third training sample.

[0017] For example, in some embodiments, when the amount of the first sample data exceeds a preset value, the first intermediate model can be trained based on the third training sample. For example, if the percentage of the amount of the first sample data to the amount of the second part and the amount of the historical abnormal sound audio exceeds any value of 5% to 10%, the first intermediate model can be trained based on the third training sample.

[0018] In a possible implementation of the first aspect, the first intermediate model is trained based on a third training sample whose corresponding sample label is the same as the sample label of the first sample data in the first training sample and the second training sample, including: when the number of historical abnormal sound audios corresponding to the first sample label in the first sample data is greater than a preset number, or when the ratio of the number of historical abnormal sound audios corresponding to the first sample label in the first sample data to the number of historical abnormal sound audios corresponding to the first sample label in the second training sample is greater than a preset ratio, the first intermediate model is trained based on the third training sample corresponding to the first sample label.

[0019] Exemplarily, in some embodiments, when the number of historical abnormal sound audios corresponding to the first sample label of the first sample data exceeds a threshold, the first intermediate model can be trained based on the third training sample corresponding to the first sample label. For example, when the number of historical abnormal sound audios corresponding to the first sample label in the first sample data accounts for a percentage of the number of historical abnormal sound audios corresponding to the first sample label in the second training sample that exceeds any value between 5% and 10%, the first intermediate model can be trained based on the third training sample corresponding to the first sample label.

[0020] In a possible implementation of the first aspect, the fault types include: scratching of the rotating shaft mechanism, collision of the synchronization mechanism, and scratching of the transmission structure.

[0021] In a possible implementation of the first aspect, the first audio is audio with abnormal sound during the folding process of the first foldable electronic device.

[0022] In a second aspect, the present application provides an electronic device, the electronic device comprising: a memory for storing instructions; at least one processor for executing instructions so that the device implements the first aspect and any possible implementation method of the first aspect. The beneficial effects that can be achieved in the second aspect can refer to the beneficial effects of the method provided in any implementation of the first aspect, and will not be repeated here.

[0023] In a third aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and when the instructions are executed by a device, the computer implements the method provided in the first aspect and any possible implementation of the first aspect. The beneficial effects that can be achieved in the third aspect can refer to the beneficial effects of the method provided in any implementation of the first aspect, and will not be repeated here.

[0024] In a fourth aspect, the present application provides a computer program product, which, when running on a device, enables the device to implement the method provided in the first aspect and any possible implementation of the first aspect. The beneficial effects that can be achieved in the fourth aspect can refer to the beneficial effects of the method provided in any implementation of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1A According to some embodiments of the present application, a schematic diagram of an electronic device is shown;

[0026] Figure 1B According to some embodiments of the present application, a partial schematic diagram of a folding shaft 30 is shown;

[0027] Figure 2 A flowchart for implementing abnormal sound detection on equipment is shown;

[0028] Figure 3 According to some embodiments of the present application, a flow chart for detecting the cause of abnormal audio sound is shown;

[0029] Figure 4 According to some embodiments of the present application, a flowchart of training a first model is shown;

[0030] Figure 5 According to some embodiments of the present application, a schematic diagram of abnormal sound audio is shown;

[0031] Fig. 6A According to some embodiments of the present application, a spectrum diagram of a speech segment frame 1 is shown;

[0032] Figure 6B According to some embodiments of the present application, a spectrum diagram of a speech segment frame 2 is shown;

[0033] Figure 6C According to some embodiments of the present application, a spectrum diagram of a speech segment frame 3 is shown;

[0034] Fig.6D According to some embodiments of the present application, a spectrum diagram of a speech segment frame 4 is shown;

[0035] Figure 7 According to some embodiments of the present application, a flowchart for implementing training a first model is shown;

[0036] Fig. 8A According to some embodiments of the present application, a comparison diagram of the predicted results and the actual results of the first model is shown in Figure 1;

[0037] Figure 8B According to some embodiments of the present application, the comparison between the predicted results of the first model and the actual results is shown. Figure 2 ;

[0038] Fig. 9 According to some embodiments of the present application, a schematic structural diagram of a detection device 00 is shown. DETAILED DESCRIPTION

[0039] Illustrative embodiments of the present application include, but are not limited to, a detection method, a readable storage medium, a program product, and an electronic device.

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0041] As mentioned above, in some scenarios, engineers are required to judge the cause of abnormal noise produced by the equipment through hearing and experience. Therefore, the results of abnormal noise detection are greatly affected by the subjective influence of engineers, and the accuracy and stability of the abnormal noise detection results are not high.

[0042] For example, Figure 1A According to some embodiments of the present application, a schematic diagram of an electronic device is shown.

[0043] like Figure 1A As shown, the electronic device 100 is taken as an example of a folding mobile phone to introduce the folding process of the folding mobile phone.

[0044] Reference Figure 1A The electronic device 100 includes a first body 10 , a second body 20 and a folding axis 30 .

[0045] The user can adjust the angle between the first body 10 and the second body 20 of the electronic device 100 through the folding axis 30, so as to fold the electronic device 100 to a corresponding state (for example, from Figure 1A In some cases, the folding axis 30 of the electronic device 100 may generate an abnormal sound (hereinafter referred to as an abnormal sound) when the folding axis 30 of the electronic device 100 generates an abnormal sound due to process problems or other reasons. Therefore, after the folding axis 30 of the electronic device 100 generates an abnormal sound, it is necessary to perform an abnormal sound detection on the folding axis of the electronic device 100.

[0046] For example, before the electronic device 100 leaves the factory, it is necessary to detect the folding process of the electronic device 100 and collect the audio generated by the folding axis 30 during the folding process of the electronic device 100. The electronic device 100 with abnormal audio is used as a sample for abnormal sound detection to determine the fault type of the abnormal sound generated by the folding axis 30 of the electronic device 100.

[0047] Figure 1B According to some embodiments of the present application, a partial schematic diagram of a folding axis 30 is shown.

[0048] For example, refer to Figure 1B As shown, the folding shaft 30 includes a first supporting plate 31 , a second supporting plate 32 , a rotating shaft 33 , a rotating shaft mechanism 34 , a synchronization mechanism 35 and a damping mechanism 36 .

[0049] In some embodiments, the rotating shaft mechanism 34 may include, for example, a main swing arm 341 and a secondary swing arm 342 .

[0050] Understandably, Figure 1B The structure of the folding axis 30 shown is only an example. In other embodiments, the folding axis 30 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently.

[0051] The first support plate 31 and the second support plate 32 can rotate around the rotation shaft 33 based on the rotation shaft mechanism 34 , the synchronization mechanism 35 , and the damping mechanism 36 .

[0052] For example, the first support plate 31 and the second support plate 32 rotate based on the main swing arm 341, and are limited based on the auxiliary swing arm 343 to prevent the first support plate 31 and the second support plate 32 from deviating in a direction perpendicular to the rotation plane. The synchronization mechanism 35 can, for example, be engaged by gears to ensure the synchronization of the rotation of the first support plate 31 and the second support plate 32. The damping mechanism 36 can provide a damping force to provide an opening and closing feel of the electronic device 100.

[0053] The first support plate 31 and the second support plate 32 are connected to the first body 10 and the second body 20 respectively.

[0054] For example, when the electronic device 100 rotates through the folding shaft 30, each structure on the rotating shaft mechanism 34, the synchronization mechanism 35 and the damping mechanism 36 on the folding shaft 30 needs to move (or rotate) relative to each other. If the above structures are not assembled accurately or the structures are not processed accurately, the various structural parts may produce abnormal sound audio after colliding and rubbing against each other. Therefore, it is necessary to detect the folding process of the electronic device 100 to check the electronic device 100 with abnormal sound and analyze the cause of the abnormal sound audio.

[0055] For example, Figure 2 A flowchart for implementing abnormal noise detection on equipment is shown.

[0056] like Figure 2 As shown, the process includes:

[0057] S201, set up abnormal sound interception equipment in the production line to detect the audio when the folding screen mobile phone is folded.

[0058] Exemplarily, in some embodiments, an abnormal sound interception device may be provided in the production line of the equipment to perform audio detection on the use process of the equipment.

[0059] For example, taking a folding phone as an example, in the production line, the folding process of the folding phone needs to be tested, and the abnormal sound detection equipment can collect the audio of the folding process of the folding phone to determine whether the folding process of the folding phone meets the requirements.

[0060] S202, based on the abnormal noise interception device, intercepts the prototype of the folding screen mobile phone that produces abnormal noise during the folding process.

[0061] For example, after collecting audio during the use of the device, the abnormal sound interception device can determine whether the audio is abnormal.

[0062] For example, in some embodiments, the volume of audio generated by the foldable phone during the bending or opening process has an audio threshold. When the volume of audio generated during the folding process of the foldable phone exceeds the audio threshold, it can be determined that the foldable phone has an abnormality during the folding process. The abnormal sound interception device can intercept the foldable phone as a prototype.

[0063] For example, refer to Figure 1A During the folding process of the foldable mobile phone, the abnormal sound interception device can collect the audio data generated during the folding process of the foldable mobile phone through the microphone 40 and detect the audio data.

[0064] S203, engineers perform a folding test on the prototype that generates the abnormal noise, and analyze the cause of the abnormal noise generated by the prototype.

[0065] For example, after determining that a prototype has abnormal noise, engineers can perform an abnormal noise test to analyze the cause of the abnormal noise generated by the equipment.

[0066] For example, after the abnormal noise interception device identifies a folding phone that makes an abnormal noise, engineers can manually fold the folding phone and determine the cause of the abnormal noise through hearing and experience.

[0067] It is understandable that in the above process, although the abnormal noise interception device can be used to determine the device with abnormal noise when performing abnormal noise detection on the device, when determining the cause of the abnormal noise, the engineer is still required to analyze the cause of the abnormal noise through hearing and experience when the device produces abnormal noise. Therefore, the result of abnormal noise detection is greatly affected by the subjective influence of the engineer, and the accuracy and stability of the abnormal noise detection result are not high.

[0068] It can be understood that there are many types of faults that produce abnormal noises when folding foldable electronic devices. For example, the structure in the hinge mechanism scrapes against other structures when folding (hereinafter referred to as the hinge mechanism scrape) due to size / surface quality / assembly / non-conformity, which will produce abnormal noises (for example, the secondary swing arm size is not qualified and scrapes against other structures), the synchronization mechanism (the structure used to support the synchronous folding and unfolding of the foldable mobile phone) is improperly installed, the size / surface quality is not qualified, etc., resulting in collision and scraping between the synchronization mechanism and other structures to produce abnormal noises (hereinafter referred to as synchronization mechanism collision), or the transmission structure in the damping mechanism and the synchronization mechanism (such as gear sets, worm gears, connecting rods, etc.) scrapes against other structures (transmission structure scrapes), etc. Generally speaking, the sound produced by the collision of objects is related to the structural characteristics / material stiffness of the objects themselves, so the audio characteristics (such as linear predictive Mel frequency cepstral coding (LPMFCC), etc.) of the abnormal noises produced by collision and friction (fault types) between different structures are usually different.

[0069] Based on this, the present application proposes a detection method, which determines the fault type (such as scratching of the hinge mechanism, collision of the synchronization mechanism, scratching of the transmission structure, etc.) that produces abnormal noise during the folding process of a foldable electronic device (as a first foldable electronic device) through a pre-trained first model based on the audio features (such as LPMFCC features, etc., as the first audio data) of the abnormal noise audio during the folding process of a foldable electronic device (as a first foldable electronic device).

[0070] In some embodiments, the first model is obtained based on training of a training set, wherein the sample data in the training set may be an abnormal sound audio (or an audio feature of an abnormal sound) when an abnormal sound occurs during the folding process of the foldable electronic device, and the label of the sample data in the training set may be a fault type that generates the abnormal sound.

[0071] The training set may include, for example, historical abnormal sound audio data of the second foldable electronic device and historical fault types corresponding to the historical abnormal sound audio data. The second foldable electronic device and the first foldable electronic device may be the same electronic device or different electronic devices.

[0072] After obtaining the abnormal sound audio generated when the foldable electronic device is folded, the abnormal sound audio features corresponding to the abnormal sound audio can be input into the first model based on the pre-trained first model, so as to determine the fault type of the abnormal sound audio generated when the foldable electronic device is folded.

[0073] In some embodiments, the abnormal sound audio feature is determined based on the LPMFCC extracted from the abnormal sound audio. For example, the abnormal sound audio is Fourier transformed to convert the abnormal sound audio from the time domain to the frequency domain, and then the LPMFCC is extracted, and the first-order difference and second-order difference of the LPMFCC are calculated to obtain the abnormal sound audio feature. The abnormal sound audio feature can represent the abnormal sound audio generated by the electronic device. Processing the abnormal sound audio through LPMFCC can reduce the noise interference of the abnormal sound audio, making the detection result more accurate.

[0074] In some embodiments, the fault types include shaft mechanism scraping (e.g., secondary swing arm scraping), synchronization mechanism collision (e.g., collision caused by inaccurate gear meshing), transmission structure scraping, etc. It is understandable that in different models of foldable electronic devices, the fault types that produce abnormal sounds may be different, and this application does not limit the fault types that produce abnormal sounds.

[0075] In some embodiments, the fault type can also be classified according to the specific structure of the folding mechanism of the foldable electronic device, according to the type of parts / components / structures in the folding mechanism, or according to the movement mode of the parts / components / structures in the folding mechanism, or according to the cause of abnormal noise in the parts / components / structures in the folding mechanism (such as friction, collision, etc.), or classified in other ways, which are not limited here.

[0076] In some embodiments, the first model is a convolutional neural network (CNN) model. The training audio features and the fault types corresponding to the training audio features are input into the convolutional neural network model, thereby training the first model.

[0077] For example, the abnormal sound audio and the fault type corresponding to the abnormal sound audio in the training sample (the label corresponding to the abnormal sound audio, hereinafter referred to as the real label) are input into the CNN model, and the abnormal sound audio data is processed by each layer in the CNN model (such as the convolution layer, the pooling layer, and the fully connected layer). Thus, the predicted label corresponding to each abnormal sound audio is output. If the difference between the predicted label and the real label is large. For example, the accuracy of the CNN model predicted label is less than the accuracy threshold (for example, the accuracy threshold is any value between 90% and 95%). Then the CNN model adjusts the parameters of each layer based on the error, thereby improving the accuracy of the CNN model until the accuracy of the CNN model exceeds the accuracy threshold.

[0078] In some embodiments of the present application, the fault type corresponding to the abnormal sound audio feature is a problem with the structure that specifically generates the abnormal sound audio, such as scratches on the shaft mechanism, collisions on the synchronization mechanism, scratches on the transmission structure, and other problems.

[0079] Through the above solution, there is no need to manually determine the specific cause of the abnormal noise generated by the equipment, thereby improving the detection efficiency of the abnormal noise of the equipment and the accuracy of the detection results.

[0080] Next, the process of detecting the cause of abnormal audio noise generated during the folding process of a foldable electronic device in an embodiment of the present application is introduced.

[0081] Figure 3 According to an embodiment of the present application, a flow chart for detecting the cause of abnormal audio sound is shown.

[0082] Exemplarily, the execution subjects of the following processes may all be detection devices. In some embodiments, the detection devices include but are not limited to mobile phones, tablet computers, head-mounted displays, laptop computers, etc. When introducing the following processes, the execution subjects of each process are not limited.

[0083] In other embodiments, the execution entities of the following processes may also be different devices, and the embodiments of the present application do not limit the execution entities of the following processes.

[0084] like Figure 3 As shown, the process includes:

[0085] S301, obtaining abnormal sound audio data during the folding process of the foldable electronic device.

[0086] Exemplarily, in some embodiments of the present application, in the folding test process of the foldable electronic device, the detection device can collect the test audio data of the foldable electronic device during the folding process, and determine whether the test audio data belongs to abnormal sound audio. For example, when the amplitude (such as decibel (db), sound pressure (pa), etc.) of the test audio data of the foldable electronic device does not fall within the range of the audio threshold, it can be determined that the test audio data of the foldable electronic device is abnormal sound audio, and thus it is necessary to analyze the fault type that produces the abnormal sound audio.

[0087] In some embodiments, the detection device may also directly obtain audio data generated when a foldable mobile phone that has been determined to have an abnormal sound is folded. For example, a foldable electronic device with an abnormal sound may be obtained through other devices, and then the detection device collects the audio generated when the foldable electronic device is folded through a microphone to obtain the abnormal sound audio.

[0088] S302: extracting abnormal sound audio features from the abnormal sound audio data.

[0089] Exemplarily, after the detection device determines the abnormal sound audio, the abnormal sound audio features corresponding to the abnormal sound audio can be extracted. For example, after the detection device obtains the abnormal sound audio, the abnormal sound audio can be framed (according to the preset time as one frame, the abnormal sound audio is divided into multiple frames of audio), and the speech segment frame (audio with abnormal sound source) is extracted to remove invalid data (audio segment without sound or interference audio) in the abnormal sound audio. And the speech segment frame is windowed (such as rectangular window, Hamming window, etc.) to maintain the continuity of each speech segment frame. Then, the speech segment frame is Fourier transformed to convert the abnormal sound audio from time domain to frequency domain, and then LPMFCC is extracted, and the first-order difference and second-order difference of LPMFCC are obtained to obtain the abnormal sound audio features.

[0090] In some embodiments, since the sound frequency generated by collision and friction between mechanisms / structures is usually high, the detection device can first enhance the high-frequency components in the abnormal sound audio (for example, pre-emphasize the high-frequency part to enhance the energy of the high-frequency part) and then extract the speech segment frame. In this way, the high-frequency information components in the abnormal sound audio can be made more obvious, which is conducive to improving the integrity of the abnormal sound audio information.

[0091] S303: Input the abnormal sound audio feature into the first model to determine the fault type of the abnormal sound generated during the folding process of the foldable electronic device.

[0092] For example, in some embodiments of the present application, after the detection device extracts the abnormal sound audio feature, the abnormal sound audio feature can be input into the first model. The first model processes the abnormal sound audio feature and can output the fault type corresponding to the abnormal sound audio feature.

[0093] For example, in some embodiments of the present application, the first model is obtained based on training of a training set, the sample data of the training set may be the audio features of the abnormal sound audio of the foldable electronic device that makes an abnormal sound during the folding process, and the label of the sample data in the training set may be the type of fault that produces the abnormal sound. After the first model is trained, the first model may be configured in a detection device, and after the detection device obtains the abnormal sound audio features, the abnormal sound audio features may be input into the trained first model, thereby obtaining a label corresponding to the abnormal sound audio, and the label indicates the type of fault corresponding to the abnormal sound audio.

[0094] Based on the above process, after determining that the foldable electronic device has an abnormal sound during the folding process, the detection device can obtain the abnormal sound audio, then extract the abnormal sound audio features corresponding to the abnormal sound audio, and input the abnormal sound audio features into the first model, and finally obtain the fault type of the foldable electronic device generating the abnormal sound. The entire process does not require manual analysis of the fault type of the foldable electronic device generating the abnormal sound based on hearing and experience, thereby avoiding the influence of manual subjective judgment on the analysis results and improving the accuracy of the detection of the cause of the abnormal sound of the foldable electronic device.

[0095] Next, the process of folding detection of foldable electronic devices is introduced.

[0096] For example, Figure 4 According to some embodiments of the present application, a flowchart for training a first model is shown.

[0097] like Figure 4 As shown, the process includes:

[0098] S410, obtaining test audio data of the foldable electronic device during a folding test.

[0099] Exemplarily, the foldable electronic device needs to undergo a folding test on the folding axis before leaving the factory, and the test audio data generated when the folding axis is folded is used to determine whether there is a problem with the folding axis. For example, in some embodiments of the present application, the abnormal sound detection device can collect the test audio data generated during the bending test of the foldable electronic device, so as to determine whether there is a problem with the folding axis of the foldable electronic device.

[0100] It is understood that in some embodiments of the present application, the abnormal sound detection device can collect the test audio data generated when the foldable electronic device is bent through a microphone. In other embodiments, the test audio data can also be collected by other devices, and then the collected test audio data is sent to the abnormal sound detection device.

[0101] S420, based on the test audio data, determining the abnormal sound audio of the foldable electronic device having the abnormal sound.

[0102] Exemplarily, after the abnormal sound detection device obtains the test audio data of the foldable electronic device, it can determine whether the size of the test audio data falls within the range of the audio threshold, and then determine whether the audio data of the foldable electronic device is abnormal sound audio.

[0103] S430, analyzing the abnormal sound audio corresponding to the foldable electronic device with the abnormal sound, determining the fault type that generates the abnormal sound audio, and processing the abnormal sound audio.

[0104] Illustratively, in some embodiments of the present application, after the abnormal sound detection device tests a foldable electronic device that has an abnormal sound during the bending process, the staff can analyze the type of fault that causes the abnormal sound based on the test audio data (abnormal sound audio) of the foldable electronic device.

[0105] It can be understood that the following S431 to S438 are all processes for processing abnormal sound audio. For example, based on the abnormal sound audio and the fault type of the corresponding abnormal sound audio analyzed, the first model is generated. In addition, the execution order of S431 to S432 is not limited.

[0106] S431, enhances the high-frequency information in abnormal sound audio.

[0107] Exemplarily, in some cases, in the abnormal sound audio data, the high-frequency energy is smaller than the low-frequency energy. For example, during the folding process of the folding axis of a foldable electronic device, due to material and other issues, high-frequency and low-energy abnormal sound audio may be generated. Therefore, in some embodiments of the present application, the abnormal sound audio can be pre-emphasized, and the pre-emphasis can keep the low-frequency part of the signal unchanged while increasing the energy of the high-frequency part of the signal. Since the high-frequency part usually contains more information, the pre-emphasis processing can effectively improve the signal-to-noise ratio of the abnormal sound audio signal and improve the quality of the abnormal sound audio.

[0108] S432, dividing the abnormal sound audio into frames, and extracting the speech segment frames containing the abnormal sound source.

[0109] For example, in some embodiments, since the audio generated by the abnormal sound source may not be throughout the entire abnormal sound audio (the abnormal sound audio may also include other interfering audio and a part without audio). Therefore, it is necessary to divide the abnormal sound audio into frames and extract the abnormal sound audio frames containing the abnormal sound source. The speech segment frame refers to the frame in the abnormal sound audio that actually contains the abnormal sound source, as opposed to the silent or background noise part.

[0110] For example, the abnormal sound audio can be framed with any duration between 0.02s and 0.03s. For example, in an embodiment of the present application, 0.025s is a frame. The frame shift can be determined based on the duration of a frame. For example, the frame shift can be half the duration of a frame. In some embodiments of the present application, the frame shift can be 0.01s. The frame shift is the amount of overlap between the previous and next frames, that is, the overlapping part of the tail of the previous frame and the head of the next frame. This overlap is to ensure that the signal can maintain a certain continuity when the frame is processed. This is done to prevent discontinuity between the two frames so that each frame of audio has the components of the previous frame.

[0111] Extracting speech segments, for example, involves dividing the abnormal sound audio into multiple audio frames, and calculating the amplitude vibration energy and zero crossing rate of each frame of audio. Then, the first few frames of speechless segments are used as a benchmark. Compare the short-time energy (the energy contained in the signal in a specific time frame) and the short-time zero crossing rate of the speechless segment to obtain the speech segment frame. The short-time zero crossing rate (ZCR) in the audio signal refers to the ratio of a signal sign change, that is, the number of times the audio signal changes from positive to negative or from negative to positive in each frame. It can be used to describe the dynamic characteristics and frequency characteristics of the audio.

[0112] For example, refer to Figure 5 , Figure 5 According to some embodiments of the present application, a schematic diagram of abnormal sound audio is shown.

[0113] like Figure 5 As shown, the schematic diagram of the abnormal sound audio is a relationship diagram between amplitude (db) and time (ms). It can be understood that the amplitude of the audio with speech frames is around 60db to 85db.

[0114] Reference Figure 5 In the abnormal sound audio, there are 6 audio bands with amplitudes exceeding 6db to 85db. When extracting the speech frame of the abnormal sound audio, the audio before the abnormal sound audio can be discarded (for example, the previous audio may be the noise that just appeared, for example, the audio of the first 100ms is discarded, and only the frames corresponding to the audio exceeding 60db in the audio after 100ms are extracted as the speech frame), and then the speech frame 1, the speech frame 2, the speech frame 3 and the speech frame 4 are extracted. Then, the four speech frames are further processed.

[0115] S433, adding windows to the frames with speech segments.

[0116] Exemplarily, since the abnormal sound audio is processed by framing, breaks will occur between frames, resulting in discontinuity of the signal. In some embodiments, each frame of audio can be multiplied by a window function by windowing, so that the audio is more continuous. The window function is a piece of data with the same length as the frame, which gradually changes from a minimum value to a maximum value, and then gradually changes back to a minimum value. Among them, the window function can include a rectangular window, a Hamming window, a Hanning window, etc. In some embodiments of the present application, the frame with a segment can be processed by a Hamming window.

[0117] S434, performing Fourier transform on the speech segment frames.

[0118] Exemplarily, after windowing the speech segment frame, Fourier transform (for example, fast Fourier transform) can be performed on the speech segment frame, so as to convert each frame in the time domain into the frequency domain to obtain the frequency spectrum of the frame.

[0119] For example, FIG. 6A to FIG. 6D According to some embodiments of the present application, a frequency spectrum diagram of a speech segment frame is shown.

[0120] in, Fig. 6A The spectrum of the 72nd frame (frame 1 with speech) is shown. It can be understood that the amplitude of the abnormal sound audio is between -0.03db and 0.03db, which is higher than the amplitude of the non-speech frame (for example, the audio with an amplitude of about 0db). Figure 6B The spectrum of the 166th frame (frame 2 with speech) is shown, and the amplitude of the abnormal sound audio is between -0.02 and 0.02 db. Figure 6C The spectrum of the 336th frame (frame 3 with speech) is shown, and the amplitude of the abnormal sound audio is between -0.008 and 0.008 dB. Fig.6D The spectrum diagram of the 342nd frame (frame 4 with speech) is shown, and the amplitude of the abnormal sound audio is between -0.015 and 0.02 db.

[0121] S435, extracting LPMFCC of the speech segment frame to determine the abnormal sound audio features.

[0122] For example, in some embodiments of the present application, after obtaining the spectrum of the speech frame, the LPMFCC coefficients can be extracted from the spectrum signal for linear prediction, thereby removing the noise in the frequency. Among them, LPMFCC combines the concepts of linear prediction coefficients (LPC) and Mel frequency cepstral coefficients (MFCC).

[0123] LPC is a method for predicting the current sample value based on the known sample values ​​of the random signal in the past. It is based on a linear equation, and its purpose is to find a set of suitable prediction coefficients so that the mean square value of the prediction error is minimized. In some embodiments of the present application, a series of coefficients can be generated by the sample values ​​of the previous frames, and the final audio signal value can be linearly approximated according to the minimum mean square error, that is, an audio signal is virtualized, thereby reducing the noise interference in the audio. For example, the interference caused by the rotation friction of the folding axis of a foldable electronic device during the folding test in the abnormal sound audio data can be reduced.

[0124] Then, the Mel coefficients are extracted from the virtual audio signal, and the first-order difference and second-order difference of the Mel coefficients are calculated to obtain the abnormal sound audio feature corresponding to the speech segment frame. For example, in some embodiments of the present application, the abnormal sound audio feature is represented by a 1×21 vector.

[0125] For example, Table 1 shows a table of abnormal sound audio features.

[0126] Table 1

[0127]

[0128] Referring to Table 1, in some embodiments, a row in Table 1 is an abnormal sound audio feature corresponding to a speech segment frame, wherein the label indicates the type of fault type corresponding to the abnormal sound audio feature.

[0129] S436, adding labels to corresponding abnormal sound audio features, and training a first intermediate model based on the abnormal sound audio features and the corresponding labels.

[0130] For example, in some embodiments of the present application, after the detection device obtains the abnormal sound audio feature, it can add a corresponding label to the abnormal sound audio based on the analysis result of the audio feature, and the label is the fault type corresponding to the abnormal sound audio feature. And the first intermediate model is trained based on the abnormal sound audio feature and the corresponding label.

[0131] For example, based on the 21×1 vector in Table 1 and the label corresponding to the vector, the first intermediate model is trained.

[0132] For example, during the folding process of a foldable electronic device, the fault types that generate abnormal sound audio include: scratching of the rotating shaft mechanism, collision of the synchronization mechanism, scratching of the transmission structure, etc. The scratching of the rotating shaft mechanism can be marked as label 1, the collision of the synchronization mechanism can be marked as label 2, and the scratching of the transmission structure can be marked as label 3.

[0133] It can be understood that in some embodiments, 100 audio features corresponding to each label can be obtained, and the 100 audio features can be based on the audio features corresponding to the audio data generated when different foldable electronic devices are folded. When there are fewer foldable electronic devices as samples, the 100 audio features can also be obtained by repeatedly folding the same foldable electronic device. In this way, each label is actually trained with only one audio feature, so the accuracy of the first intermediate model may not be very good, so the first intermediate model needs to be further trained.

[0134] S437, continue to obtain abnormal sound audio features, and input the abnormal sound audio features into the first intermediate model to obtain an output result.

[0135] For example, in some embodiments of the present application, after the first intermediate model is trained, an abnormal sound audio feature may be further input to the first intermediate model, and the abnormal sound audio feature may be obtained based on the process from S431 to S435. Then, an output result of the first intermediate model for the abnormal sound audio feature is obtained.

[0136] S438: Based on the samples whose output results are different from the labels corresponding to the input abnormal sound audio features, the first intermediate model is trained to obtain a first model.

[0137] For example, in some embodiments of the present application, since the accuracy of the first intermediate model is not high, the first intermediate model needs to be further trained. For example, based on the sample data with errors in the labels corresponding to the output results of the first intermediate model and the corresponding abnormal sound audio features, the first intermediate model is continued to be trained, and after the accuracy of the first intermediate model exceeds the output accuracy threshold (for example, the output accuracy threshold is any value between 95% and 100%), it can be determined that the first model is generated.

[0138] For example, if the label corresponding to the audio data input to the first intermediate model is 1, but the label output by the first intermediate model is 3, the first intermediate model is trained based on the currently input abnormal sound audio data and the historical abnormal sound audio data corresponding to the label 1, and label 1. Thus, the first intermediate model learns to output samples with errors, and the processes of S437 and S438 are repeated until the accuracy of the output result of the first intermediate model for the new sample exceeds the accuracy threshold.

[0139] Based on the above process, after obtaining the abnormal sound audio of the foldable electronic device with abnormal sound, the abnormal sound audio can be preprocessed first, such as pre-emphasis, framing, windowing, etc., which can effectively extract the segment frame containing the abnormal sound source, and extract LPMFC from the segment frame to reduce the interference of noise on the abnormal sound audio, and then extract the abnormal sound audio features. Then, the first intermediate model is trained based on the abnormal sound audio features and the labels of the abnormal sound audio feature analysis. Then, the first intermediate model is gradually trained according to the subsequent training samples, so as to improve the accuracy of the first intermediate model and finally obtain the first model.

[0140] Next, the training process of the first model is introduced.

[0141] For example, Figure 7 According to some embodiments of the present application, a flowchart for implementing training a first model is shown.

[0142] like Figure 7 As shown, the process includes:

[0143] S701, obtaining a first training sample and a second training sample.

[0144] Exemplarily, in some embodiments of the present application, a certain amount of training samples are required when training the first model. The training sample may be, for example, the audio of a foldable electronic device (prototype) when the fault type has been determined and the folding is performed. For example, the training sample includes sample data and sample labels (fault type of the prototype generating abnormal sound). In the training sample, at least one training sample corresponding to each sample label is included.

[0145] In some embodiments of the present application, the abnormal sound audio generated by the prototype in the training sample when folded corresponds to only one sample label in a time period. In other words, a section of abnormal sound audio does not include audio formed by the superposition of audio corresponding to multiple sample labels.

[0146] In some embodiments of the present application, the training samples may be divided into a first training sample and a second training sample, wherein in the first training sample, each sample label corresponds to at least one abnormal sound audio.

[0147] S702: Train a first group of models based on a first training sample to obtain a first intermediate model.

[0148] Exemplarily, a first group of models may be trained based on the first training samples to obtain a first intermediate model. It is understandable that the first intermediate model has a lower applicability due to the small number of training samples (training is performed only through the first training samples).

[0149] In some embodiments of the present application, the first model may be a convolutional neural network model (CNN model). Exemplarily, the convolutional neural network model includes: an input layer, a convolution layer, an activation function, a pooling layer, a fully connected layer, and an output layer.

[0150] The input layer is the starting point of the CNN model and is used to receive abnormal sound audio features as input data.

[0151] Convolutional layer: The convolutional layer is the core part of the CNN model. The convolutional layer includes learnable filters (or convolution kernels) to perform convolution operations on the input data to extract local features. Each filter slides on the input data, performs a dot product operation, and generates a feature map.

[0152] Activation function: After the convolution operation, an activation function (such as ReLU) is usually applied to increase the nonlinearity of the model.

[0153] Pooling layer: The pooling layer is usually located after the convolution layer to reduce the spatial size of the data (such as downsampling), reduce the number of parameters, prevent overfitting, and increase the robustness of the model. Common pooling operations include maximum pooling and average pooling. For example, the feature map generated by the convolution layer.

[0154] Fully connected layer: After multiple convolutional and pooling layers, there are usually one or two fully connected layers to integrate the previously extracted features and output the final prediction results.

[0155] In an embodiment of the present application, the CNN model can minimize a loss function (such as a cross entropy loss function, a function for comparing two probability distributions) based on a back propagation algorithm and a gradient descent optimizer. For example, the cross entropy between the predicted probability distribution (the distribution of sample labels inferred by the CNN model based on the abnormal sound audio) and the true probability distribution (the distribution of the true sample labels corresponding to the abnormal sound audio) can be used to evaluate the performance of the model. The smaller the cross entropy, the closer the predicted probability distribution is to the true probability distribution, and the better the performance of the model. Therefore, the goal of the training model is to minimize the cross entropy loss function. By continuously adjusting the network parameters to minimize the loss function, the CNN model can gradually learn how to extract effective features from the input abnormal sound audio features and make accurate predictions. For example, if the predicted result does not match the true result (the corresponding sample label in the training sample) (or the accuracy of the predicted result is lower than the accuracy threshold), the CNN model can adjust the parameters of each layer (such as the filter of the convolution layer and the parameters in the fully connected layer) based on the back propagation algorithm and the gradient descent optimizer. So that the accuracy of the prediction result of the fully connected layer exceeds the accuracy threshold, the CNN model is trained.

[0156] Output layer: In some embodiments of the present application, it may be a softmax layer of a classification task, which is used to input the classification results of the abnormal sound audio. For example, the output layer can output the sample label corresponding to the abnormal sound audio feature, thereby determining the corresponding fault type.

[0157] It can be understood that the first intermediate model can be trained based on the training samples in the first training samples.

[0158] For example, in some cases, there are fewer training samples, for example, in the first training sample, there is only one prototype corresponding to one sample label (multiple sample labels may correspond to one prototype, in which case the abnormal sound source caused by each sample label during the prototype folding process is in different time periods of the abnormal sound audio). In this way, in order to train the first group of models, multiple folding tests can be performed on the prototype corresponding to the first training sample, and the abnormal sound audio of the multiple folding tests can be collected, so as to obtain more abnormal sound audio corresponding to the first training sample.

[0159] S703: Use the first intermediate model to infer sample data in the second training sample to obtain an inference result of the first intermediate model on the second training sample.

[0160] For example, in some embodiments of the present application, after the training of the first intermediate model is completed, the second training sample can be input into the first intermediate model, and the inference result corresponding to the second training sample can be obtained.

[0161] It is understood that in some embodiments of the present application, the accuracy of the first intermediate model trained only by the first training sample may not be very high, therefore, the first intermediate model can be inferred by the second training sample, and the inference result can be compared with the sample result in the second sample. In this way, the accuracy of the first intermediate model can be determined.

[0162] S704, obtaining first sample data in the second training sample, whose inference result is different from the sample label.

[0163] For example, in some embodiments of the present application, the training samples corresponding to the same sample label in the second training sample may be significantly different from the training samples in the first training sample. Therefore, when the first intermediate model is inferred through the second training sample, the first intermediate model may make inference errors. Therefore, the training samples in the second training sample that are inferred incorrectly by the first intermediate model can be counted as the first sample data.

[0164] S705, when the quantity of the first sample data meets the conditional threshold, re-input the training samples with the same sample results as the first sample data into the first intermediate model to train the first model.

[0165] Exemplarily, in some embodiments of the present application, the first sample data is a training sample in which the first intermediate model has reasoning errors. The conditional threshold is, for example, the percentage of the number of first sample data to the number of second training samples (for example, the conditional threshold is any value between 5% and 10%). Therefore, if the number of first sample data meets the conditional threshold, it means that the error rate of the first intermediate model is too high when reasoning about the second training sample. Therefore, the first intermediate model needs to continue to learn the features of the training samples in the first sample data to improve the adaptability of the first intermediate model.

[0166] For example, the training samples in the first training sample and the second training sample whose sample labels are the same as the sample labels of the first sample data (as the third training sample) are input into the first intermediate model, so as to train the first intermediate model and finally obtain the first model.

[0167] In other embodiments, the conditional threshold may be, for example, the percentage of the first sample data in the second training sample. If the percentage of the first sample data in the second training sample exceeds the conditional threshold (for example, the error rate threshold is any value between 5% and 10%), the training sample with the same sample result as the first sample data may be re-input into the first intermediate model and trained again.

[0168] In other embodiments, the conditional threshold may also be, for example, the proportion of the sample labels in the first sample data to the corresponding sample labels in the second training sample (for example, the conditional threshold is any value between 5% and 10%). If the proportion of the number of first sample data corresponding to the same sample label to the number of the corresponding second training sample exceeds the conditional threshold, all training samples corresponding to the sample label may be input into the first intermediate model for retraining.

[0169] For example, in some embodiments of the present application, the first sample data includes an abnormal sound audio with a sample label of scratching of the rotating shaft mechanism. When retraining the first group of models, the training samples with the corresponding sample label of scratching of the rotating shaft mechanism in the first training samples and the second training samples can be input into the first group of models, so as to retrain the first intermediate model to obtain the first model.

[0170] It can be understood that in other embodiments, more groups of training samples can be divided into multiple groups to gradually train multiple groups of models, so that the model trained with the last group of training samples is used as the first model. In this way, the first model trained by the above process can adapt to more diverse training samples with the same sample label. Thus, the prediction result of the first model is more accurate.

[0171] Next, the detection results of the first model for abnormal sound audio of a foldable electronic device in an embodiment of the present application are introduced.

[0172] For example, FIG. 8A to FIG. 8B According to some embodiments of the present application, a comparison chart of the predicted value (represented by one color) and the true value (represented by another color) of the first model is shown.

[0173] For example, FIG. 8A to FIG. 8B The horizontal axis is the number of samples, and the vertical axis is the label. Fig. 8A There are 201 prediction samples in total, of which 30 samples have a label of 1 (for example, the corresponding fault type is the shaft mechanism scratching). 50 samples have a label of 2 (for example, the corresponding fault type is the synchronization mechanism collision). 30 samples have a label of 3, 50 samples have a label of 4, 1 sample has a label of 5, and 40 samples have a label of 6. It is understandable. Fig. 8A The predicted values ​​and true values ​​of the first model for 199 samples are all the same, and there are differences between the predicted values ​​and true values ​​of two samples. Fig. 8A , there are two samples with a label of 6, but the predicted label is 3. Therefore, the current prediction accuracy is 99.005%.

[0174] Reference Figure 8B , Figure 8BThe results of the first model's prediction of 23 samples are shown, of which 9 samples have a label of 1, 4 samples have a label of 2, 4 samples have a label of 3, 3 samples have a label of 4, and 3 samples have a label of 6. It can be understood that the predicted values ​​are all the same as the true values, and the prediction accuracy reaches 100%.

[0175] The following is a detailed introduction to the detection devices involved in some embodiments of the present invention.

[0176] further, Fig. 9 According to some embodiments of the present application, a schematic diagram of the structure of a detection device is shown. The detection device can be used to implement the detection methods provided in the above embodiments.

[0177] like Fig. 9 As shown, the detection device includes one or more processors 101, system memory 102, non-volatile memory (NVM) 103, communication interface 104, input / output device 105, and system control logic 106 for coupling the processor 101, system memory 102, non-volatile memory 103, communication interface 104 and input / output (I / O) device 105. Among them:

[0178] The processor 101 may include one or more processing units, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro-programmed control unit (MCU), an artificial intelligence (AI) processor or a programmable logic device (field programmable gate array, FPGA), a neural network processor (neural-network processing unit, NPU), etc. The processing module or processing circuit may include one or more single-core or multi-core processors. In some embodiments, the CPU can be used to optimize the neural network model to be run, and the NPU can be used to run the neural network model to be run. For example, in some embodiments of the present application, the NPU can run a first model to detect abnormal sound audio of a foldable electronic device based on the first model.

[0179] The system memory 102 is a volatile memory, such as a random-access memory (RAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), etc. The system memory is used to temporarily store data and / or instructions. For example, in some embodiments, the system memory 102 can be used to store the instructions of the first model provided in the aforementioned embodiments.

[0180] The non-volatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), a compact disc (CD), a digital versatile disc (DVD), a solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 103 may also be a removable storage medium, such as a secure digital (SD) memory card, etc. In other embodiments, the non-volatile memory 103 may be used to store instructions of the detection method provided in the aforementioned embodiments, etc.

[0181] In particular, the system memory 102 and the non-volatile memory 103 may respectively include: a temporary copy and a permanent copy of the instruction 107. The instruction 107 may include: when executed by at least one of the processors 101, enabling the detection device to implement the detection method provided in each embodiment of the present application.

[0182] The communication interface 104 may include a transceiver for providing a wired or wireless communication interface for the detection device, and then communicating with any other suitable device through one or more networks. In some embodiments, the communication interface 104 may be integrated into other components of the detection device, for example, the communication interface 104 may be integrated into the processor 101. In some embodiments, the detection device may communicate with other devices through the communication interface 104, for example, the detection device may obtain abnormal sound audio or abnormal sound audio features from other electronic devices through the communication interface 104.

[0183] The input / output (I / O) device 105 may be an input device such as a keyboard, a mouse, etc., and an output device such as a display, etc. A user may interact with the detection device through the input / output (I / O) device 105 .

[0184] System control logic 106 may include any suitable interface controller to detect other modules of the device and provide any suitable interface. For example, in some embodiments, system control logic 106 may include one or more memory controllers to provide interfaces connected to system memory 102 and non-volatile memory 103.

[0185] In some embodiments, at least one of the processors 101 may be packaged together with the logic of one or more controllers for the system control logic 106 to form a system in package (SiP). In other embodiments, at least one of the processors 101 may also be integrated with the logic of one or more controllers for the system control logic 106 on the same chip to form a system-on-chip (SoC).

[0186] Understandably, Fig. 9 The structure of the detection device shown is only an example. In other embodiments, the detection device may include more or fewer components than shown, or combine some components, or split some components, or arrange the components differently. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0187] It is understandable that the detection device can be any electronic device, including but not limited to mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, ultra-mobile personal computers (UMPC), netbooks, etc., and the embodiments of the present application are not limited thereto.

[0188] The embodiments of the present application also provide a program product, which, when executed on an electronic device, can enable the electronic device to implement the methods provided in the aforementioned embodiments.

[0189] An embodiment of the present application further provides a readable storage medium, in which one or more programs are stored. When the one or more programs are executed by an electronic device, the electronic device implements the methods provided in the aforementioned embodiments.

[0190] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0191] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor, a microcontroller, an application specific integrated circuit, or a microprocessor.

[0192] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0193] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including, but not limited to, floppy disks, optical disks, optical discs, compact disc-read only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random-access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information using the Internet in an electrical, optical, acoustic or other form of propagation signal (e.g., carrier wave, infrared signal, digital signal, etc.). Therefore, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0194] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0195] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0196] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0197] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. A method for detecting abnormal sound, characterized in that: include: Acquire first audio data corresponding to an abnormal sound audio generated during the folding process of a first foldable electronic device to be detected, and the first audio data is processed by increasing the energy of a high-frequency part of the abnormal sound audio; The first audio data is input into the first model to obtain the fault type of abnormal sound generated during the folding process of the first foldable electronic device, wherein: The first model is obtained by training based on first-category features of multiple historical abnormal sound audios obtained by folding multiple second foldable electronic devices multiple times during a historical folding process, and historical fault types corresponding to each of the historical abnormal sound audios, wherein the first-category features are linear predicted Mel cepstral coefficients of the historical abnormal sound audios or features corresponding to the linear predicted Mel cepstral coefficients of the historical abnormal sound audios, and both the first foldable electronic device and the second foldable electronic device are foldable mobile phones; The training process of the first model includes: Based on the first training sample, a first intermediate model is trained, wherein the first training sample includes a first part of historical audios among the plurality of historical abnormal sound audios, and the historical fault type corresponding to the first part of historical audios; Inputting sample data in a second training sample into the first intermediate model to obtain an inference result of the sample data in the second training sample, wherein the sample data in the second training sample includes a second part of historical abnormal sound audios in the plurality of historical abnormal sound audios except the first part of historical audios, and the sample label of the sample data of the second training sample is the historical fault type corresponding to the second part of historical abnormal sound audios; Determine, among the sample data in the second training sample, first sample data corresponding to the inference result and having a different corresponding sample label; The first intermediate model is trained based on a third training sample in the first training sample and the second training sample, whose corresponding sample label is the same as the sample label of the first sample data, to obtain the first model.

2. The method according to claim 1, characterized in that The first foldable electronic device and the second foldable electronic device have the same foldable structure.

3. The method according to claim 1, characterized in that The first audio data is the first type of feature.

4. The method according to claim 1, characterized in that The training of the first intermediate model based on a third training sample in the first training sample and the second training sample, whose corresponding sample label is the same as the sample label of the first sample data, comprises: When the amount of the first sample data is greater than a preset amount, or the ratio of the amount of the first sample data to the amount of sample data in the second training sample is greater than a preset ratio, the first intermediate model is trained based on the third training sample.

5. The method according to claim 1, characterized in that The training of the first intermediate model based on a third training sample in the first training sample and the second training sample, whose corresponding sample label is the same as the sample label of the first sample data, comprises: When the number of historical abnormal sound audios corresponding to the first sample label in the first sample data is greater than a preset number, or the ratio of the number of historical abnormal sound audios corresponding to the first sample label in the first sample data to the number of historical abnormal sound audios corresponding to the first sample label in the second training sample is greater than a preset ratio, the first intermediate model is trained based on the third training sample corresponding to the first sample label.

6. The method according to claim 1, characterized in that The fault types include: shaft mechanism scratching, synchronization mechanism collision, and transmission structure scratching.

7. The method according to claim 1, characterized in that The first audio data is data corresponding to audio with abnormal sound during the folding process of the first foldable electronic device.

8. An electronic device, characterized in that: include: A memory for storing instructions; At least one processor is used to execute the instructions so that the electronic device implements the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The readable storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is executed on a device, the device is caused to perform the method according to any one of claims 1 to 7.

Images