Processing method, electronic equipment and storage medium
By outputting and receiving audio signals in electronic devices, and using the characteristic parameters of the audio signal to determine the distance of the object, the problem of low gesture recognition accuracy in low light environments is solved, and higher recognition accuracy is achieved.
Patent Information
- Application Number
- CN202411546888.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-25
AI Technical Summary
In the case of dark ambient light, the accuracy of image recognition gestures collected by electronic devices through the camera is low.
By outputting the target audio signal and receiving the feedback audio signal of the object, the distance of the object is determined using the characteristic parameters of the audio signal, and gesture recognition is realized.
The accuracy of gesture recognition of electronic devices in low-light environments is improved, and the distance between electronic devices and objects can be accurately determined.
Smart Images
Figure CN119513568A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology, and in particular to a processing method, an electronic device and a storage medium. Background Art
[0002] With the widespread application of electronic devices (such as mobile phones, tablets, smart watches, etc.), in order to enrich the interaction between electronic devices and users, more and more electronic devices can interact with users through gesture recognition. The most important thing about gesture recognition is to accurately recognize the user's gestures. In related technologies, the user's image is collected by the camera of the electronic device, and the user's gesture is recognized by analyzing the image. However, in the case of dim ambient light, the accuracy of gesture recognition through the collected image is low. Summary of the invention
[0003] In view of this, the embodiments of the present application at least provide a processing method, an electronic device, and a storage medium.
[0004] The technical solution of the embodiment of the present application is implemented as follows:
[0005] In a first aspect, an embodiment of the present application provides a processing method, the method comprising:
[0006] Outputting a target audio signal, the target audio signal comprising a first partial signal and a second partial signal located after the first partial signal, wherein the first partial signal is used to identify the target audio signal, and the second partial signal is used to characterize a reference signal;
[0007] receiving a feedback audio signal resulting from the target audio signal encountering an object, wherein the feedback audio signal includes the second portion of the target audio signal and a third portion of the signal reflecting the encounter with the object;
[0008] Based on the first part of the target audio signal and the third part of the feedback audio signal, obtaining characteristic parameters of the target audio signal, wherein the characteristic parameters are used to characterize time domain response characteristics of the feedback audio signal encountering an object;
[0009] Based on the characteristic parameters, the distance to the object is obtained.
[0010] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:
[0011] a transmitting unit, configured to output a target audio signal, wherein the target audio signal includes a first partial signal and a second partial signal located after the first partial signal, wherein the first partial signal is used to identify the target audio signal, and the second partial signal is used to characterize a reference signal;
[0012] a receiving unit, configured to receive a feedback audio signal when the target audio signal encounters an object, wherein the feedback audio signal includes the second part of the target audio signal and a third part of the signal reflecting the encounter with the object;
[0013] A processing unit is used to obtain characteristic parameters of the target audio signal based on the first part of the target audio signal and the third part of the feedback audio signal, wherein the characteristic parameters are used to characterize the time domain response characteristics of the feedback audio signal encountering an object; and obtain the distance to the object based on the characteristic parameters.
[0014] In a third aspect, an embodiment of the present application provides a processing device, the processing device comprising:
[0015] A feature determination unit, configured to obtain a feature parameter of the target audio signal based on a first portion of the target audio signal and a third portion of the feedback audio signal, wherein the feature parameter is used to characterize a time domain response feature of the feedback audio signal encountering an object;
[0016] a distance determination unit, configured to obtain a distance from the object based on the characteristic parameter;
[0017] The target audio signal also includes a second part of the signal located after the first part of the signal, wherein the first part of the signal is used to identify the target audio signal and the second part of the signal is used to characterize the reference signal; the feedback audio signal is a signal generated when the target audio signal encounters an object; the feedback audio signal includes the second part of the signal of the target audio signal and a third part of the signal used to reflect the result of encountering the object.
[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements some or all of the steps in the above method when executed by a processor.
[0019] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which implement some or all of the steps in the above method when executed by a processor.
[0020] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.
[0022] Figure 1 A schematic diagram of an implementation process of a processing method provided in an embodiment of the present application Figure 1 ;
[0023] Figure 2a A second schematic diagram of an implementation flow of a processing method provided in an embodiment of the present application;
[0024] Figure 2b A schematic diagram of a scenario of a processing method provided in an embodiment of the present application;
[0025] Figure 3 A schematic diagram of an implementation process of a processing method provided in an embodiment of the present application Figure 3 ;
[0026] Figure 4 A schematic diagram of an implementation process of a processing method provided in an embodiment of the present application Figure 4 ;
[0027] Figure 5 A schematic diagram of an implementation process of a processing method provided in an embodiment of the present application Figure 5 ;
[0028] Figure 6 A hardware entity diagram of an electronic device in an embodiment of the present application Figure 1 ;
[0029] Figure 7 This is a second schematic diagram of a hardware entity of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0032] The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged with a specific order or sequence where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.
[0034] In order to solve the technical problem of low accuracy of gesture recognition in electronic devices in the related art, an embodiment of the present application provides a processing method, which can be applied to electronic devices. By way of example, the electronic devices include but are not limited to smart phones, tablet computers, wearable devices, personal computers (PCs), netbooks, etc., and the present application does not impose fixed restrictions on their implementation forms.
[0035] like Figure 1 As shown, the above processing method can be implemented through steps S101 to S104:
[0036] Step S101: output a target audio signal, the target audio signal comprising a first partial signal and a second partial signal following the first partial signal, wherein the first partial signal is used to identify the target audio signal, and the second partial signal is used to characterize a reference signal.
[0037] Here, the target audio signal may be a signal generated by a specific coding signal. The specific coding signal may be a signal with a strong autocorrelation. It is understandable that autocorrelation is a measure of the similarity of a signal itself at different time points, which can reveal repeated patterns in the signal, so that when a specific coding signal is offset due to encountering an object, the offset of the signal can be calculated based on the autocorrelation of the signal. In other words, because the target audio signal is generated based on the above-mentioned specific coding signal with a strong autocorrelation, when the electronic device outputs the target audio signal, the electronic device receives a received signal in which the target audio signal changes due to encountering an object, and can determine the offset of the target audio signal based on the received signal and the target audio signal.
[0038] In an embodiment of the present application, the target audio signal includes a first part of the signal for identifying the target audio signal, and a second part of the signal for characterizing the reference signal. The first part of the signal can be generated based on the specific coding signal, and has a strong autocorrelation and a lower side lobe. The second part of the signal can be a reference signal of the target audio signal, and the reference signal will not change when encountering an object, so that the reference signal can be used to determine the offset first part of the signal from the received signal received by the electronic device. In other words, the received signal received by the electronic device also includes the second part of the signal. Exemplarily, the second part of the signal can be a silent signal, that is, all 0.
[0039] In some embodiments, the electronic device includes a transmitting unit, through which the target audio signal is transmitted. Exemplarily, when the electronic device is a mobile phone or a tablet computer, the target audio signal can be transmitted through a speaker.
[0040] Step S102: receiving a feedback audio signal caused by a target audio signal encountering an object, wherein the feedback audio signal includes a second portion of the target audio signal and a third portion of the signal reflecting the encountering of the object.
[0041] Here, the feedback audio signal may be a reception signal received by the electronic device. When the electronic device receives the feedback audio signal, it indicates that there is an object around the electronic device, and the transmitted target audio signal is offset due to encountering the object, thereby obtaining the feedback audio signal.
[0042] It is understandable that the target audio signal emitted by the electronic device and the feedback audio signal are both propagated through the channel path in the air, and different channel paths have different effects on the signal. Therefore, when the target audio signal encounters an object, the channel path of the target audio signal changes. After the changed channel path affects the first part of the signal in the target audio signal, the third part of the signal in the feedback audio signal can be obtained, which is used to reflect the signal caused by encountering the object. Because the second part of the signal in the target audio signal is a reference signal and will not be affected by the channel path on the signal, the feedback audio signal also includes the second part of the signal.
[0043] Step S103: obtaining characteristic parameters of the target audio signal based on the first portion of the target audio signal and the third portion of the feedback audio signal, wherein the characteristic parameters are used to characterize the time domain response characteristics of the feedback audio signal encountering the object.
[0044] Here, the characteristic parameter of the target audio signal is used to characterize the time domain response characteristic of the feedback audio signal encountering the object. The time domain response characteristic can indicate the influence degree of the channel transmitting the second part of the signal on the second part of the signal. Exemplarily, the time domain response characteristic can be a channel impulse response (CIR) characteristic.
[0045] In some embodiments, the first part of the signal and the third part of the signal may be calculated using a least mean square algorithm (LMS algorithm) to obtain characteristic parameters of the target audio signal.
[0046] Step S104: obtaining the distance to the object based on the characteristic parameters.
[0047] In an embodiment of the present application, the real-time moving distance of the object at the current moment can be determined through characteristic parameters, and then the distance between the electronic device and the object at the previous moment and the real-time moving distance of the object at the current moment can be determined.
[0048] It is understandable that any signal has its own influence distance. When the object is outside the influence distance, the electronic device will not receive the above feedback audio signal, and the current state can be used as the initialization state. At a certain moment, the electronic device receives the feedback audio signal at that moment, and the real-time moving distance of the object at that moment can be calculated. Because the previous moment is the initialization state, that is, the object has not yet entered the influence distance of the target audio signal, the distance between the electronic device and the object at the previous moment can be determined as the above-mentioned influence distance. The distance between the electronic device and the object at the current moment is determined by the influence distance and the real-time moving distance of the object at the current moment. Afterwards, according to the above steps, the distance of the electronic device from the object at each moment is determined in real time.
[0049] In an embodiment of the present application, the electronic device first outputs a target audio signal including a first part of the signal for identifying the target audio signal and a second part of the signal for characterizing the reference signal. Then, when a feedback audio signal is received that the target audio signal encounters an object, the time domain response characteristics of the feedback audio signal for characterizing the object encountered can be obtained based on the first part of the signal of the target audio signal and the third part of the signal of the feedback audio signal. Finally, the distance to the object is obtained based on the feature parameter. In this way, the electronic device can accurately determine whether there is an object around the electronic device by whether the feedback audio signal is received, and in the case of an object, the distance between the electronic device and the object can be accurately determined by characterizing the time domain response characteristics of the feedback audio signal that encounters the object, thereby solving the problem of inaccurate gesture recognition due to the influence of ambient light in the related art.
[0050] In some embodiments, Figure 2a As shown, the above step S101 can be implemented through steps S201 and S202:
[0051] Step S201, determining the real-time moving distance of the object in the current time period based on the characteristic parameters of the target audio signal in the current time period.
[0052] Here, the current time period refers to a time period whose time length is less than or equal to a time length threshold. Exemplarily, the time length threshold may be 1 second. That is, in the embodiment of the present application, the real-time moving distance of the object in a smaller time period may be determined.
[0053] In some embodiments, the feedback audio signal can be segmented to obtain multiple feedback sub-audio signals, and then characteristic sub-parameters corresponding to the multiple feedback sub-audio signals are determined, and finally the real-time movement distance of the object in the current time period is determined based on the multiple characteristic sub-parameters.
[0054] In some embodiments, Figure 2a As shown, the above step S201 can be implemented through step S2011 and step S2022:
[0055] Step S2011, based on the characteristic parameters of the target audio signal in the current time period, determine the speed information of the object in the current time period; the speed information includes the speed direction and the speed value.
[0056] In the embodiment of the present application, the feedback audio signal can be first segmented to obtain multiple feedback sub-audio signals, and then the characteristic sub-parameters corresponding to the multiple feedback sub-audio signals are determined. The acceleration information of the object corresponding to each characteristic parameter is determined, and then the speed value and speed direction of the object in the current time period are determined based on the acceleration information corresponding to the multiple characteristic sub-parameters.
[0057] It is understandable that the acceleration of the object is proportional to the influence of the channel on the signal (ie, the characteristic parameter). The greater the acceleration of the object, the greater the influence of the channel on the signal. Therefore, the acceleration information of the body corresponding to each characteristic parameter can be determined.
[0058] For example, Figure 2b As shown, FIG2 shows the distribution of the corresponding characteristic parameters when the object is moving. The object in FIG2 is a finger, and the finger is in reciprocating motion. When the finger moves to the leftmost and rightmost ends, the acceleration of the fingertip is the largest, and the corresponding characteristic parameter is also the largest, such as the first pixel 201 in FIG2; while the acceleration of the base of the finger is the smallest, and the corresponding characteristic parameter is the smallest, such as the second pixel 202 in FIG2.
[0059] Step S2012, based on the speed direction of the object, the speed value of the object and the current time period, determine the real-time moving distance of the object in the current time period.
[0060] In the embodiment of the present application, the absolute value of the real-time moving distance of the object in the current time period can be determined based on the product of the speed value of the object and the duration corresponding to the current time period, and then the real-time moving distance of the object in the current time period can be determined based on the speed direction of the object. Wherein, if the speed direction of the object is moving toward the electronic device, the real-time moving distance of the object is a positive value, and if the speed direction of the object is moving away from the electronic device, the real-time moving distance of the object is a negative value.
[0061] In the embodiments of the present application, the speed information of the object in the current time period is determined through the characteristic parameters of the target audio signal in the current time period; then, based on the speed direction of the object, the speed value of the object, and the current time period, the real-time moving distance of the object in the current time period is determined. In this way, the real-time moving distance of the object can be determined through the characteristic parameters of the target audio signal, thereby improving the accuracy of locating the object.
[0062] Step S202: Determine the distance from the object based on the maximum influence distance of the target audio signal and the real-time moving distance of the object.
[0063] In the embodiments of the present application, the previous time period of the current time period is the time period corresponding to the initialization state of the electronic device. That is to say, in the previous time period, no object appeared within the maximum influence distance of the target audio signal, that is, the electronic device did not receive a feedback audio signal. Then, in the current time period, an object appears within the maximum influence distance of the target audio signal. At this time, the difference between the maximum influence distance and the real-time moving distance can be used as the distance between the electronic device and the object.
[0064] In some embodiments, when calculating the distance between the electronic device and the object in the next time period of the current time period, the difference between the distance between the electronic device and the object corresponding to the current time period and the real-time moving distance of the object corresponding to the next time period can be used as the distance between the electronic device and the object in the next time period. By analogy, the distance between the electronic device and the object in the current time period can be determined through the difference between the distance between the electronic device and the object in the previous time period and the real-time moving distance of the object in the current time period.
[0065] In some embodiments, the distance between the electronic device and the object can be implemented by formula (1):
[0066] l(t) = l(t - 1) - V * S, 0 < l(t) < L Formula (1);
[0067] Where l(t) is the distance between the electronic device and the object in the current time period; l(t - 1) is the distance between the electronic device and the object in the previous time period; V is the average speed of the object in the current time period; S is the duration of the current time period; L is the maximum influence distance of the target audio signal. When the electronic device was in the initialization state at the previous moment, l(t - 1) is equal to L.
[0068] In the embodiment of the present application, the time domain response characteristics generated by the first part of the signal used to identify the target audio signal when encountering an object can be used to accurately determine the real-time moving distance of the object in the current time period, and then the maximum impact distance of the target audio signal and the real-time moving distance of the object are used to determine the distance to the object. In this way, the distance between the electronic device and the object can be determined in real time, thereby improving the accuracy of locating the object.
[0069] In some embodiments, the maximum impact distance of the target audio signal may be achieved by the following steps:
[0070] Step a, controlling the object to move toward the electronic device at a fixed speed.
[0071] Step b: while the object is moving, the electronic device continuously sends a target audio signal to the object.
[0072] Step c: the electronic device obtains the received signal, and when the received signal is different from the target audio signal, determines the current distance between the object and the electronic device as the maximum impact distance of the target audio signal.
[0073] Exemplarily, the maximum influence distance may be 20 to 30 cm.
[0074] In some embodiments, the above step S2011 may be implemented by step S2011a and step S2011b:
[0075] Step S2011a, using the trained speed recognition model to perform direction recognition processing on the feature parameters to obtain the speed direction of the object in the current time period.
[0076] Step S2011b, using the trained speed recognition model to perform regression prediction processing on the feature parameters to obtain the speed value of the object in the current time period.
[0077] Among them, the trained speed recognition model is trained based on the sample characteristic parameters of the target audio signal, the speed direction of the sample object and the sample speed information of the sample object; the sample characteristic parameters are used to characterize the time domain response characteristics of the sample feedback signal; the sample feedback signal is the signal generated when the target audio signal encounters a sample object moving with the sample speed information.
[0078] Here, the speed recognition model may be a multimodal model. Exemplarily, the multimodal model may be a convolutional recurrent neural network (CRNN) model.
[0079] In the embodiment of the present application, the input of the trained speed recognition model is the characteristic parameters in the current time period, and the output of the trained speed recognition model has two, one is the speed direction of the object in the current time period, and the other is the speed value of the object in the current time period. The speed value can be the average speed of the object in the current time period.
[0080] It can be understood that, because the characteristic parameters of the target audio signal have a corresponding relationship with the moving direction of the object, the speed recognition model obtained by training the sample characteristic parameters of the target audio signal and the speed direction of the sample object can recognize the speed direction of the object in the current time period. Figure 2b As shown, the directions of the reciprocating motion of the finger are approaching the electronic device and moving away from the electronic device. The first straight line is a straight line fitted based on the characteristic parameters of various parts of the finger during the movement. The slope of the straight line is a negative slope, which represents that the finger is approaching the electronic device. The second straight line is a straight line fitted based on the characteristic parameters of various parts of the finger during the movement. The slope of the straight line is a positive slope, which represents that the finger is moving away from the electronic device.
[0081] In an embodiment of the present application, when performing regression prediction processing, the trained speed recognition model can first determine the acceleration value of each characteristic parameter in the current time period, and then determine the average speed of the object in the current time period based on multiple acceleration values in the current time period.
[0082] In an embodiment of the present application, when constructing a sample set, the sample object can be controlled to move at different moving speeds and different moving directions within the maximum influence distance of the electronic device, and then the electronic device can be controlled to send a target audio signal, and then the time domain response features corresponding to different moving speeds and different moving directions can be calculated. In this way, multiple time domain response features and two label information of the time domain response features, namely, the speed direction of the sample object and the sample speed information of the sample object, can be obtained. Then, the speed recognition model is trained by a sample set including multiple time domain response features and two label information of each time domain response feature, and the trained speed recognition model is obtained when the difference between the output of the speed recognition model and the label information is within a preset range, or the number of training iterations of the speed recognition model is greater than or equal to the preset number. Among them, the speed direction of the sample object refers to approaching the electronic device or moving away from the electronic device. Therefore, the direction recognition processing of the feature parameters by the trained speed recognition model belongs to classification processing.
[0083] In the embodiment of the present application, the trained speed recognition model is used to perform direction recognition processing on the characteristic parameters in the current time period to obtain the speed direction of the object in the current time period, and the trained speed recognition model is used to perform regression prediction processing on the characteristic parameters in the current time period to obtain the speed value of the object in the current time period. In this way, the speed recognition model can be trained through the correspondence between the movement information of the object during the movement process (i.e., the speed direction and speed value) and the time domain response characteristics generated by the target audio signal when encountering the moving object, so that the movement information of the object can be obtained through the trained speed recognition model, and then the object can be located more accurately.
[0084] In some embodiments, the characteristic parameter of the target audio signal in the current time period includes a plurality of characteristic sub-parameters; the above step S2011b is implemented by the following steps:
[0085] Step 1: For each characteristic sub-parameter, acceleration information is obtained based on the trained speed recognition model and the characteristic sub-parameter.
[0086] Here, the characteristic sub-parameter may be a characteristic sub-parameter corresponding to the current sub-time period in the current time period. In an embodiment of the present application, the feedback audio signal received by the electronic device in the current time period may be segmented to obtain a plurality of feedback audio sub-signals, and then the characteristic sub-parameter between each feedback audio sub-signal and the target audio signal may be calculated.
[0087] In the embodiment of the present application, each characteristic sub-parameter may be input into a trained speed recognition model to obtain acceleration information corresponding to the characteristic sub-parameter.
[0088] It is understandable that the time domain response characteristics of the target audio signal generated when encountering a moving object correspond to the acceleration information of the object. Figure 2b As shown in FIG. 1 , the time domain response feature corresponding to the maximum acceleration of the finger tip during movement is also the largest. Therefore, the velocity recognition model can be trained through the correspondence between the time domain response feature and the acceleration information of the object, thereby obtaining the acceleration information corresponding to the characteristic sub-parameter.
[0089] In some embodiments, in order to improve the accuracy of determining the acceleration information of an object, multiple characteristic sub-parameters can be first differentially processed, and then the multiple characteristic sub-parameters after differential processing can be input into the trained speed recognition model to obtain the acceleration information corresponding to each characteristic sub-parameter after differential processing.
[0090] Step 2: Determine the velocity value of the object in the current time period based on the acceleration information corresponding to the multiple characteristic sub-parameters.
[0091] In the embodiment of the present application, the acceleration information corresponding to the multiple characteristic sub-parameters can be input into the trained speed recognition model to obtain the average speed of the object in the current time period.
[0092] In some embodiments, multiple acceleration information may be integrated to obtain the velocity value of the object in the current time period.
[0093] In the embodiment of the present application, the acceleration information corresponding to each characteristic sub-parameter can be determined by the trained speed recognition model. Then, the speed value of the object in the current time period is determined by the acceleration information corresponding to the multiple characteristic sub-parameters. In this way, the speed recognition model can be trained through the correspondence between the acceleration information of the object during movement and the time domain response characteristics generated by the target audio signal when encountering the moving object, so that the multiple acceleration information of the object in the current time period can be obtained through the trained speed recognition model, and then the speed value of the object in the current time period can be determined through the multiple acceleration information of the object.
[0094] In some embodiments, the above-mentioned characteristic parameters include a channel response vector; Figure 3 As shown, the above step S103 can be implemented through steps S301 to S303:
[0095] Step S301, determining the independent variable sequence in the first part of the signal.
[0096] Here, the independent variable sequence may be a signal in the target audio signal that changes due to encountering an object. Therefore, the independent variable sequence may be the entire first part signal or a part of the first part signal.
[0097] In some embodiments, if the target audio signal is generated based on a sequence with autocorrelation, the sequence with autocorrelation can be used as an independent variable sequence. Exemplarily, the sequence with autocorrelation can be a Barker code sequence. In some embodiments, because the length of the current Barker code sequence is up to 13 points, when generating the target audio signal based on the Barker code sequence, it is necessary to first perform interpolation processing on the Barker code sequence, so the above independent variable sequence can be an interpolated Barker code sequence.
[0098] In some embodiments, in order to increase the speed of calculating the channel response vector, part of the first part of the signal may be determined as an independent variable sequence. For example, the sequence of the first 140 points in the first part of the signal may be used as the independent variable sequence.
[0099] Step S302: Determine the dependent variable sequence in the third part of the signal based on the bit number information of the independent variable sequence and the bit number information of the first part of the signal.
[0100] In the embodiment of the present application, the electronic device can continuously send the target audio signal, and after receiving the feedback audio signal in the current time period, the feedback audio signal with a time length of the current time period can be segmented according to the preset time length to obtain at least two feedback audio sub-signals, that is, the time length of each feedback audio sub-signal is the preset time length. Among them, because the time lengths of the first part signal and the second part signal in the target audio signal are both less than the preset time length, the time lengths of the second part signal and the third part signal in the feedback audio signal when the target audio signal encounters the object are also less than the preset time length. Therefore, each feedback audio sub-signal obtained by segmentation includes at least part of the second part signal and at least part of the third part signal.
[0101] In an embodiment of the present application, the bit information of the dependent variable sequence can be determined based on the bit information of the independent variable sequence and the bit information of the first part of the signal, and then the dependent variable sequence can be determined based on the bit information of the dependent variable sequence. Among them, the bit information of the independent variable sequence and the difference of the bit information of the first part of the signal can be determined as the bit information of the dependent variable sequence. Exemplarily, when the length of the independent variable sequence is 140 points and the bit information of the first part of the signal is 312 points, 172 points can be used as the bit information of the dependent variable sequence. Because the target audio signal will be offset when encountering an object, the last 172 points of the feedback audio sub-signal can be used as the dependent variable sequence.
[0102] Step S303: determining a channel response vector of the target audio signal based on the independent variable sequence and the dependent variable sequence.
[0103] It can be understood that when the transmitted signal propagates in the air, the channel in the air can affect the transmitted signal, thereby changing the signal and obtaining the received signal, and the change between the received signal and the transmitted signal is a linear change. Therefore, the channel in the air can be regarded as a linear system, and the linear change coefficient of the linear system can be solved based on the independent variable sequence and the dependent variable sequence, that is, the influence of the channel in the air on the signal can be obtained.
[0104] In the embodiment of the present application, the dependent variable sequence in the feedback audio signal is determined by the bit information of the independent variable sequence and the bit information of the first part of the signal. In this way, the dependent variable sequence obtained by the offset characteristic of the signal when encountering an object in the feedback audio signal can be accurately determined.
[0105] In some embodiments, Figure 3 As shown, the above step S303 can be implemented through step S3031 and step S3032:
[0106] Step S3031: construct a channel matrix based on the independent variable sequence and the dependent variable sequence.
[0107] Exemplarily, when the sequence of the first 140 points in the first part of the signal is used as the independent variable sequence and the signal of the last 172 points in the audio sub-signal is fed back as the dependent variable sequence, a 140*172 channel matrix can be constructed.
[0108] Step S3032: determine a channel response coefficient based on the channel matrix, and determine the channel response coefficient as a channel response vector of the target audio signal.
[0109] The channel response coefficient is used to characterize the influence of the channel on the independent variable sequence during the transmission of the independent variable sequence in the channel.
[0110] In the embodiment of the present application, the channel response coefficients of the channel matrix may be calculated by a least mean square (LMS) algorithm, that is, a channel response vector of each feedback audio sub-signal may be obtained.
[0111] In the embodiment of the present application, by constructing a channel matrix including an independent variable sequence and a dependent variable sequence, the degree of influence of the channel in the air on the signal when transmitting the signal can be accurately calculated. Therefore, the movement information of the object can be predicted through the degree of influence of the channel on the signal, thereby improving the accuracy of locating the object.
[0112] In some embodiments, Figure 4 As shown, the above processing method can also be implemented through steps S401 to S404:
[0113] Step S401, obtaining a specific coded signal.
[0114] Here, the specific coded signal refers to a signal with lower side lobes and higher autocorrelation. Exemplarily, the specific coded signal may be a Bark code.
[0115] It can be understood that, because the correlation coefficient of the specific coding signal has a very low side lobe, when the target audio signal sent by the electronic device is offset due to encountering an object, the offset can be determined through the received feedback audio signal.
[0116] Step S402: perform copy processing on the specific coded signal, and perform difference processing on the copied specific coded signal to obtain an identification signal.
[0117] Here, the identification signal corresponds to the first portion of the target audio signal.
[0118] It is understandable that when the specific coded signal is a Barker code, since the maximum length of the Barker code is 13 points, directly using the specific coded signal as the target audio signal will reduce the accuracy of determining the signal deviation.
[0119] Exemplarily, when the specific coded signal is a 13-point Barker code, the 13-point Barker code may be replicated twice to obtain a 26-point signal, and then the 26-point signal is subjected to difference processing to obtain a 312-point identification signal.
[0120] Step S403: synthesize the identification signal and the mute signal to obtain a specific sequence.
[0121] Here, the mute signal corresponds to the second part of the target audio signal. Each point signal in the mute signal is 0. In this way, the mute signal will not be affected by the channel during the channel transmission process. Therefore, based on the mute signal, the third part of the signal caused by the object can be determined from the target audio signal.
[0122] Step S404: Load the specific sequence onto the carrier signal to obtain a target audio signal.
[0123] Here, after the identification signal in the specific sequence is loaded onto the carrier signal, the first part of the signal is obtained; after the silence signal in the specific sequence is loaded onto the carrier signal, the second part of the signal is obtained.
[0124] In the embodiment of the present application, the specific coded signal is firstly copied, and the specific coded signal after the copying process is subjected to difference processing to obtain an identification signal; then the identification signal and the mute signal are synthesized to obtain a specific sequence; finally, the specific sequence is loaded onto the carrier signal to obtain a target audio signal. In this way, by synthesizing the identification signal and the mute signal to obtain a specific sequence, the accuracy of determining the third part signal in the feedback audio signal can be improved.
[0125] In some embodiments, Figure 5 As shown, the above method can also be implemented by the following steps:
[0126] Step S501, modulating and generating a sequence.
[0127] The above step S501 can be implemented by the following steps:
[0128] Step S5011, copy the 13-point Barker code signal twice, and perform interpolation processing on the copied Barker code signal to obtain an interpolated Barker code signal.
[0129] Here, the copied Barker code signal may be interpolated 11 times to obtain a 312-point interpolated Barker code signal.
[0130] Step S5012: Add a silence signal to the end of the interpolated Barker code signal to obtain a processed Barker code signal.
[0131] Here, the mute signal is a 168-point signal, and each point is 0. The processed Barker code signal finally obtained is a 480-point signal.
[0132] Step S5012: multiply the processed Barker code signal by the high frequency signal to obtain a generated sequence.
[0133] Step S502: measuring the maximum influence distance of the generated sequence.
[0134] The above step S2 can be implemented by the following steps:
[0135] Step S5021: continuously transmit the generated sequence to the moving object through the speaker of the electronic device in the form of a carrier wave.
[0136] The moving object moves toward the electronic device at a slower speed.
[0137] Step S5022, receiving the sent generated sequence through the microphone of the electronic device.
[0138] Step S5023: Based on the sent generated sequence, determine whether the received generated sequence is interfered.
[0139] Step S5024: when the received generated sequence is interfered with, the current distance between the mobile object and the electronic device is determined as the maximum impact distance.
[0140] Step S503: continuously transmit the generated sequence in a carrier wave manner through the speaker of the electronic device.
[0141] Step S504: receiving the sent generated sequence through a microphone of the electronic device to obtain a received signal.
[0142] Step S505: perform high-frequency bandpass filtering on the received signal to obtain a filtered signal.
[0143] Step S506, using the least mean square algorithm to calculate the CIR vector for the filtered signal.
[0144] The above step S506 is implemented by the following steps:
[0145] Step S5061, segmenting the filtered signal to obtain multiple subsequences.
[0146] Here, the filtered signal may be divided into a plurality of subsequences according to a time length of 10 ms.
[0147] Step S5062: For each subsequence, construct a linear matrix based on the first 140 - point signals of the generated sequence and the last 172 - point signals of the subsequence.
[0148] Step S5063: Use the least - mean - square algorithm to solve each subsequence's linear matrix, obtain the independent variable coefficients of each linear matrix, and determine the CIR vector of each subsequence as the independent variable coefficients of each linear matrix.
[0149] Step S507: Input multiple CIR vectors within a preset time period into a multi - modal model to determine the moving speed and moving direction of the target object within this preset time period.
[0150] Among them, the above - mentioned Step S507 is implemented through the following steps:
[0151] Step S5071: Perform differential processing on multiple CIR vectors within a preset time period to obtain multiple differentially - processed CIR vectors.
[0152] Step S5072: Input the multiple differentially - processed CIR vectors into a multi - modal model to determine the moving speed and moving direction of the target object within the preset time period.
[0153] Here, the multi - modal model has two outputs. One is the classification output used to determine whether the object is approaching or moving away, and the other regression prediction output represents the moving speed of the object.
[0154] Step S508: Determine the distance between the target object and the electronic device according to the maximum influence distance of the generated sequence, the moving speed of the target object within the preset time period, and the moving direction.
[0155] Here, the distance between the above - mentioned target object and the electronic device can be achieved through formula (2):
[0156] l(t)=l(t - 1)-V*S, 0 < l(t) < L Formula (2);
[0157] Among them, when the electronic device is initialized for the first time (indicating that there is no object within the farthest ultrasonic influence distance), l(t - 1)=L; L is the farthest influence distance, V represents the average speed of the object recognized by the multi - modal model, and S represents the time interval. Therefore, as the multi - modal model output progresses, l(t) is continuously updated to the current distance.
[0158] Figure 6 This is a schematic diagram of the composition structure of an electronic device provided by an embodiment of the present application. As Figure 6 shown, the electronic device 600 includes: a transmitting unit 601, a receiving unit 602, and a processing unit 603, where:
[0159] The transmitting unit 601 is used to output a target audio signal, where the target audio signal includes a first portion of the signal and a second portion of the signal located after the first portion of the signal, wherein the first portion of the signal is used to identify the target audio signal, and the second portion of the signal is used to characterize the reference signal;
[0160] A receiving unit 602 is used to receive a feedback audio signal when a target audio signal encounters an object, where the feedback audio signal includes a second part of the target audio signal and a third part of the signal reflecting the encounter with the object;
[0161] The processing unit 603 is used to obtain characteristic parameters of the target audio signal based on the first part of the target audio signal and the third part of the feedback audio signal, where the characteristic parameters are used to characterize the time domain response characteristics of the feedback audio signal encountering the object; and obtain the distance to the object based on the characteristic parameters.
[0162] In some embodiments, the processing unit 603 is further used to determine the real-time moving distance of the object in the current time period based on the characteristic parameters of the target audio signal in the current time period; and determine the distance from the object based on the maximum impact distance of the target audio signal and the real-time moving distance of the object.
[0163] In some embodiments, the processing unit 603 is also used to determine the speed information of the object in the current time period based on the characteristic parameters of the target audio signal in the current time period; the speed information includes the speed direction and the speed value; based on the speed direction of the object, the speed value of the object and the current time period, determine the real-time moving distance of the object in the current time period.
[0164] In some embodiments, the processing unit 603 is further used to use the trained speed recognition model to perform direction recognition processing on the feature parameters to obtain the speed direction of the object in the current time period; use the trained speed recognition model to perform regression prediction processing on the feature parameters to obtain the speed value of the object in the current time period; wherein the trained speed recognition model is trained based on the sample feature parameters of the target audio signal, the speed direction of the sample object and the sample speed information of the sample object; the sample feature parameters are used to characterize the time domain response characteristics of the sample feedback signal; the sample feedback signal is a signal generated when the target audio signal encounters a sample object moving with the sample speed information.
[0165] In some embodiments, the processing unit 603 is further used to obtain acceleration information for each characteristic sub-parameter based on the trained speed recognition model and the characteristic sub-parameter; and determine the speed value of the object in the current time period based on the acceleration information corresponding to multiple characteristic sub-parameters.
[0166] In some embodiments, the processing unit 603 is also used to determine the independent variable sequence in the first part of the signal; determine the dependent variable sequence in the third part of the signal based on the bit number information of the independent variable sequence and the bit number information of the first part of the signal; and determine the channel response vector of the target audio signal based on the independent variable sequence and the dependent variable sequence.
[0167] In some embodiments, the processing unit 603 is further used to construct a channel matrix based on the independent variable sequence and the dependent variable sequence; determine the channel response coefficient based on the channel matrix, and determine the channel response coefficient as the channel response vector of the target audio signal; wherein the channel response coefficient is used to characterize the degree of influence of the channel on the independent variable sequence during the transmission of the independent variable sequence in the channel.
[0168] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.
[0169] It should be noted that in the embodiment of the present application, if the above-mentioned data processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.
[0170] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.
[0171] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.
[0172] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.
[0173] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.
[0174] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.
[0175] Figure 7 is a hardware entity schematic diagram of an electronic device in an embodiment of the present application, such as Figure 7 As shown, the hardware entity of the electronic device 700 includes: a processor 701, a communication interface 702 and a memory 703, wherein:
[0176] The processor 701 generally controls the overall operation of the electronic device 700 , and the overall operation may be to implement the processing method provided in the embodiment of the present application.
[0177] The communication interface 702 enables the computer device to communicate with other terminals or servers through a network.
[0178] The memory 703 is configured to store instructions and applications executable by the processor 701, and can also cache data to be processed or processed by the processor 701 and each module in the electronic device 700 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data transmission can be performed between the processor 701, the communication interface 702, and the memory 703 through the bus 704.
[0179] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the processing method of any of the above embodiments.
[0180] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.
[0181] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.
[0182] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.
[0183] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.
[0184] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0185] The above are only implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.
Claims
1. A processing method comprising: Outputting a target audio signal, the target audio signal comprising a first partial signal and a second partial signal located after the first partial signal, wherein the first partial signal is used to identify the target audio signal, and the second partial signal is used to characterize a reference signal; receiving a feedback audio signal resulting from the target audio signal encountering an object, wherein the feedback audio signal includes the second portion of the target audio signal and a third portion of the signal reflecting the encounter with the object; Based on the first part of the target audio signal and the third part of the feedback audio signal, obtaining characteristic parameters of the target audio signal, wherein the characteristic parameters are used to characterize time domain response characteristics of the feedback audio signal encountering an object; Based on the characteristic parameters, the distance to the object is obtained.
2. The method according to claim 1, wherein obtaining the distance to the object based on the characteristic parameter comprises: Determine the real-time moving distance of the object in the current time period based on the characteristic parameters of the target audio signal in the current time period; The distance to the object is determined based on the maximum impact distance of the target audio signal and the real-time moving distance of the object.
3. The method according to claim 2, wherein determining the real-time moving distance of the object in the current time period based on the characteristic parameters of the target audio signal in the current time period comprises: Determining speed information of the object in the current time period based on the characteristic parameters of the target audio signal in the current time period; The speed information includes speed direction and speed value; Based on the speed direction of the object, the speed value of the object and the current time period, a real-time moving distance of the object in the current time period is determined.
4. The method according to claim 3, wherein determining the speed information of the object in the current time period based on the characteristic parameters of the target audio signal in the current time period comprises: Using the trained speed recognition model to perform direction recognition processing on the characteristic parameters to obtain the speed direction of the object in the current time period; Using the trained speed recognition model to perform regression prediction processing on the characteristic parameters to obtain the speed value of the object in the current time period; The trained speed recognition model is obtained based on the sample characteristic parameters of the target audio signal, the speed direction of the sample object and the sample speed information of the sample object; the sample characteristic parameters are used to characterize the time domain response characteristics of the sample feedback signal; The sample feedback signal is a signal generated when the target audio signal encounters a sample object moving at the sample velocity information.
5. According to the method of claim 4, the characteristic parameter of the target audio signal in the current time period includes a plurality of characteristic sub-parameters; and the step of using the trained speed recognition model to perform regression prediction processing on the characteristic parameter to obtain the speed value of the object in the current time period comprises: For each of the characteristic sub-parameters, based on the trained speed recognition model and the characteristic sub-parameters, obtaining acceleration information; Based on the acceleration information respectively corresponding to the plurality of characteristic sub-parameters, a velocity value of the object in the current time period is determined.
6. The method according to any one of claims 1 to 5, wherein the characteristic parameter comprises a channel response vector; and obtaining the characteristic parameter of the target audio signal based on the first part of the target audio signal and the third part of the feedback audio signal comprises: determining a sequence of independent variables in the first portion of the signal; Determine the dependent variable sequence in the third part of the signal based on the bit information of the independent variable sequence and the bit information of the first part of the signal; A channel response vector of a target audio signal is determined based on the independent variable sequence and the dependent variable sequence.
7. The method according to claim 6, wherein determining the channel response vector of the target audio signal based on the independent variable sequence and the dependent variable sequence comprises: constructing a channel matrix based on the independent variable sequence and the dependent variable sequence; Determine a channel response coefficient based on the channel matrix, and determine the channel response coefficient as a channel response vector of the target audio signal; The channel response coefficient is used to characterize the influence of the channel on the independent variable sequence during the transmission of the independent variable sequence in the channel.
8. The method according to any one of claims 1 to 5, further comprising: Obtaining a specific coded signal; Performing a copy process on the specific coded signal, and performing a difference process on the copied specific coded signal to obtain an identification signal; Performing synthesis processing on the identification signal and the mute signal to obtain a specific sequence; The specific sequence is loaded onto a carrier signal to obtain the target audio signal.
9. An electronic device, comprising: a transmitting unit, configured to output a target audio signal, wherein the target audio signal includes a first partial signal and a second partial signal located after the first partial signal, wherein the first partial signal is used to identify the target audio signal, and the second partial signal is used to characterize a reference signal; a receiving unit, configured to receive a feedback audio signal when the target audio signal encounters an object, wherein the feedback audio signal includes the second part of the target audio signal and a third part of the signal reflecting the encounter with the object; A processing unit is used to obtain characteristic parameters of the target audio signal based on the first part of the target audio signal and the third part of the feedback audio signal, wherein the characteristic parameters are used to characterize the time domain response characteristics of the feedback audio signal encountering an object; and obtain the distance to the object based on the characteristic parameters.
10. A processing device, comprising: A feature determination unit, configured to obtain a feature parameter of the target audio signal based on a first portion of the target audio signal and a third portion of the feedback audio signal, wherein the feature parameter is used to characterize a time domain response feature of the feedback audio signal encountering an object; a distance determination unit, configured to obtain a distance from the object based on the characteristic parameter; Among them, the target audio signal also includes a second part of the signal located after the first part of the signal; the first part of the signal is used to identify the target audio signal, and the second part of the signal is used to characterize the reference signal; the feedback audio signal is a signal generated when the target audio signal encounters an object; the feedback audio signal includes the second part of the signal of the target audio signal and a third part of the signal used to reflect the encounter with the object.