An acoustic wave projection search collection control method and device, electronic equipment and storage medium

Abstract

The application discloses a sound wave projection search collection control method and device, electronic equipment and a storage medium. The method obtains service state information of a terminal device, determines an audio collection source and creates an audio collection instance when each service state meets the allowed collection condition, collects sound wave signals to form audio sampling data and sends the audio sampling data to a decoder, stops and releases the current audio collection instance, switches the audio collection source and creates a new audio collection instance when the start grace period, the continuous duration without obtaining the valid payload and the minimum switching interval all meet the conditions according to the diagnostic feedback information output by the decoder, and stops audio collection and releases the recording resource in response to a projection connection confirmation instruction. The application can improve the device adaptability and collection stability of sound wave search, and reduce unnecessary recording resource occupation.

Description

Technical Field

[0001] This application relates to the field of audio signal acquisition and device discovery technology for terminal devices, and in particular to an acquisition control method, device, electronic device, and computer-readable storage medium for sound wave projection search. Background Technology

[0002] Sound wave projection search is a device search method that uses sound wave signals to discover projection devices. During the sound wave projection search process, the terminal device can call its audio acquisition interface to collect sound wave signals sent by the device to be discovered through audio input devices such as microphones, and then transmit the collected audio sample data to a decoder. The decoder parses the audio sample data to obtain device information used to identify the device to be discovered.

[0003] In one existing implementation, audio capture begins as soon as the terminal device enters the screen mirroring search page. Because the timing of audio capture initiation is not clearly correlated with user actions, page status, device connection status, or screen mirroring service status, situations may arise where the audio capture instance still occupies recording resources even when the user has not actively initiated sound wave search, the page is no longer visible, or the screen mirroring connection process has already begun.

[0004] Furthermore, different terminal devices may have different device models, system versions, and audio processing frameworks, and different audio acquisition sources may also perform different preprocessing on the sound wave signals. For example, some audio acquisition sources may perform filtering, gain control, noise suppression, or other audio processing on the sound wave signals. When using a fixed audio acquisition source, a payload may be continuously unavailable on some terminal devices.

[0005] If the current audio acquisition source cannot obtain a valid payload, switching the audio acquisition source immediately or frequently may cause the audio acquisition instance to repeatedly stop and rebuild, affecting the stability of the audio acquisition process. If the current audio acquisition source is maintained indefinitely, the device to be projected may not be detected for an extended period. Therefore, it is necessary to determine the appropriate timing for switching the audio acquisition source.

[0006] Furthermore, the lifecycle of the audio capture instance may not be consistent with the screen mirroring connection process. Even after the user has confirmed the connection to the target device or the screen mirroring connection process has been initiated, the audio capture instance and the corresponding decoding processing may continue to run, thereby continuously consuming recording and processor resources and potentially affecting the state management of the candidate device list.

[0007] Therefore, it is necessary to coordinate and control user authorization, business status gating, audio acquisition source selection, audio acquisition instance creation, decoding diagnosis, dynamic source switching, and resource release after screen projection connection in the sound wave projection search. Summary of the Invention

[0008] This application provides a method, apparatus, electronic device, and computer-readable storage medium for acquisition control of acoustic wave projection search, which solves technical problems such as terminal devices starting audio acquisition when acoustic wave search service conditions are not met, fixed audio acquisition sources having difficulty obtaining effective payloads on some terminal devices, audio acquisition source switching being too early or too frequent, and audio acquisition instances continuing to occupy recording resources after the projection connection is confirmed.

[0009] To achieve the above objectives, this application adopts the following technical solution:

[0010] Firstly, a method for acquiring and controlling sound wave projection search is provided, applied to terminal devices. This method includes:

[0011] Obtain the service status information of the terminal device, and determine whether to allow audio capture to start based on the service status information;

[0012] When the business status information meets the corresponding allowed collection conditions, the first audio collection source is determined from the audio collection source set, and the current audio collection instance is created based on the first audio collection source;

[0013] Acquire sound wave signals through the current audio acquisition instance, obtain audio sampling data, and send the audio sampling data to the decoder;

[0014] Receive decoding diagnostic feedback information output by the decoder for the audio sample data;

[0015] When the start-up grace period of the current audio acquisition instance expires, the duration of the failure to acquire a valid payload reaches the preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval, the current audio acquisition instance is stopped and released. A second audio acquisition source different from the first audio acquisition source is determined from the audio acquisition source set, and a new audio acquisition instance is created based on the second audio acquisition source to continue acquiring sound wave signals.

[0016] In response to the screen mirroring connection confirmation command, stop the currently running audio acquisition instance on the terminal device and release the recording resources occupied by the audio acquisition instance.

[0017] Creating an audio capture instance is prohibited if the business status information does not meet the corresponding allowed capture conditions.

[0018] Optionally, before obtaining service status information, the terminal device detects the user's startup operation for the sound wave search entry; in response to the startup operation, it outputs sound wave search function description information; in response to the user's confirmation operation for the sound wave search function description information, it calls the terminal device's permission management interface to request recording permission; after the recording permission is granted, it obtains service status information.

[0019] Optionally, when recording permission is denied, the terminal device sets the sound wave search function status flag to the off state, clears the candidate device list, and records the recording permission denial status. Unless a new user activation operation is detected targeting the sound wave search entry, the terminal device will not again call the permission management interface to request recording permission.

[0020] Optionally, the business status information includes at least two of the following: the on / off status of the sound wave search function, the status of consent to the privacy agreement, the status of page visibility, the status of device connection, the status of screen mirroring session, the status of automatic connection, and the status of connection confirmation interface display.

[0021] The terminal device determines whether each service status meets the corresponding allowed data collection conditions. If all service statuses meet the allowed data collection conditions, audio data collection is allowed; if any service status does not meet the allowed data collection conditions, audio data collection is prohibited.

[0022] Optionally, the terminal device checks the following in sequence according to a preset check priority: the status of the sound wave search function switch, the status of privacy agreement agreement, the status of page visibility, the status of device connection, the status of screen projection session, the status of automatic connection, and the status of connection confirmation interface display. If any service status does not meet the corresponding allowed collection conditions, the check of the remaining service status is terminated, and it is determined that audio collection is prohibited.

[0023] Optionally, the terminal device creates the current audio acquisition instance using a first sampling rate, a mono pulse code modulation format, and a first audio acquisition source. If creating the current audio acquisition instance using the first sampling rate fails, a second sampling rate lower than the first sampling rate and a first audio acquisition source are used to create the current audio acquisition instance.

[0024] In conjunction with the first aspect above, in one possible implementation, in one embodiment, the first sampling rate is 48 kHz and the second sampling rate is 44.1 kHz.

[0025] Optionally, the decoded diagnostic feedback information includes the diagnostic event type and the time the diagnostic event occurred. The diagnostic event type includes at least one of frame header check rejection event and search miss event.

[0026] The terminal device counts the diagnostic events corresponding to the decoding diagnostic feedback information and records the first timestamp of the first diagnostic event during the current audio acquisition instance's operation. When the decoder outputs the payload, the terminal device resets the count value and the first timestamp of the diagnostic events.

[0027] Optionally, the terminal device records the start time of the current audio acquisition instance, determines the runtime of the current audio acquisition instance based on the current time and the start time, determines the duration of the period without a valid payload based on the current time and the first timestamp, and determines the time elapsed since the last audio acquisition source switch based on the current time and the last audio acquisition source switch time.

[0028] When the runtime reaches the startup grace period, the duration of the period without acquiring a valid payload reaches the preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval, the terminal device stops and releases the current audio acquisition instance.

[0029] Optionally, the audio acquisition source set includes, in sequence, a microphone audio acquisition source, a speech recognition audio acquisition source, and an unprocessed audio acquisition source. The terminal device determines the audio acquisition source located after the first audio acquisition source in the audio acquisition source set as the second audio acquisition source.

[0030] Optionally, before creating the current audio acquisition instance, the terminal device obtains at least one of the following: device model, system version information, and historical decoding success rate of each audio acquisition source. Based on the obtained information, it determines the usage priority of each audio acquisition source in the audio acquisition source set and sorts the audio acquisition sources according to the usage priority.

[0031] Optionally, the terminal device obtains the historical successful decoding count and historical acquisition count for each audio acquisition source, determines the historical decoding success rate for each audio acquisition source based on the ratio of the historical successful decoding count to the historical acquisition count, and determines the usage priority of each audio acquisition source in descending order of historical decoding success rate. If no historical decoding success rate exists, the usage priority of each audio acquisition source is determined according to a preset default order.

[0032] Optionally, the screen mirroring connection confirmation command is generated for the target candidate device in the candidate device list. After stopping the currently running audio acquisition instance and releasing the recording resources, the terminal device clears the sound wave discovery source identifier corresponding to the target candidate device from the candidate device list, and initiates the screen mirroring connection process based on the device information of the target candidate device.

[0033] Secondly, a sound wave projection search acquisition and control device is provided. This device includes a status gating module, an audio acquisition module, a diagnostic source-cutting module, and a resource release module.

[0034] The status gating module is used to obtain the service status information of the terminal device, determine whether to allow audio acquisition based on the service status information, and output a prohibition command when the service status information does not meet the corresponding allowed acquisition conditions.

[0035] The audio acquisition module is used to determine the first audio acquisition source from the audio acquisition source set when the service status information meets the corresponding allowed acquisition conditions, create the current audio acquisition instance based on the first audio acquisition source, acquire sound wave signals through the current audio acquisition instance to obtain audio sampling data, and send the audio sampling data to the decoder through the decoding interface.

[0036] The diagnostic source switching module is used to receive decoding diagnostic feedback information output by the decoder through the decoding interface, and to send an audio acquisition source switching command to the audio acquisition module when the start-up grace period of the current audio acquisition instance expires, the duration of failure to obtain effective payload reaches the preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval.

[0037] The audio acquisition module is also used to respond to audio acquisition source switching instructions by stopping and releasing the current audio acquisition instance, determining a second audio acquisition source from the audio acquisition source set, and creating a new audio acquisition instance based on the second audio acquisition source.

[0038] The resource release module is used to send a stop acquisition command to the audio acquisition module in response to the screen casting connection confirmation command, so that the audio acquisition module stops the currently running audio acquisition instance and releases the recording resources occupied by the audio acquisition instance.

[0039] Optionally, the device also includes a user authorization module. The user authorization module is used to detect the user's activation operation for the sound wave search entry; in response to the activation operation, output sound wave search function description information; in response to the user's confirmation operation for the sound wave search function description information, call the permission management interface to request recording permission; and after the recording permission is granted, send a status check command to the status gating module.

[0040] Optionally, the status gating module obtains the device connection status, screen projection session status, and automatic connection status from the system service layer, obtains the page visibility status from the page lifecycle callback interface, and obtains the connection confirmation interface display status from the interface management module.

[0041] Optionally, the diagnostic source switching module maintains a diagnostic event counter, the startup timestamp of the current audio acquisition instance, the timestamp of the first diagnostic event, and the timestamp of the most recent audio acquisition source switch.

[0042] Optionally, in response to an audio acquisition source switching command, the audio acquisition module releases the current audio acquisition instance, creates a new audio acquisition instance according to the usage priority of the audio acquisition source, and transmits the pulse code modulation data acquired by the new audio acquisition instance to the decoder.

[0043] Thirdly, an electronic device is provided. This electronic device includes an audio input interface, a processor, and a memory. The memory stores a computer program; the audio input interface is used to receive sound wave signals; when the computer program is executed by the processor, it causes the electronic device to perform the aforementioned sound wave projection search acquisition control method.

[0044] Fourthly, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the aforementioned acquisition and control method for sound wave projection search.

[0045] Beneficial effects:

[0046] 1. By obtaining and checking the service status information of the terminal device before creating an audio acquisition instance, the number of times audio acquisition is started when the sound wave search function is off, the privacy agreement has not been agreed to, the page is not visible, the device is already connected, the screen projection session has been established, the automatic connection process is being executed, or the connection confirmation interface is being displayed can be reduced.

[0047] 2. By checking the status of each service sequentially according to a preset check priority, and terminating the remaining checks when any service status fails to meet the corresponding allowed acquisition conditions, unnecessary status queries and processing operations can be reduced. By determining the usage priority of the audio acquisition source based on at least one of the following: device model, system version information, and historical decoding success rate, the compatibility between the initial audio acquisition source and the terminal device's audio processing environment can be improved.

[0048] 3. By recreating the audio acquisition instance at a 44.1kHz sampling rate when creation at a 48kHz sampling rate fails, the success rate of creating the audio acquisition instance on different terminal devices can be improved. By determining the timing of audio acquisition source switching based on decoding diagnostic feedback, start-up grace period, duration of no payload acquisition, and minimum switching interval, other audio acquisition sources can be tried when the current audio acquisition source continuously fails to acquire a payload, reducing the frequency of audio acquisition instance reconstruction due to short-term decoding anomalies.

[0049] 4. By stopping and releasing the current audio acquisition instance when switching audio acquisition sources, and then creating a new audio acquisition instance based on the second audio acquisition source, the situation where multiple audio acquisition instances simultaneously occupy recording resources can be reduced. By stopping the audio acquisition instance and releasing recording resources after receiving the screen casting connection confirmation command, the lifecycle of the audio acquisition instance can be matched with the screen casting search and screen casting connection process. Attached Figure Description

[0050] Figure 1 This is a diagram illustrating the overall architecture of a sound wave projection search system provided in one embodiment of this application.

[0051] Figure 2 This is a flowchart illustrating the overall process of acoustic wave projection search and acquisition control in one embodiment of this application.

[0052] Figure 3 This is a flowchart illustrating decoding diagnosis and audio acquisition source switching according to one embodiment of this application.

[0053] Figure 4 This is a functional block diagram of an acoustic wave projection search and acquisition control device provided in one embodiment of this application. Reference numerals: 100, terminal device; 110, user authorization module; 111, permission management interface;

[0054] 112. Sound wave search entry; 113. Function configuration storage area; 114. Privacy status storage area;

[0055] 120. State gating module; 121. System service layer; 122. Page lifecycle callback interface;

[0056] 123. Interface Management Module; 124. Business Status Information; 130. Audio Acquisition Source Priority Management Unit;

[0057] 131. Audio acquisition source set; 132. Microphone audio acquisition source; 133. Speech recognition audio acquisition source;

[0058] 134. Unprocessed audio acquisition source; 135. Historical decoding result storage unit; 140. Audio input interface;

[0059] 141. Sound wave transmitter; 142. Sound wave signal; 150. Audio acquisition module; 151. Current audio acquisition instance; 152. New audio acquisition instance; 153. Audio sampling data; 154. Decoding interface;

[0060] 160. Decoder; 161. Payload; 162. Decoding diagnostic feedback information; 170. Diagnostic source switching module; 171. Diagnostic event counter; 172. Startup timestamp; 173. First timestamp; 174. Most recent audio acquisition source switching timestamp; 175. Audio acquisition source switching command; 180. Resource release module; 181. Stop acquisition command; 190. Candidate device list; 191. Target candidate device;

[0061] 192. Sound wave source identification; 193. Screen casting connection confirmation command; 194. Screen casting connection process. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0063] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0064] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0065] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0066] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0067] The acoustic wave projection search acquisition and control method provided in this embodiment can be executed by an acoustic wave projection search acquisition and control device. The acoustic wave projection search acquisition and control device can be implemented through software and / or hardware, and can consist of two or more physical entities, or a single physical entity. The acoustic wave projection search acquisition and control device can be located in the projection sending end, the projection receiving end, or other electronic devices communicatively connected to the projection sending end. The electronic devices can be mobile phones, tablets, laptops, desktop computers, interactive whiteboards, conference terminals, projection devices, or other intelligent devices with data processing and communication functions.

[0068] For ease of understanding, this application embodiment uses a screen projector as an application carrier for the acquisition and control device of sound wave screen projection search as an example description. The screen projector can be an electronic device used to establish a communication connection between the screen projection transmitter and the display device, and to transmit image data, audio data, video data, control data, or other data to be projected from the screen projection transmitter to the display device. The screen projector can be set up independently or integrated into a monitor, television, interactive whiteboard, conference terminal, projection device, or other display device.

[0069] A screen mirroring device can function as a screen mirroring sender, transmitting locally generated or acquired data from other devices to a screen mirroring receiver. It can also function as a screen mirroring receiver, receiving screen mirroring data from mobile phones, tablets, laptops, or other screen mirroring senders, and outputting that data to a display device connected to the screen mirroring device. Furthermore, a screen mirroring device can simultaneously perform both screen mirroring sending and receiving functions.

[0070] Generally, a screen mirroring device may include at least one of a processor, memory, communication module, and display interface. The communication module may include a wireless communication module and / or a wired communication module. The wireless communication module may support wireless LAN, Bluetooth, ultra-wideband, near-field communication, mobile communication, acoustic communication, or other wireless communication methods; the wired communication module may support Ethernet, Universal Serial Bus, or other wired communication methods. The display interface may include a high-definition multimedia interface, a DisplayPort interface, a Universal Serial Bus interface, a video graphics array interface, or other interfaces capable of transmitting image and / or audio data.

[0071] The screen mirroring device can access a local area network (LAN) via Wi-Fi and search for available screen mirroring receivers within the LAN using a network service discovery mechanism. The device can receive device discovery messages broadcast by one or more screen mirroring receivers within the LAN and, based on these messages, obtain at least one of the following: device name, device identifier, network address, service type, port number, connection status, and device capabilities, thereby generating or updating the list of available screen mirroring devices.

[0072] The screen mirroring device can also be equipped with a microphone, a sound wave receiving module, or other sound acquisition modules to collect sound wave signals sent by the screen mirroring receiver. The screen mirroring device can demodulate and analyze the collected sound wave signals to obtain sound wave candidate information. The sound wave candidate information may include at least one of the following: screen mirroring code, network address, device identifier, device name, timestamp, verification information, and sound wave signal strength. The screen mirroring device can associate the sound wave candidate information with the corresponding device in the list of eligible screen mirroring devices based on the matching relationship between the sound wave candidate information and the device information obtained through a network service discovery mechanism.

[0073] The screen mirroring device can establish an association between candidate acoustic wave information and screen-sharing devices based on a device identifier, network address, screen mirroring code, device name, or a combination of at least two of the above information. This association can be stored independently of the list row objects in the screen-sharing device list, so that even when the screen-sharing device list is refreshed, the list row objects are rebuilt, or the device display order changes, the candidate acoustic wave information can still be re-associated to the corresponding screen-sharing device based on the combined identifier.

[0074] Typically, the screen mirroring device can install at least one operating system, including but not limited to Android, Linux, Windows, HarmonyOS, or an embedded real-time operating system. The screen mirroring device can run at least one screen mirroring application based on the operating system. The screen mirroring application can be an application that comes pre-installed with the operating system, or an application downloaded and installed from a server or other electronic device. The sound wave screen mirroring search acquisition and control device can be a standalone application or implemented as a functional module within the screen mirroring application.

[0075] The screen mirroring application can perform at least one of the following operations: device discovery, device list generation, acoustic signal parsing, candidate information caching, device matching, connection request sending, screen mirroring code verification, screen mirroring link establishment, and screen mirroring status update. For example, a user can select a target screen mirroring device from the list of available devices. The screen mirroring device obtains the corresponding network address and / or screen mirroring code based on the selected target device, sends a connection request to the target device, and establishes a screen mirroring communication link after successful connection verification.

[0076] Understandably, users can select a target projection device or initiate a projection operation via touchscreen, keyboard, mouse, remote control, physical buttons, voice commands, or other input methods. After receiving the user's operation, the projection device can determine the target projection device and its corresponding connection parameters based on the user's selected device information, acoustic wave candidate information, and the current network connection status, and then execute the projection connection operation.

[0077] The projection device may include one or more processing cores. The processing core can implement the acoustic wave projection search acquisition and control method provided in this application embodiment using pure software, or it can be implemented using a combination of software and hardware. For example, it can be implemented using at least one hardware form selected from digital signal processors, field-programmable gate arrays, application-specific integrated circuits, and programmable logic arrays. The projection device may also integrate one or more of a central processing unit, graphics processor, network processor, audio processor, and modem.

[0078] The memory of the projection device can be used to store at least one of the following: computer program, list of projectable devices, device discovery information, acoustic wave candidate information, device association relationships, projection connection parameters, and projection status information. The processor can call and execute the computer program stored in the memory to cause the projection device to execute the acoustic wave projection search acquisition and control method provided in the embodiments of this application.

[0079] The computer program can be presented in a form suitable for the operation of the projection device, such as an application, system service, background process, system plugin, software development kit, firmware program, or web page plugin. In some embodiments, the acquisition and control method for acoustic wave projection search can also be executed collaboratively by multiple software modules in the projection device. The multiple software modules may include at least one of a device discovery module, an acoustic wave parsing module, a candidate information management module, a device matching module, a list update module, and a connection control module.

[0080] Example 1

[0081] like Figure 1 As shown, the sound wave projection search system can run on a terminal device. The terminal device can be a mobile terminal, tablet computer, personal computer, projection control terminal, or other electronic device with audio acquisition and projection control capabilities.

[0082] The sound wave projection search system may include a user authorization module, a status gating module, an audio acquisition source priority management function, an audio input interface, an audio acquisition module, a decoder, a diagnostic source-switching module, a candidate device list, a projection connection process, and a resource release module.

[0083] The user authorization module is used to detect the user's startup operation for the sound wave search entry, output sound wave search function description information in response to the startup operation, and call the permission management interface to request recording permission in response to the user's confirmation operation.

[0084] After recording permission is granted, the user authorization module sends a status check command to the status gating module. When recording permission is denied, the user authorization module can set the sound wave search function status flag to the off state, clear the candidate device list, and record the recording permission denial status.

[0085] The status gating module is used to acquire service status information and determine whether to allow audio capture to start based on the service status information. Service status information can be provided by at least one of the following: system service layer, page lifecycle callback interface, interface management module, function configuration storage area, and privacy status storage area.

[0086] The audio acquisition source priority management function determines the usage priority of audio acquisition sources based on at least one of the following: device model, system version information, and historical decoding success rate of each audio acquisition source. This function can be implemented by the audio acquisition module, the diagnostic source-cutting module, or a separate software functional unit.

[0087] The audio input interface is used to receive sound wave signals sent by the device to be discovered. The audio acquisition module creates an audio acquisition instance based on the selected audio acquisition source and obtains audio sampling data through the audio acquisition instance.

[0088] The audio acquisition module transmits the audio sample data to the decoder. The decoder performs frame header detection, frame header verification, and payload parsing on the audio sample data, and outputs the payload or decoding diagnostic feedback information to the diagnostic source-cutting module.

[0089] The diagnostic source switching module maintains a diagnostic event count and corresponding timestamps based on the decoded diagnostic feedback information, and determines whether the audio acquisition source switching conditions are met. When the audio acquisition source switching conditions are met, the diagnostic source switching module sends an audio acquisition source switching command to the audio acquisition module.

[0090] In response to the audio acquisition source switching command, the audio acquisition module stops and releases the current audio acquisition instance, determines the second audio acquisition source, and creates a new audio acquisition instance based on the second audio acquisition source.

[0091] When the decoder obtains the payload, the terminal device generates or updates the candidate device record based on the device information carried by the payload, and can set the sound wave discovery source identifier in the corresponding candidate device record.

[0092] When a user confirms the connection to a target device in the candidate device list, the screen mirroring service generates a screen mirroring connection confirmation command. In response to the screen mirroring connection confirmation command, the resource release module sends a stop-capture command to the audio capture module to stop the current audio capture instance and release the recording resources.

[0093] After the recording resources are released, the terminal device clears the sound wave discovery source identifier corresponding to the target candidate device and initiates the screen casting connection process based on the device information of the target candidate device.

[0094] Example 2

[0095] like Figure 2 As shown, this embodiment provides a sound wave projection search and acquisition control method executed by a terminal device.

[0096] In execution Figure 2 Before the process shown, the terminal device can first execute the user authorization process.

[0097] The terminal device detects the user's activation operation for the sound wave search entry and responds by outputting sound wave search function description information. After the user confirms the sound wave search function description information, the terminal device calls the permission management interface to request recording permission.

[0098] After recording permission is granted, the terminal device enters the service status check process. When recording permission is denied, the terminal device sets the sound wave search function status flag to the off state, clears the candidate device list, and records the recording permission denial status.

[0099] The terminal device will not request recording permissions again by calling the permission management interface unless a new user action is detected targeting the sound wave search entry. This reduces the likelihood of repeatedly requesting recording permissions based on page entry events, page refresh events, or scheduled tasks.

[0100] Figure 2 The method shown may include the following steps.

[0101] Step S201: Obtain service status information and determine whether to allow audio capture to start.

[0102] The terminal device obtains at least two service status information items.

[0103] Business status information may include the on / off status of the sound wave search function, the status of consent to the privacy agreement, the status of page visibility, the status of device connection, the status of screen mirroring session, the status of automatic connection, and the status of connection confirmation interface display.

[0104] In one implementation, the allowable data collection conditions corresponding to each service status include:

[0105] The sound wave search function is enabled.

[0106] The privacy policy agreement has been agreed to.

[0107] The page visibility status indicates that the sound wave search page is visible;

[0108] The device connection status indicates that the terminal device is not connected to the target device and is not in the process of connecting;

[0109] The screen mirroring session status indicates that no screen mirroring session has been established yet;

[0110] The automatic connection status indicates that the automatic connection process is not currently being executed;

[0111] The connection confirmation screen is currently not displayed.

[0112] The above-mentioned conditions for allowing data collection can be adjusted according to the specific implementation of the screen projection service.

[0113] The status gating module can obtain the device connection status, screen projection session status, and automatic connection status from the system service layer, obtain the page visibility status from the page lifecycle callback interface, and obtain the connection confirmation interface display status from the interface management module.

[0114] The status gating module can also obtain the on / off status of the acoustic search function from the function configuration storage area, and the privacy agreement consent status from the privacy status storage area.

[0115] In one implementation, the status gating module checks the status of each service item sequentially according to the following preset check priority:

[0116] The status of the sound wave search function switch, privacy agreement agreement, page visibility, device connection, screen mirroring session, automatic connection, and connection confirmation interface display.

[0117] When any service status does not meet the corresponding allowed collection conditions, the status gating module terminates the inspection of the remaining service statuses, determines that audio collection is prohibited from starting, and prohibits the creation of audio collection instances.

[0118] When all the checked business statuses meet the corresponding allowed collection conditions, the status gating module determines that audio collection can be started.

[0119] Step S202: Determine the first audio acquisition source and create the current audio acquisition instance.

[0120] After confirming that audio acquisition is allowed, the terminal device determines the first audio acquisition source from the set of audio acquisition sources.

[0121] The audio acquisition source set can include, in sequence, microphone audio acquisition sources, speech recognition audio acquisition sources, and unprocessed audio acquisition sources.

[0122] Microphone audio acquisition sources can correspond to the ordinary microphone acquisition path of the terminal device. Speech recognition audio acquisition sources can correspond to the audio acquisition path provided by the operating system for speech recognition scenarios. Unprocessed audio acquisition sources can correspond to audio acquisition paths that do not perform or perform only a portion of audio preprocessing.

[0123] In different operating systems or audio processing frameworks, the names of each audio acquisition source can be different, as long as multiple audio acquisition sources correspond to different audio acquisition paths or audio preprocessing methods.

[0124] In one implementation, before creating the current audio acquisition instance, the terminal device obtains at least one of the following: device model, system version information, and historical decoding success rate of each audio acquisition source.

[0125] The terminal device determines the usage priority of each audio acquisition source based on the acquired information, and sorts the audio acquisition sources in the audio acquisition source set according to the usage priority. The first audio acquisition source after sorting can be determined as the first audio acquisition source.

[0126] If no corresponding device model configuration, system version configuration, or historical decoding success rate is available, a preset default order can be used. The default order can be microphone audio acquisition source, speech recognition audio acquisition source, and unprocessed audio acquisition source.

[0127] The audio acquisition module uses the first sampling rate, mono pulse code modulation format, and the first audio acquisition source to create the current audio acquisition instance.

[0128] In one implementation, the first sampling rate is 48 kHz.

[0129] When creating the current audio acquisition instance using a 48kHz sampling rate fails, the audio acquisition module recreates the current audio acquisition instance using a second sampling rate lower than the first sampling rate, a mono pulse code modulation format, and the first audio acquisition source.

[0130] In one implementation, the second sampling rate is 44.1 kHz.

[0131] Instance creation failures may include the audio acquisition interface returning unsupported sampling parameters, unavailable recording resources, audio acquisition object initialization failure, or other results that indicate that an audio acquisition instance cannot be created under the current parameters.

[0132] Step S203: Acquire sound wave signals and send audio sampling data to the decoder.

[0133] Once the current audio acquisition instance is started, it receives sound wave signals through the audio input interface of the terminal device and generates audio sampling data according to the corresponding sampling rate, number of channels and encoding format.

[0134] Audio sampling data can be pulse code modulation data. The audio acquisition module can send the audio sampling data to the decoder through a data callback interface, buffer queue, shared buffer, or other data transmission interface.

[0135] The decoder performs frame header detection, frame header verification, and payload parsing on the audio sample data.

[0136] When the decoder successfully obtains the verified payload data, it identifies the payload data as the valid payload. The valid payload may include at least one of the following: device identifier, device name, connection address, or other device information required for screen mirroring connection.

[0137] The terminal device can generate candidate device records based on the device information in the payload, or update the existing candidate device records in the candidate device list that correspond to the device information, and set the acoustic wave discovery source identifier in the candidate device records obtained through acoustic wave signals.

[0138] Step S204: Receive decoding diagnostic feedback information

[0139] The decoder outputs decoding diagnostic feedback information based on the audio sample data.

[0140] The decoded diagnostic feedback information includes the diagnostic event type and the time the diagnostic event occurred. The diagnostic event type can include at least one of the following: frame header check rejection event and search miss event.

[0141] A frame header verification rejection event indicates that the decoder detected a candidate frame header, but the candidate frame header failed the preset verification. A search miss event indicates that the decoder did not detect a matching sound wave data frame in the corresponding audio sample data or search window.

[0142] The diagnostic source-cutting module counts the diagnostic events corresponding to the decoding diagnostic feedback information and records the first timestamp of the first diagnostic event that occurs during the current audio acquisition instance's operation.

[0143] During the current audio acquisition instance, if the first timestamp has already been recorded, subsequent diagnostic events can increment the corresponding count value without changing the first timestamp.

[0144] When the decoder outputs a payload, the diagnostic source-cutting module resets the diagnostic event count and the first timestamp. The first timestamp can be cleared, set to an invalid value, or updated to a preset initial value.

[0145] Step S205: Determine whether the audio acquisition source switching condition is met.

[0146] The diagnostic source-cutting module records the startup time of the current audio acquisition instance and determines the runtime of the current audio acquisition instance based on the current time and the startup time.

[0147] The diagnostic source switching module determines the duration of the absence of a valid payload based on the current time and the first timestamp, and determines the time elapsed since the last audio source switching based on the current time and the most recent audio source switching time.

[0148] The diagnostic source-cutting module determines whether the following three conditions are met simultaneously:

[0149] The startup grace period for the current audio capture instance has expired;

[0150] The duration of failure to acquire a payload reaches a preset duration threshold;

[0151] The time elapsed since the last audio source switch has reached the minimum switching interval.

[0152] If none of the above three conditions are met simultaneously, the audio acquisition module continues to acquire sound wave signals using the current audio acquisition instance.

[0153] When all three conditions above are met, the diagnostic source switching module sends an audio acquisition source switching command to the audio acquisition module.

[0154] The startup grace period is used to reserve startup, buffering, and stable operation time for newly created audio acquisition instances. The preset duration threshold is used to distinguish between intermittent diagnostic events and a state where a payload cannot be obtained continuously. The minimum switching interval is used to limit the shortest time between two adjacent audio acquisition source switches.

[0155] Step S206: Stop and release the current audio acquisition instance, and create a new audio acquisition instance.

[0156] In response to the audio acquisition source switching command, the audio acquisition module stops the current audio acquisition instance, cancels the audio sampling data callback, and releases the recording and cache resources occupied by the current audio acquisition instance.

[0157] The audio acquisition module determines a second audio acquisition source that is different from the first audio acquisition source from the set of audio acquisition sources.

[0158] When using audio acquisition sources in the default order, the second audio acquisition source can be an audio acquisition source that is located after the first audio acquisition source in the set of audio acquisition sources.

[0159] For example, when the first audio acquisition source is a microphone audio acquisition source, the second audio acquisition source can be a speech recognition audio acquisition source; when the first audio acquisition source is a speech recognition audio acquisition source, the second audio acquisition source can be an unprocessed audio acquisition source.

[0160] If the audio acquisition source set has already been sorted according to usage priority, the second audio acquisition source can be the audio acquisition source that is located after the first audio acquisition source in the sorted audio acquisition source set.

[0161] The audio acquisition module creates a new audio acquisition instance based on the second audio acquisition source, and continues to acquire sound wave signals through the new audio acquisition instance. The audio sampling data acquired by the new audio acquisition instance is then sent to the decoder.

[0162] After the audio acquisition source switch is completed, the diagnostic source switching module updates the time of the most recent audio acquisition source switch and can reset the diagnostic event count value and first timestamp corresponding to the previous audio acquisition instance.

[0163] Step S207: Respond to the screen mirroring connection confirmation command and release the recording resources.

[0164] The candidate device list can display one or more candidate devices. The screen mirroring connection confirmation command can be generated for the target candidate device in the candidate device list.

[0165] For example, when a user selects a target candidate device from the candidate device list and performs a connection confirmation operation, the terminal device generates a screen mirroring connection confirmation command.

[0166] In response to the screen mirroring connection confirmation command, the resource release module sends a stop acquisition command to the audio acquisition module.

[0167] The audio acquisition module stops the currently running audio acquisition instance, cancels the audio sampling data callback, and releases the recording resources occupied by the audio acquisition instance.

[0168] After stopping the audio acquisition instance and releasing the recording resources, the terminal device clears the sound wave discovery source identifier corresponding to the target candidate device from the candidate device list, and initiates the screen casting connection process based on the device information of the target candidate device.

[0169] Therefore, once the target candidate device has been identified and the screen mirroring connection process has begun, the terminal device will no longer continue to perform the audio acquisition corresponding to this sound wave search.

[0170] Example 3

[0171] like Figure 3 As shown in the figure, this embodiment further explains the decoding diagnostic feedback information processing and audio acquisition source switching process.

[0172] When the audio acquisition module creates the current audio acquisition instance, it records the start time Ts of the current audio acquisition instance.

[0173] The diagnostic source-cutting module receives decoded diagnostic feedback information output from the decoder. This feedback information includes the diagnostic event type and the time the event occurred.

[0174] When the decoder outputs a payload, the diagnostic source-cutting module resets the diagnostic event counter and the first timestamp, and continues to use the current audio acquisition instance to acquire sound wave signals.

[0175] When the decoder does not output a payload, and the decoding diagnostic feedback indicates that a frame header check rejection event or a search miss event has occurred, the diagnostic source-cutting module increments the count value of the corresponding diagnostic event.

[0176] For the first diagnostic event occurring during the current audio acquisition instance's operation, the diagnostic source-cutting module records the first timestamp T0. The first timestamp T0 indicates the start time of the diagnostic process where the current audio acquisition instance has not continuously acquired a payload.

[0177] The diagnostic source switching module also records the most recent audio acquisition source switching time Tc. For audio acquisition instances that are newly created and have not yet undergone audio acquisition source switching, the most recent audio acquisition source switching time Tc can be initialized to the current audio acquisition instance's startup time Ts, or initialized to a preset time that can satisfy the first source switching judgment.

[0178] Given the current time is T, the diagnostic source module can determine:

[0179] The runtime of the current audio acquisition instance is the difference between T and Ts;

[0180] The duration of no effective payload is the difference between T and T0;

[0181] The time elapsed since the last audio source switch is the difference between T and Tc.

[0182] The diagnostic source switching module determines whether the startup grace period of the current audio acquisition instance has expired, whether the duration of the failure to obtain a valid payload has reached the preset duration threshold, and whether the time since the last audio acquisition source switch has reached the minimum switching interval.

[0183] When at least one of the three conditions is not met, the diagnostic source switching module does not send an audio acquisition source switching command, and the audio acquisition module continues to use the current audio acquisition instance to acquire sound wave signals.

[0184] When all three conditions are met, the diagnostic source switching module sends an audio acquisition source switching command to the audio acquisition module.

[0185] The audio acquisition module stops and releases the current audio acquisition instance, and determines the second audio acquisition source according to the usage priority of the audio acquisition source.

[0186] In one fixed-order implementation, the audio acquisition source set is arranged in the order of microphone audio acquisition source, speech recognition audio acquisition source, and unprocessed audio acquisition source.

[0187] When the sound wave search is initiated for the first time, the audio acquisition module can select a microphone audio acquisition source to create the current audio acquisition instance.

[0188] When the audio acquisition instance corresponding to the microphone audio acquisition source meets the switching conditions, the audio acquisition module selects the speech recognition audio acquisition source as the second audio acquisition source.

[0189] When the audio acquisition instance corresponding to the speech recognition audio acquisition source meets the switching conditions again, the audio acquisition module selects the unprocessed audio acquisition source as the next audio acquisition source.

[0190] In one dynamic sorting implementation, the terminal device determines the usage priority of each audio acquisition source based on at least one of the following: device model, system version information, and historical decoding success rate.

[0191] The historical decoding success rate can be determined by the ratio of the number of successful historical decodings to the number of historical acquisitions of the corresponding audio acquisition source.

[0192] For example, the historical decoding success rate Pi of audio acquisition source i can be expressed as:

[0193] Pi = Ni / Mi;

[0194] Where Ni represents the number of successful decodings in the history corresponding to audio acquisition source i, and Mi represents the number of acquisitions in the history corresponding to audio acquisition source i.

[0195] The terminal device determines the usage priority of each audio acquisition source according to the historical decoding success rate, from highest to lowest. If no corresponding historical decoding success rate exists, the usage priority of each audio acquisition source is determined according to the preset default order.

[0196] Historical decoding results can be categorized and stored according to device model, system version, or a combination of both. Therefore, the same audio source can have different usage priorities on different device models or different system versions.

[0197] After determining the second audio acquisition source, the audio acquisition module prioritizes using a 48kHz sampling rate, mono pulse code modulation format, and the second audio acquisition source to create a new audio acquisition instance.

[0198] When creating a new audio acquisition instance using a 48kHz sampling rate fails, the audio acquisition module creates a new audio acquisition instance using a 44.1kHz sampling rate, a mono pulse code modulation format, and a second audio acquisition source.

[0199] After a new audio acquisition instance is successfully created, the audio acquisition module continues to acquire sound wave signals and sends the new audio sample data to the decoder.

[0200] When the audio acquisition instance corresponding to the last audio acquisition source in the audio acquisition source set still cannot obtain a valid payload, the terminal device can continue to use the last audio acquisition source for acquisition, or terminate the current sound wave search according to the preset service strategy. This process does not affect the aforementioned audio acquisition source switching conditions and audio acquisition instance release mechanism.

[0201] Example 4

[0202] like Figure 4 As shown, this embodiment provides a data acquisition and control device for acoustic wave projection search.

[0203] The device includes a status gating module, an audio acquisition module, a diagnostic source-cutting module, and a resource release module. In some embodiments, the device also includes a user authorization module.

[0204] The aforementioned modules can be software functional modules formed by the processor executing computer programs, or they can be implemented through hardware circuits, or through a combination of software and hardware.

[0205] The user authorization module communicates with the permission management interface to detect the user's activation operation for the sound wave search entry; in response to the activation operation, it outputs sound wave search function description information; in response to the user's confirmation operation for the sound wave search function description information, it calls the permission management interface to apply for recording permission; and after the recording permission is granted, it sends a status check command to the status gating module.

[0206] When recording permission is denied, the user authorization module can also set the sound wave search function status flag to the off state, clear the candidate device list, and record the recording permission denial status.

[0207] The status gating module is used to obtain the service status information of the terminal device and determine whether to allow audio acquisition to start based on the service status information.

[0208] The status gating module can obtain the device connection status, screen projection session status, and automatic connection status from the system service layer, obtain the page visibility status from the page lifecycle callback interface, and obtain the connection confirmation interface display status from the interface management module.

[0209] The status gating module can also obtain the on / off status of the acoustic search function from the function configuration storage area, and the privacy agreement consent status from the privacy status storage area.

[0210] When all service states meet the corresponding allowed acquisition conditions, the status gating module sends an allowed start command to the audio acquisition module; when any service state does not meet the corresponding allowed acquisition conditions, it sends a prohibited start command to the audio acquisition module.

[0211] The audio acquisition module communicates with the audio input interface of the terminal device to determine the first audio acquisition source from the set of audio acquisition sources, create the current audio acquisition instance based on the first audio acquisition source, and acquire sound wave signals through the current audio acquisition instance to obtain audio sampling data.

[0212] The audio acquisition module sends the audio sampling data to the decoder through the decoding interface.

[0213] The audio acquisition module can also implement audio acquisition source priority management, or communicate with the software unit used to implement audio acquisition source priority management.

[0214] The audio acquisition source priority management function determines the usage priority of audio acquisition sources based on at least one of the following: device model, system version information, and historical decoding success rate of each audio acquisition source, and sorts the audio acquisition sources according to their usage priority.

[0215] The diagnostic source-cutting module receives the decoding diagnostic feedback information output by the decoder through the decoding interface.

[0216] The diagnostic source switching module maintains a diagnostic event counter, the startup timestamp of the current audio acquisition instance, the timestamp of the first diagnostic event, and the timestamp of the most recent audio acquisition source switch.

[0217] When the startup grace period of the current audio acquisition instance expires, the duration of the period without obtaining a valid payload reaches the preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval, the diagnostic source switching module sends an audio acquisition source switching command to the audio acquisition module.

[0218] In response to the audio acquisition source switching command, the audio acquisition module stops and releases the current audio acquisition instance, determines the second audio acquisition source according to the usage priority of the audio acquisition source, and creates a new audio acquisition instance based on the second audio acquisition source.

[0219] The audio acquisition module transmits the pulse code modulation data acquired from the new audio acquisition instance to the decoder.

[0220] The resource release module is used to receive the screen mirroring connection confirmation command and send a stop acquisition command to the audio acquisition module.

[0221] In response to the stop acquisition command, the audio acquisition module stops the currently running audio acquisition instance and releases the recording resources occupied by the audio acquisition instance.

[0222] In some implementations, the resource release module can also send an identifier cleanup command to the candidate device management function to remove the acoustic discovery source identifier corresponding to the target candidate device.

[0223] The candidate device management function can generate or update candidate device records based on the payload output by the decoder, and manage the acoustic discovery source identifiers in the candidate device records.

[0224] It should be noted that, Figure 4 The module division shown is merely an example of functional division. In other implementations, multiple modules can be merged into one module, or a single module can be divided into multiple sub-modules, as long as the corresponding functions can be achieved.

[0225] Example 5

[0226] This application also provides a computer-readable storage medium storing a computer program thereon, the instructions of which are adapted to be loaded by a processor and executed by the method steps of Embodiment 1 shown above. The specific execution process can be found in the detailed description of the embodiments, and will not be repeated here. The device containing the storage medium can be an electronic device such as a personal computer, laptop computer, smartphone, or tablet computer.

[0227] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0228] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0229] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function selected in one or more boxes.

[0230] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function selected in one or more boxes.

[0231] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0232] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0233] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0234] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0235] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for acquiring and controlling acoustic wave projection search, applied to a terminal device, characterized in that, include: Obtain the service status information of the terminal device, and determine whether to allow audio capture to start based on the service status information; When the service status information meets the corresponding allowed collection conditions, a first audio collection source is determined from the audio collection source set, and a current audio collection instance is created based on the first audio collection source; The current audio acquisition instance acquires sound wave signals to obtain audio sampling data, and then sends the audio sampling data to the decoder. Receive decoding diagnostic feedback information output by the decoder for the audio sample data; When the start-up grace period of the current audio acquisition instance expires, the duration of the failure to acquire a valid payload reaches a preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval, the current audio acquisition instance is stopped and released. A second audio acquisition source different from the first audio acquisition source is determined from the audio acquisition source set, and a new audio acquisition instance is created based on the second audio acquisition source to continue acquiring the sound wave signal. In response to the screen mirroring connection confirmation command, the currently running audio acquisition instance of the terminal device is stopped, and the recording resources occupied by the audio acquisition instance are released; Specifically, if the business status information does not meet the corresponding allowed collection conditions, the creation of the audio collection instance is prohibited.

2. The acquisition and control method for acoustic wave projection search according to claim 1, characterized in that, Before obtaining the business status information, the method further includes: Detect the user's startup action targeting the sound wave search entry point; In response to the aforementioned activation operation, output sound wave search function description information; In response to the user's confirmation operation of the description information input for the sound wave search function, the terminal device's permission management interface is invoked to request recording permission; After the recording permission is granted, the service status information is obtained.

3. The acquisition and control method for acoustic wave projection search according to claim 2, characterized in that, The method further includes: When the recording permission is denied, the sound wave search function status flag is set to the off state, and the candidate device list is cleared; Record the recording permission denial status, and do not call the permission management interface again to request recording permission if no new startup operation is detected by the user for the sound wave search entry.

4. The acquisition and control method for acoustic wave projection search according to claim 1, characterized in that, The business status information includes at least two of the following: The status of the sound wave search function switch, the privacy agreement agreement, the page visibility, the device connection status, the screen mirroring session status, the automatic connection status, and the connection confirmation interface display status. The step of determining whether to allow audio capture to be started based on the service status information includes: Determine whether each service status meets the allowable data collection conditions corresponding to that service status; When all business statuses meet the corresponding conditions for allowing data collection, audio data collection is allowed to start.

5. The acquisition and control method for acoustic wave projection search according to claim 4, characterized in that, The process of determining whether the status of each service item meets the corresponding allowed data collection conditions includes: The following are checked in sequence according to the preset check priority: the on / off status of the sound wave search function, the privacy agreement agreement status, the page visibility status, the device connection status, the screen casting session status, the automatic connection status, and the connection confirmation interface display status. If any service status does not meet the corresponding allowed collection conditions, terminate the check on the remaining service statuses and determine that audio collection should be prohibited.

6. The acquisition and control method for acoustic wave projection search according to claim 1, characterized in that, The step of creating the current audio acquisition instance based on the first audio acquisition source includes: The current audio acquisition instance is created using a first sampling rate, a mono pulse code modulation format, and the first audio acquisition source; If creating the current audio acquisition instance using the first sampling rate fails, the current audio acquisition instance is created using a second sampling rate lower than the first sampling rate and the first audio acquisition source. The first sampling rate is 48 kHz, and the second sampling rate is 44.1 kHz.

7. The acquisition and control method for acoustic wave projection search according to claim 1, characterized in that, The decoded diagnostic feedback information includes the type of diagnostic event and the time when the diagnostic event occurred; The diagnostic event types include at least one of frame header verification rejection event and search miss event; The method further includes: The diagnostic events corresponding to the decoded diagnostic feedback information are counted, and the first timestamp of the first occurrence of the diagnostic event during the current audio acquisition instance operation is recorded; When the decoder outputs a payload, the count value of the diagnostic event and the first timestamp are reset.

8. The acquisition and control method for acoustic wave projection search according to claim 7, characterized in that, The step of stopping and releasing the current audio acquisition instance when the startup grace period of the current audio acquisition instance expires, the duration of not acquiring a valid payload reaches a preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval includes: Record the start time of the current audio acquisition instance; The runtime of the current audio acquisition instance is determined based on the current time and the start time. The duration of the failure to obtain a payload is determined based on the current time and the first timestamp; The time elapsed since the last audio source switch is determined by comparing the current time with the time of the most recent audio source switch. When the runtime reaches the startup grace period, the duration of the failure to acquire a payload reaches the preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval, the current audio acquisition instance is stopped and released.

9. The acquisition and control method for acoustic wave projection search according to claim 1, characterized in that, The audio acquisition source set includes, in sequence, microphone audio acquisition source, speech recognition audio acquisition source, and unprocessed audio acquisition source; Determining the second audio source from the set of audio sources includes: The audio source located after the first audio source in the set of audio sources is identified as the second audio source.

10. The acquisition and control method for acoustic wave projection search according to claim 1, characterized in that, Before creating the current audio acquisition instance, the method further includes: Obtain at least one of the following: device model of the terminal device, system version information, and historical decoding success rate of each audio acquisition source; The usage priority of each audio acquisition source in the audio acquisition source set is determined based on at least one of the device model, the system version information, and the historical decoding success rate. The audio sources in the audio source set are sorted according to the usage priority.

11. The acquisition and control method for acoustic wave projection search according to claim 10, characterized in that, Determining the usage priority based on the historical decoding success rate includes: Obtain the historical successful decoding count and historical acquisition count for each audio acquisition source; The historical decoding success rate of each audio acquisition source is determined based on the ratio of the historical successful decoding count to the historical acquisition count. The priority of each audio acquisition source is determined according to its historical decoding success rate from high to low. In the absence of the historical decoding success rate, the priority of each audio acquisition source is determined according to the preset default order.

12. The acquisition and control method for acoustic wave projection search according to claim 1, characterized in that, The screen mirroring connection confirmation command is generated for the target candidate device in the candidate device list; After responding to the screen mirroring connection confirmation command, stopping the currently running audio acquisition instance of the terminal device, and releasing the recording resources occupied by the audio acquisition instance, the method further includes: Remove the acoustic detection source identifier corresponding to the target candidate device from the candidate device list; The screen mirroring connection process is initiated based on the device information of the target candidate device.

13. A sound wave projection search acquisition and control device, characterized in that, include: The status gating module is used to acquire the service status information of the terminal device, determine whether to allow audio acquisition based on the service status information, and output a prohibition command when the service status information does not meet the corresponding allowed acquisition conditions. The audio acquisition module is used to determine a first audio acquisition source from the audio acquisition source set when the service status information meets the corresponding allowed acquisition conditions, create a current audio acquisition instance based on the first audio acquisition source, acquire sound wave signals through the current audio acquisition instance to obtain audio sampling data, and send the audio sampling data to the decoder through the decoding interface. The diagnostic source switching module is used to receive decoding diagnostic feedback information output by the decoder through the decoding interface, and send an audio acquisition source switching command to the audio acquisition module when the start-up grace period of the current audio acquisition instance expires, the duration of failure to obtain effective payload reaches a preset duration threshold, and the time since the last audio acquisition source switch reaches the minimum switching interval. The audio acquisition module is also configured to respond to the audio acquisition source switching command by stopping and releasing the current audio acquisition instance, determining a second audio acquisition source from the audio acquisition source set, and creating a new audio acquisition instance based on the second audio acquisition source; The resource release module is used to send a stop acquisition command to the audio acquisition module in response to the screen casting connection confirmation command, so that the audio acquisition module stops the currently running audio acquisition instance and releases the recording resources occupied by the audio acquisition instance.

14. The acquisition and control device for acoustic wave projection search according to claim 13, characterized in that, The device also includes a user authorization module; The user authorization module is used for: Detect the user's startup action targeting the sound wave search entry point; In response to the aforementioned activation operation, output sound wave search function description information; In response to the user's confirmation operation of the description information of the sound wave search function, the permission management interface is called to request recording permission; After the recording permission is granted, a status check command is sent to the status gating module.

15. The acquisition and control device for acoustic wave projection search according to claim 13, characterized in that: The status gating module is used to obtain the device connection status, screen projection session status and automatic connection status from the system service layer, obtain the page visibility status from the page lifecycle callback interface, and obtain the connection confirmation interface display status from the interface management module. The diagnostic source switching module is used to maintain a diagnostic event counter, the start timestamp of the current audio acquisition instance, the timestamp of the first diagnostic event, and the timestamp of the most recent audio acquisition source switch. The audio acquisition module is used to respond to the audio acquisition source switching command, release the current audio acquisition instance, create a new audio acquisition instance according to the usage priority of the audio acquisition source, and transmit the pulse code modulation data acquired by the new audio acquisition instance to the decoder.

16. An electronic device, characterized in that, include: Audio input interface; processor; A memory, wherein a computer program is stored; The audio input interface is used to receive sound wave signals; When the computer program is executed by the processor, it causes the electronic device to perform the acquisition and control method for acoustic wave projection search as described in any one of claims 1 to 12.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the acquisition and control method for acoustic wave projection search as described in any one of claims 1 to 12.

Images