Voice wake-up method, device and system

Abstract

The embodiment of the invention discloses a voice wake-up method, device and system. A target voice signal is obtained, a first energy value of the target voice signal relative to a first device is determined, a first wake-up event is broadcasted to a second device in a network group, the second device is a device except the first device in the network group where the current device is located, and the wake-up event at least comprises the first energy value. And receiving a second wake-up event broadcasted by each second device, the second wake-up event at least comprising two energy values of a target voice signal relative to each second device, and determining a wake-up judgment result according to the first energy value and the second energy value, the wake-up judgment result comprising wake-up and non-wake-up. Therefore, the situation that multiple devices are awakened by voice at the same time can be reduced, confusion response is reduced, and the accuracy of voice response of the devices and the user experience are improved.

Description

Technical Field

[0001] The present invention relates to the field of voice signal processing technology, and in particular to a voice wake-up method, device and system. Background Art

[0002] Smart home technology, a core application area merging the Internet of Things and artificial intelligence, has seen explosive growth in recent years. Voice interaction, a mainstream control method, offers significant convenience and enhances the user experience. However, with the increasing prevalence and number of smart home devices in homes, the incidence of voice wake-up conflicts has exceeded 80%. This high density of devices can easily lead to conflicts during voice wake-up, resulting in confusion among device responses. For example, when a user issues a voice command at home, multiple devices may be woken simultaneously, causing them to respond simultaneously or with mixed responses. This not only impacts the user experience but can also cause some devices to malfunction or execute incorrect commands. Summary of the Invention

[0003] In view of this, the purpose of the embodiments of the present invention is to provide a voice wake-up method, device and system, which can reduce the situation where multiple devices are woken up by voice at the same time, reduce confusing responses, and improve the accuracy of the device's voice response and user experience.

[0004] In a first aspect, an embodiment of the present invention provides a voice wake-up method, the method comprising:

[0005] Acquire target speech signal;

[0006] Determine a first energy value corresponding to the target speech signal, where the first energy value is an energy value of the target speech signal relative to a first device, where the first device is a current device;

[0007] Broadcasting a first wake-up event to a second device in a network group, where the second device is a device other than the first device in the network group where the current device is located, and the first wake-up event includes at least the first energy value;

[0008] receiving a second wake-up event broadcast by each second device, where the second wake-up event includes at least a second energy value, where the second energy value is an energy value of the target voice signal relative to each second device;

[0009] A wake-up decision result is determined according to the first energy value and the second energy value, where the wake-up decision result includes wake-up and not wake-up.

[0010] In some embodiments, the target voice signal is a predetermined wake-up voice.

[0011] In some embodiments, determining the first energy value corresponding to the target speech signal includes:

[0012] Obtaining an audio sample corresponding to the target speech signal;

[0013] performing a square operation on each of the audio samples to obtain a first intermediate value;

[0014] performing an averaging operation on each of the first intermediate values to obtain a second intermediate value;

[0015] A square root operation is performed on the second intermediate value to obtain the first energy value.

[0016] In some embodiments, the first device is connected to the second device in the network group via one or more of Wi-Fi, power line communication (PLC), and Bluetooth mesh.

[0017] In some embodiments, broadcasting the wake-up event to the second device in the network group includes:

[0018] The wake-up event is broadcast to the second device in the network group via one or more of Wi-Fi, power line communication (PLC), and Bluetooth mesh.

[0019] In some embodiments, the first wake-up event further includes a unique identifier of the first device.

[0020] In some embodiments, determining the wake-up decision result according to the first energy value and the second energy value includes:

[0021] comparing the received second energy value with the first energy value;

[0022] In response to the second energy value being greater than the first energy value, a wake-up decision result is determined to be not wake-up.

[0023] In some embodiments, determining the wake-up decision result according to the first energy value and the second energy value further includes:

[0024] In response to the second energy value being less than the first energy value, continuing to wait for receiving a second wake-up event until a predetermined stop condition is satisfied;

[0025] The predetermined stop condition is that a predetermined waiting time is reached or the number of received second wake-up events meets a predetermined number, and the predetermined number is determined according to the total number of devices in the network group.

[0026] In some embodiments, determining the wake-up decision result according to the first energy value and the second energy value further includes:

[0027] In response to a predetermined stop condition being met, there is no second energy value greater than the first energy value, and there is a second energy value equal to the first energy value, and a wake-up decision result is determined according to the historical wake-up weight.

[0028] In a second aspect, an embodiment of the present invention provides a voice wake-up device, the device comprising:

[0029] A speech signal acquisition unit, configured to acquire a target speech signal;

[0030] a first energy value determining unit, configured to determine a first energy value corresponding to the target speech signal, where the first energy value is an energy value of the target speech signal relative to a first device, where the first device is a current device;

[0031] a first wake-up event broadcasting unit, configured to broadcast a first wake-up event to a second device in a network group, where the second device is a device other than the first device in the network group where the current device is located, and the first wake-up event includes at least the first energy value;

[0032] a second wake-up event receiving unit, configured to receive a second wake-up event broadcast by each second device, where the second wake-up event includes at least a second energy value, where the second energy value is an energy value of the target voice signal relative to each second device;

[0033] A decision result acquiring unit is configured to determine a wake-up decision result according to the first energy value and the second energy value, where the wake-up decision result includes wake-up and not wake-up.

[0034] In a third aspect, an embodiment of the present invention provides a voice wake-up system, which includes multiple devices, and the multiple devices are connected to each other through one or more communication methods including Wi-Fi, power line communication PLC and Bluetooth mesh. Each device includes a memory and a processor, and the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0035] In a fourth aspect, an embodiment of the present invention provides an electronic device comprising a memory, a processor, and a communication component, wherein the communication component comprises one or more of a Wi-Fi module, a power line communication PLC module, and a Bluetooth mesh module, and the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in the first aspect.

[0036] In a fifth aspect, an embodiment of the present invention provides a computer program product, which includes a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect above.

[0037] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer program instructions, which implement the method described in the first aspect when executed by a processor.

[0038] The technical solution of the embodiment of the present invention obtains a target voice signal, determines a first energy value of the target voice signal relative to a first device, broadcasts a first wake-up event to a second device in a network group, wherein the second device is a device other than the first device in the network group where the current device is located, and the wake-up event includes at least the first energy value; receives a second wake-up event broadcast by each second device, wherein the second wake-up event includes at least two energy values of the target voice signal relative to each second device; determines a wake-up decision result based on the first energy value and the second energy value, and the wake-up decision result includes wake-up or not wake-up. In this way, the situation where multiple devices are woken up by voice at the same time can be reduced, confusing responses can be reduced, and the accuracy of the device's response to voice and the user experience can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0040] Figure 1 is a schematic diagram of a voice wake-up system according to an embodiment of the present invention;

[0041] Figure 2 is a schematic diagram of functional modules of a device according to an embodiment of the present invention;

[0042] Figure 3 is a circuit diagram of a device according to an embodiment of the present invention;

[0043] Figure 4 is a flow chart of a voice wake-up method according to an embodiment of the present invention;

[0044] Figure 5 is a schematic diagram of network grouping according to an embodiment of the present invention;

[0045] Figure 6 is a flow chart of obtaining a first energy value according to an embodiment of the present invention;

[0046] Figure 7 This is a flowchart of obtaining a wake-up decision result according to an embodiment of the present invention;

[0047] Figure 8This is an interactive diagram of voice wake-up according to an embodiment of the present invention;

[0048] Figure 9 2 is a schematic diagram of a voice wake-up device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present application is described below based on the following embodiments, but the present application is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0050] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0051] Unless the context clearly requires otherwise, words like “include”, “comprising” and the like throughout this application should be interpreted as including rather than exclusive or exhaustive; that is, as meaning “including but not limited to”.

[0052] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0053] Where the solutions described in this specification and in the examples involve the processing of personal information, such processing will be conducted with a legitimate basis (e.g., with the consent of the personal information subject or as necessary for the performance of a contract) and only within the prescribed or agreed scope. A user's refusal to process personal information other than that required for basic functions will not affect the user's use of these basic functions.

[0054] Figure 1 Schematic diagram of the voice wake-up system according to an embodiment of the present invention. Figure 1 As shown, the voice wake-up system of the embodiment of the present invention includes multiple devices. Figure 1 In the description, a voice wake-up system including four devices 1a-1d is taken as an example, wherein each device is communicatively connected with each other.

[0055] The device may be any type of smart device with voice recognition capabilities, such as a smart speaker, a smart central control screen, a smart phone, a tablet computer, a laptop computer, a desktop computer, etc. The smart central control screen may be a central control screen for any smart home, such as a smart lighting system (bulb, switch, dimmer, etc.), a smart appliance (refrigerator, washing machine, etc.), a smart security device (camera, etc.), a smart curtain, a smart socket, etc.

[0056] In some embodiments, multiple devices are connected through one or more of Wi-Fi, power line communication PLC, and Bluetooth mesh. In an embodiment of the present invention, multiple devices are connected simultaneously through three communication methods: Wi-Fi, power line communication PLC, and Bluetooth mesh, which can achieve the dual protection of wireless and wired communication and form a stable and reliable communication network. The unique wake-up requires high real-time message transmission. The Wi-Fi transmission link will be affected by the network environment in which the device is located and the degree of routing congestion. In order to ensure the stability and real-time performance of transmission and reduce the failure rate of unique wake-up, the multi-link transmission of messages is realized through the multi-link fusion strategy of Wi-Fi, PLC, and Bluetooth Mesh, thereby meeting the stable and real-time transmission of messages between devices.

[0057] Wi-Fi (mobile hotspot) is a wireless local area network (WLAN) communication technology that allows devices to exchange data via radio waves without a physical connection. Developed based on the IEEE 802.1 / XMLSchema standard, this technology is designed to provide a convenient and flexible way to access the Internet, suitable for diverse network environments such as homes, offices, and public areas. Wi-Fi allows users to easily connect laptops, smartphones, tablets, and other smart devices to the Internet or share resources with other devices. Its coverage and transmission speeds continue to improve with technological development, providing users with a smoother online experience.

[0058] Power Line Communication (PLC) is a technology that utilizes existing power lines to transmit data and information. By superimposing a carrier signal on standard power lines, PLC enables bidirectional data transmission while simultaneously providing power, eliminating the need for dedicated communication lines. This technology is widely used in smart homes, IoT device interconnection, remote meter reading systems, and other fields, providing users with a convenient network connection solution. PLC's ease of installation, cost-effectiveness, and wide coverage make it particularly suitable for environments where traditional network cabling is difficult to deploy.

[0059] Bluetooth mesh is a wireless communication protocol based on Bluetooth Low Energy technology, designed for large-scale networks of Internet of Things (IoT) devices. It allows devices to form a decentralized network, where each device acts as a signal relay, passing information to other nodes, thereby extending the network's coverage and ensuring reliable transmission of messages to their destination. Bluetooth Mesh is particularly well-suited for applications such as lighting control, sensor networks, and automation systems in smart buildings and smart homes, providing a secure, stable, and flexible connectivity solution. Its unique management and control approach allows users to easily manage and monitor large numbers of Bluetooth Mesh devices using a smartphone or other control device.

[0060] Figure 2 FIG is a schematic diagram of the functional modules of the device according to an embodiment of the present invention. Figure 2 As shown, the device of the embodiment of the present invention is a smart central control screen, which includes a display module F1, a voice module F2, a functional logic module F3, a communication management module F4, and a network transmission module F5. The functions of each module can be implemented by corresponding software programs, hardware modules, or a combination of software and hardware.

[0061] The main function of the display module F1 is to provide a user interaction interface, through which functions such as UI (User Interface Design) display, function switch, user guidance, and status display can be realized.

[0062] UI presentation is about presenting the results of user interface design to users in an intuitive and vivid way. UI design is not just about making the software interface beautiful; more importantly, it builds a bridge of effective communication between users and the software, enabling users to complete various operational tasks efficiently and comfortably.

[0063] Function switch refers to the switch of the unique wake-up function. For example, if you choose to turn off the unique wake-up function for a device, the device will not participate in Figure 4 If you choose to enable the only wake-up function, the device will participate in Figure 4 It should be noted that if you choose to disable the unique wake-up function, the device can execute the relevant policy according to the default setting, where the default setting can be single wake-up or no wake-up.

[0064] User guidance refers to providing relevant instructions for user operations.

[0065] The status display can show the status of the functions currently available on the central control screen. For example, for a lighting central control screen, the on / off status of each lamp can be displayed.

[0066] The voice module F2 is used to implement basic voice functions, such as waking up the engine and voice interaction.

[0067] The wake-up engine is the starting point for voice interaction, allowing users to activate the device by speaking a specific wake-up word, switching it from standby mode to receiving and processing user commands. The wake-up engine accurately identifies the user's wake-up command while minimizing false wake-ups.

[0068] Voice interaction is used to receive and process user voice commands, generate and play voice interaction results.

[0069] The functional logic module F3 is used to implement the main functional logic of the settings, such as energy calculation, wake-up decision, role determination, function tracking, UI display and function control.

[0070] Energy calculation refers to calculating the energy value of a speech signal after receiving the speech signal.

[0071] Wake-up decision refers to deciding whether to wake up the device based on the calculated energy value, so that it can switch from the standby state to the state of receiving and processing user commands.

[0072] Role determination refers to the feature recognition of the received voice signal to determine whether the user who makes the voice is the specified role.

[0073] Function tracking refers to code snippets specifically inserted to analyze user usage or device performance indicators. When a user operation triggers specific functional logic, the tracking code will execute, record the relevant data, and send this data to the server for analysis, so as to provide personalized services to users.

[0074] The UI display provides background data support for the functional implementation of the display module F1.

[0075] Function control is used to realize the control functions of the central control screen, for example, executing instructions to open or close curtains, or executing related voice interaction functions.

[0076] The communication management module F4 is used to implement communication functions, such as broadcast services, device discovery, decision center, heartbeat detection, transmission channels and other functions.

[0077] Broadcast services are used to implement various broadcast functions, such as broadcast connection requests, broadcast wake-up events, etc.

[0078] Device discovery is used to discover other devices, such as detecting connection requests broadcast by other devices.

[0079] The decision center is the component or mechanism responsible for processing, managing, and optimizing communication strategies between local devices.

[0080] Heartbeat detection is a mechanism used to check whether a system, service, or device is operating normally. This mechanism works by periodically sending a signal (the "heartbeat") and waiting for a corresponding confirmation signal. If the expected response is not received within a specified time, the target is considered to have failed or is unavailable.

[0081] Transport channels are used to manage transport channels.

[0082] The network transmission module F5 includes Wi-Fi, Bluetooth mesh and PLC to realize communication functions.

[0083] Figure 3 : is a circuit diagram of a device according to an embodiment of the present invention. Figure 3 As shown, the device according to the embodiment of the present invention includes a processor 11 , a memory 12 , a communication component 13 and a microphone 14 .

[0084] The microphone 14 is used to obtain the target voice signal.

[0085] The communication component 13 receives and sends data under the control of the processor 11 .

[0086] The memory 12 stores instructions that can be executed by at least one processor 11. The instructions are executed by at least one processor 11 to implement the voice wake-up method according to an embodiment of the present invention.

[0087] Furthermore, the communication component 13 includes one or more of a PLC module 131 , a Wi-Fi module 132 and a Bluetooth module 133 .

[0088] Figure 4 Flowchart of the voice wake-up method according to an embodiment of the present invention. Figure 4 As shown, the voice wake-up method of the embodiment of the present invention includes the following steps:

[0089] Step S110: Acquire a target speech signal.

[0090] In this embodiment, a first device acquires a target voice signal. The first device is the current device, that is, the device that executes the voice wake-up method of the embodiment of the present invention.

[0091] Specifically, the network group includes multiple devices, and each device performs the same method steps when voice wakes up. For the sake of convenience of description, the embodiment of the present invention refers to the current device as the first device, and all other devices in the network group except the first device are referred to as second devices.

[0092] Taking the smart central control screen as an example, when connecting a new device, after connecting the new device to the network, you need to select a network group for the new device. At this time, you can select an existing network group or create a new network group. The devices in a group use the same group name, which is used as a broadcast topic and is mainly used to transmit messages or send control instructions between devices in the same group. Among them, the group name can be named according to various application scenarios, and the embodiments of the present invention do not limit this. For example, it can be classified according to function, such as "lighting", "security", "temperature control", etc., so that devices with the same or similar functions can be easily managed in a unified manner. For another example, it can be divided by room or area, such as "living room", "bedroom", "kitchen", etc. This method helps to operate and manage devices in specific areas. For another example, you can also customize scene names, such as "home mode", "away mode", "night mode", etc. This naming method is more suitable for device combinations based on specific usage scenarios.

[0093] Among them, multiple devices are connected through one or more communication methods including Wi-Fi, power line communication PLC and Bluetooth mesh. The embodiment of the present invention is explained by taking the three communication methods of Wi-Fi, power line communication PLC and Bluetooth mesh as an example.

[0094] For Wi-Fi, you first need to have a wireless network (SSID) that has been set up and know the password of the network (if it is an encrypted network). Make sure that all devices that want to connect to the network have built-in Wi-Fi modules and are able to search and connect to Wi-Fi networks. For devices that are connecting to a Wi-Fi network for the first time, start the Wi-Fi function on the device and search for available wireless networks, which can usually be found in the "Settings" or "Network Settings" menu of the device. Select the name (SSID) of the target Wi-Fi network from the search results. If the selected network is encrypted, you need to enter the correct network password to complete the authentication. After entering the correct password, the device will try to establish a connection with the router, the device will successfully connect to the Wi-Fi network, and obtain an IP address. Devices connected to the same Wi-Fi network can communicate with each other.

[0095] For power line communication PLC, each device that needs to be connected to the network needs to be equipped with a PLC modem (also called a PLC module) so that it can send and receive data through the power line. Make sure that all PLC devices are properly connected to the power line and that these devices are in the same electrical loop to ensure good communication quality. When using it for the first time, you may need to perform some basic settings on the PLC device, including selecting a network name (SSID), setting encryption method and password, etc., which is similar to the setup process of a Wi-Fi network. PLC devices support automatic discovery and automatic networking functions. After the devices are powered on and the initialization settings are completed, they will automatically try to search for other PLC devices on the same electrical loop and establish a communication connection. After a successful connection, the necessary network parameters, such as IP address assignment, subnet mask, etc., will be exchanged between the devices to ensure that they can communicate normally within the same local area network.

[0096] For Bluetooth mesh, the process of adding a new device to an existing Bluetooth mesh network is often referred to as "provisioning." First, a provisioner is selected, meaning a device already in the mesh network is chosen as the provisioner. Then, the new device authenticates with the provisioner to ensure a secure connection. If authentication is successful, the provisioner assigns the new device a unique unicast address and provides it with the network key (NetKey), device key (DevKey), and other necessary information to enable the new device to participate in communications within the mesh network.

[0097] Figure 5 FIG is a schematic diagram of a network grouping according to an embodiment of the present invention. Figure 5 As shown, the system includes five devices, 1a to 1e, which are divided into two network groups. Devices 1a, 1b, and 1c are added to network group 2a, while devices 1d and 1e are added to network group 2b.

[0098] exist Figure 5 In the network group shown, if the first device (current device) is 1a, the second devices are 1b and 1c. If the first device (current device) is 1d, the second device is 1e.

[0099] Furthermore, the first device acquires a target voice signal through a microphone, where the target voice signal is a predetermined wake-up voice. The predetermined wake-up voice can be a default wake-up voice of the first device or a voice preset by the user. For example, the predetermined wake-up voice can be a name such as "xx classmate," "xx elf," or "little xx," or a short sentence such as "Hello, xxx," or "Hey xxx."

[0100] Among them, the microphone of the first device continuously checks the audio stream in the environment and pre-processes the captured audio stream, such as noise reduction, echo cancellation, etc., to improve the accuracy of subsequent processing steps. Then, Voice Activity Detection (VAD) is used to determine whether there is human voice in the input audio stream, so as to decide when to start processing possible wake-up words. By using a pre-built wake-up word detection model to identify whether there is a specific wake-up word in the audio stream, it is possible to accurately identify the wake-up word said by the user in various environmental noises and trigger the device to respond. In this embodiment, when a specific wake-up word is detected in the audio stream, it means that the target voice signal is obtained.

[0101] Step S120: Determine a first energy value corresponding to the target speech signal.

[0102] In this embodiment, the first energy value is the energy value of the target speech signal relative to a first device, and the first device is the current device.

[0103] Specifically, sound energy attenuates as it propagates through air, and this attenuation becomes more pronounced as the propagation distance increases. Based on this principle, embodiments of the present invention estimate the distance between the speaker and the device by analyzing the sound energy intensity captured by the microphone. Simply put, the lower the sound energy captured by the microphone, the farther the speaker is likely to be from the device; conversely, the higher the energy captured, the closer the speaker is.

[0104] Figure 6 This is a flow chart of obtaining the first energy value according to an embodiment of the present invention. Figure 6 As shown, determining the first energy value corresponding to the target speech signal includes the following steps:

[0105] Step S121: Acquire an audio sample corresponding to the target speech signal.

[0106] In this embodiment, as described above, the target voice signal is a predetermined wake-up voice. Thus, upon detecting that the captured ambient audio data includes the predetermined wake-up voice, an audio segment corresponding to the wake-up voice is extracted from the complete audio data. For the extracted audio segment, audio samples are extracted from the audio segment at a predetermined sampling rate, where the predetermined sampling rate can be set based on actual conditions, for example, a sampling rate of 22050 Hz.

[0107] Step S122: Perform a square operation on each of the audio samples to obtain a first intermediate value.

[0108] In this embodiment, the extracted audio samples are a series of numbers arranged in time sequence, representing the amplitude of the sound wave, and can be regarded as a discretized representation of the actual sound wave at a certain sampling rate.

[0109] Since the energy of the sound is proportional to the square of its amplitude, the first intermediate value is obtained by performing a square operation on each extracted audio sample. Assume that the value corresponding to the i-th audio sample is x i , then the first intermediate value corresponding to the i-th audio sample is x i 2 Wherein, i = 1, 2, ..., N. N is the total number of audio samples.

[0110] Step S123: performing an average operation on each of the first intermediate values to obtain a second intermediate value.

[0111] In this embodiment, after obtaining the first intermediate value corresponding to each audio sample, an average operation is performed on each of the first intermediate values to obtain a second intermediate value.

[0112] The calculation formula for the second intermediate value is as follows:

[0113]

[0114] Where E is the second intermediate value, N is the total number of audio samples, and x i 2 is the first intermediate value of the i-th audio sample, where i=1, 2, ..., N.

[0115] Step S124: Perform a square root operation on the second intermediate value to obtain the first energy value.

[0116] In this embodiment, a square root operation is performed on the second intermediate value to obtain the first energy value. The calculation formula of the first energy value is as follows:

[0117]

[0118] Where RMS1 is the first energy value, E is the second median value, N is the total number of audio samples, and x i 2 is the first intermediate value of the i-th audio sample, where i=1, 2, ..., N.

[0119] In this way, the first energy value of the target speech signal relative to the first device can be obtained.

[0120] It should be noted that when the first device detects the target voice signal and calculates the first energy value, other second devices in the network group may also be able to detect the target voice signal. When the second device detects the target voice signal, like the first device, each second device will also calculate the energy value of the target voice signal relative to itself, which is referred to as the second energy value in this embodiment of the present invention. This embodiment of the present invention is explained using the example of M devices included in the network group. At this time, there is a first device and M-1 second devices. Assuming that all M-1 second devices have detected the target voice signal, all M-1 second devices will calculate the second energy value of the target voice signal. For ease of explanation, this embodiment of the present invention takes the second energy value of the j-th second device as RMS. 2,j , where j = 2, 3, ..., M.

[0121] Step S130: Broadcast the first wake-up event to the second device in the network group.

[0122] In this embodiment, after calculating the first energy value, the first device broadcasts a first wake-up event to each second device in the network group. The wake-up event includes the first energy value and a unique identifier of the first device. The unique identifier can be a device identifier or information such as a communication address. The second device is a device other than the first device in the network group to which the current device belongs.

[0123] In some embodiments, the first wake-up event also includes the subject of the network group the first device is in. This allows, when there are multiple network groups in the same network, each device only needs to process the wake-up event in its own network group according to the subject in the wake-up event.

[0124] In some embodiments, the wake-up event is broadcast to the second device in the network group via one or more of Wi-Fi, power line communication (PLC), and Bluetooth mesh.

[0125] It should be noted that after the second device calculates the second energy value of the target voice signal, similar to the first device, it will also broadcast a second wake-up event to other devices in the network group. The first wake-up event includes the second energy value and the unique identifier of the second device. In other words, after detecting the target voice signal, each device in the network group (the first device and the second device) will calculate the energy value of the target voice signal relative to its own energy value and broadcast the energy value and unique identifier to other devices.

[0126] Step S140: Receive a second wake-up event broadcast by each of the second devices.

[0127] In this embodiment, the second wake-up event includes at least a second energy value, where the second energy value is an energy value of the target voice signal relative to each of the second devices.

[0128] As described above, after calculating the second energy value of the target voice signal, the second device, similar to the first device, also broadcasts a second wake-up event to other devices in the network group. The first wake-up event includes the second energy value and the unique identifier of the second device. Thus, the first device can receive the second wake-up event sent by each second device.

[0129] Step S150: Determine a wake-up decision result according to the first energy value and the second energy value.

[0130] In this embodiment, the wake-up decision result includes wake-up and not wake-up.

[0131] Specifically, Figure 7 FIG. 1 is a flowchart of obtaining a wake-up decision result according to an embodiment of the present invention. Figure 7 As shown, determining the wake-up decision result according to the first energy value and the second energy value includes the following steps:

[0132] Step S151: Wait for receiving a second wake-up event.

[0133] In this embodiment, after receiving the target voice signal, the first device executes the above steps S110-S130 and waits for receiving a second wake-up event.

[0134] In some embodiments, after the first device receives the target voice signal, it starts timing at a specified time, where the specified time can be set in advance, for example, it can be the time when the target voice signal is received, the time when the first energy value is obtained, the time when the first wake-up event is broadcast, etc.

[0135] Step S152: Receive a second wake-up event.

[0136] In this embodiment, the first device receives a second wake-up event sent by the second device in a network packet.

[0137] Step S153: Detect whether the second energy value is greater than the first energy value.

[0138] In this embodiment, the received second energy value is compared with the first energy value to detect whether the second energy value in the second wake-up event detected and received by the first device is greater than the first energy value.

[0139] In response to the second energy value being greater than the first energy value, step S157 is executed.

[0140] In response to the second energy value being less than the first energy value, step S154 is executed.

[0141] Step S154: Check whether the predetermined waiting time has been reached.

[0142] In this embodiment, the predetermined waiting time T is preset. th , for example, 10 milliseconds, 20 milliseconds, etc. In response to the second energy value being less than the first energy value, it indicates that the energy value of the second device corresponding to the second wake-up event received this time is less than that of the first device, that is, the speaker is closer to the first device, and the time period T from the start of the timing to the current time is detected. S Whether the predetermined waiting time T is reached th .

[0143] If the predetermined waiting time is reached, it indicates that the first device has not detected any device closer than itself within the predetermined time period, and the process proceeds to step S156.

[0144] If the predetermined waiting time has not been reached, proceed to step S155.

[0145] Step S155: Detect whether the number of received second wake-up events reaches a predetermined number.

[0146] In this embodiment, the predetermined number is determined based on the total number of devices in the network group. Generally, when all devices in the network group are working normally, the predetermined number is the total number of devices in the network group minus one, that is, M-1, where M is the total number of devices in the network group. That is, the number N of the second wake-up events currently received is detected. RMS2 Is it equal to M-1? The number of received second wake-up times is the number between the time when the current timing starts and the current time.

[0147] If N RMS2 =M-1, indicating that the first device receives the second wake-up events of all the second devices in the network group, and the process goes to step S156.

[0148] If N RMS2 ≠M-1, indicating that the first device has not yet received the second wake-up events of all the second devices in the network group, and the waiting time has not yet been reached, and the process returns to step S151 to continue waiting for receiving the second wake-up events sent by other second devices.

[0149] Step S156: Determine that the wake-up decision result is wake-up.

[0150] In this embodiment, in response to the second energy value being less than the first energy value, the waiting for receiving the second wake-up event continues until a predetermined stop condition is met; wherein, the predetermined stop condition is that a predetermined waiting time is reached or the number of received second wake-up events meets a predetermined number, and the predetermined number is determined based on the total number of devices in the network group.

[0151] That is, if it is detected in step S154 that the predetermined waiting time has been reached, it means that the first device has not detected a device closer than itself within the predetermined time period, and the wake-up decision result is determined to be wake-up.

[0152] Alternatively, if in step S155, N RMS2 =M-1, it means that the first device receives the second wake-up events of all second devices in the network group, and the device closer than itself determines that the wake-up decision result is wake-up.

[0153] When the wake-up decision result is wake-up, the wake-up process is activated, wherein the wake-up process may be activating the device and playing a predetermined answering voice message, such as voice messages such as "I am here" and "Please speak".

[0154] It should be noted that in step S156 above, when the predetermined stop condition is met, there is no second energy value greater than the first energy value, and there is a second energy value equal to the first energy value, and the wake-up decision result is determined based on the historical wake-up weight. The historical wake-up weight is determined based on the historical response of the device.

[0155] Specifically, the first device obtains the number of responses within a predetermined historical time period and uses the number of responses as a historical response weight. At the same time, each second device obtains the number of responses within a predetermined historical time period and uses the number of responses as a historical response weight. When the first device broadcasts the first wake-up event, it adds its own historical response weight to the first wake-up event. When each second device broadcasts the second wake-up event, it adds its own historical response weight to the second wake-up event. Thus, when the predetermined stop condition is met, the first device detects that there is no second energy value greater than the first energy value, and there is a second energy value equal to the first energy value, and determines the wake-up decision result according to the historical wake-up weight.

[0156] The wake-up decision result is determined based on the historical wake-up weights, including:

[0157] Obtain a second energy value equal to the first energy value, determine a second wake-up event corresponding to the second energy value, and obtain the corresponding historical response weight from the second wake-up event. Compare the historical response weight in the second wake-up event with the first device's own historical response weight. If the historical response weight in the second wake-up event is greater than the first device's own historical response weight, the first device determines that the wake-up judgment result is not wake-up. If the historical response weight in the second wake-up event is smaller than the first device's own historical response weight, the first device determines that the wake-up judgment result is wake-up. In addition, if the historical response weight in the second wake-up event is equal to the first device's own historical response weight, in order to avoid the situation where no device wakes up, the first device determines that the wake-up judgment result is wake-up. At this time, two devices will wake up at the same time.

[0158] Step S157: Determine whether the wake-up decision result is not to wake up.

[0159] In this embodiment, if the second energy value is detected to be greater than the first energy value in step S153, it indicates that a second device closer than the device is found, and the wake-up decision result is determined to be not wake-up. When the wake-up decision result is wake-up, the device is not activated.

[0160] That is, the received second energy value is compared with the first energy value, and in response to the second energy value being greater than the first energy value, the wake-up decision result is determined to be no wake-up. In response to the second energy value being less than the first energy value, the second wake-up event is continued to be waited for until a predetermined wait time is reached, or in response to the second energy value being less than the first energy value, the second wake-up event is continued to be waited for until the number of received second wake-up events meets a predetermined number, where the predetermined number is determined based on the total number of devices in the network group.

[0161] The embodiment of the present invention obtains a target voice signal, determines a first energy value of the target voice signal relative to a first device, broadcasts a first wake-up event to a second device in a network group, wherein the second device is a device other than the first device in the network group where the current device is located, and the wake-up event includes at least the first energy value. The embodiment receives a second wake-up event broadcast by each second device, wherein the second wake-up event includes at least two energy values of the target voice signal relative to each second device. The wake-up decision result is determined based on the first energy value and the second energy value, and the wake-up decision result includes wake-up and non-wake-up. In this way, the situation where multiple devices are woken up by voice at the same time can be reduced, confusing responses can be reduced, and the accuracy of the device's response to voice and the user experience can be improved.

[0162] Figure 8 This is an interactive diagram of voice wake-up in an embodiment of the present invention. Figure 8As shown, an embodiment of the present invention shows an interaction diagram between devices in a network group where each device is in a normal working state, wherein the network group includes three devices 1a-1c, and the specific interaction process between each device is as follows.

[0163] For device 1a, the steps include:

[0164] Step S211: Acquire the target voice signal.

[0165] In this embodiment, when the user needs to interact with the device, he speaks a wake-up word (for example, "Hello, xxx"). When the device 1a detects the presence of a specific wake-up word in the ambient audio stream, it obtains the target voice signal.

[0166] Step S212: Determine the energy value.

[0167] In this embodiment, device 1a extracts an audio segment corresponding to the target speech signal from the complete audio data. Audio samples are extracted from the extracted audio segment at a predetermined sampling rate. The extracted audio samples are a series of numbers arranged in chronological order, representing the amplitude of the sound wave and can be considered a discretized representation of the actual sound wave at a specific sampling rate. A square operation is performed on each extracted audio sample to obtain a first intermediate value. Each of these first intermediate values is averaged to obtain a second intermediate value. A square root operation is performed on the second intermediate value to obtain the energy value.

[0168] Step S213: broadcast a wake-up event.

[0169] Step S214: broadcast a wake-up event.

[0170] For steps S213-S214, for the sake of convenience of explanation, the embodiment of the present invention splits the broadcast wake-up event into two steps, transmitting the wake-up event to different devices respectively. In the actual process, device 1a only needs to broadcast the wake-up event to the network, and other devices in the network can obtain the wake-up event.

[0171] Specifically, after obtaining the energy value of the target voice signal relative to its own, the device 1a broadcasts a wake-up event, which includes information such as the energy value, a unique identifier, and a subject of the network group.

[0172] Meanwhile, devices 1b and 1c execute a similar process to device 1a. Device 1b executes steps S221-S224 to obtain the energy value of the target voice signal relative to its own signal and broadcasts a wake-up event. Device 1c executes steps S231-S234 to obtain the energy value of the target voice signal relative to its own signal and broadcasts a wake-up event.

[0173] Step S215: Compare energy values.

[0174] In this embodiment, after receiving the wake-up events from devices 1b and 1c, device 1a parses the wake-up events to obtain energy values, and compares the energy values of devices 1b and 1c with its own energy value.

[0175] Step S216: Determine not to wake up.

[0176] In this embodiment, device 1a compares the energy values of devices 1b and 1c with its own energy value. Assuming that the energy value of device 1a is not the largest, that is, one of the energy values of devices 1b and 1c is larger than the energy value of device 1a, device 1a determines not to wake up.

[0177] Correspondingly, device 1b executes steps S225-S226. After receiving the wake-up events from device 1a and device 1c, it analyzes the wake-up events to obtain the energy values contained therein and compares the energy values of device 1a and device 1c with its own energy value. Assuming that device 1b has the highest energy value, device 1b determines to wake up.

[0178] Correspondingly, device 1c executes steps S235-S236. After receiving the wake-up events from device 1a and device 1b, it parses the wake-up events to obtain the energy values contained therein and compares the energy values of devices 1a and 1b with its own energy value. Assuming that device 1c's energy value is not the highest—that is, if one of the energy values of devices 1a and 1b is greater than that of device 1c—device 1c determines not to wake up.

[0179] Furthermore, for the device 1b determined to be awakened, the following steps are further included:

[0180] Step S227, respond.

[0181] In this embodiment, after the device 1b determines that it is awake, it initiates a response process to the user, after which the user can interact with the device.

[0182] It should be noted that when multiple network groups are arranged, if the wake-up word for the devices in each network group is the same, at this time, one device in each network group will be awakened. The process of waking up the devices in each network group is as described above, and the embodiments of the present invention will not be repeated here.

[0183] It should also be noted that when there are abnormal devices in the network group, there may be multiple devices awakened in one network group. Figure 8For example, let's assume that device 1c experiences a communication anomaly (it cannot receive wake-up events from other devices, nor can it send wake-up events to other devices). During the wake-up process, devices 1a and 1b determine that device 1b has a higher energy level, and thus wake up. Simultaneously, because device 1c cannot obtain the energy levels of either device 1a or 1b, it wakes up as well. This means that both device 1b and device 1c wake up simultaneously.

[0184] In the existing technology, single-network communication methods such as Wi-Fi and Bluetooth mesh are generally used for networking. However, pure Wi-Fi solutions experience latency exceeding 500ms during network congestion, a high false wakeup rate, and are unable to cross-Wi-Fi networks in multi-router scenarios. Pure Bluetooth mesh solutions have limited coverage (less than 20 meters) and poor wall penetration. Therefore, embodiments of the present invention utilize a three-mode communication network between devices: Wi-Fi, PLC, and Bluetooth Mesh. When multiple devices in the same space are awakened by a wake-up word, each device calculates the wake-up signal strength and simultaneously distributes it through the three-mode network. All other devices in the network receive wake-up events from other devices in real time. Based on the signal strength information from different devices, they dynamically select the appropriate device to respond to the user and engage in subsequent conversational interaction. By utilizing Wi-Fi, PLC, and Bluetooth Mesh, dual wireless and wired communication is achieved, forming a stable and reliable communication network. The multi-link fusion strategy of WiFi, PLC, and Mesh enables multi-link message transmission, ensuring stable and real-time message transmission between devices.

[0185] Furthermore, this embodiment of the present invention leverages decentralization and distributed independent decision-making. Users can group devices, dividing all devices in a household into multiple groups. Each group is considered a decision-making network, enabling device service publishing and subscription. Device networking is accomplished based on user grouping and device type, forming individual decision-making networks. When a device wakes up, it broadcasts the wake-up event. All other devices within the same decision-making network receive the wake-up event, and each device then makes a distributed decision.

[0186] Furthermore, after the device detects the wake-up word, all devices broadcast their own wake-up events (carrying energy values) to the decision-making network in which they are located, and at the same time wait to receive wake-up events broadcast by other devices within the timeout period. Each device makes its own decision. If it finds that a device has a higher energy value than its own, it ends the session. If it does not find a device with a higher energy value than its own, it continues to wait for other devices. This continues until all devices in the decision-making network have made a judgment and only one device remains to respond. If the timeout expires and no wake-up events are received from all devices, it means that the actual number of awakened devices is less than the total number of devices in the group. At this time, only one device responds, and there is a communication problem. At this time, multiple devices may respond.

[0187] The embodiment of the present invention obtains a target voice signal, determines a first energy value of the target voice signal relative to a first device, broadcasts a first wake-up event to a second device in a network group, wherein the second device is a device other than the first device in the network group where the current device is located, and the wake-up event includes at least the first energy value. The embodiment receives a second wake-up event broadcast by each second device, wherein the second wake-up event includes at least two energy values of the target voice signal relative to each second device. The wake-up decision result is determined based on the first energy value and the second energy value, and the wake-up decision result includes wake-up and non-wake-up. In this way, the situation where multiple devices are woken up by voice at the same time can be reduced, confusing responses can be reduced, and the accuracy of the device's response to voice and the user experience can be improved.

[0188] Figure 9 Schematic diagram of a voice wake-up device according to an embodiment of the present invention. Figure 9 As shown, the voice wake-up device of an embodiment of the present invention includes a voice signal acquisition unit 91, a first energy value determination unit 92, a first wake-up event broadcast unit 93, a second wake-up event receiving unit 94, and a decision result acquisition unit 95. The voice signal acquisition unit 91 is configured to acquire a target voice signal. The first energy value determination unit 92 is configured to determine a first energy value corresponding to the target voice signal, where the first energy value is the energy value of the target voice signal relative to a first device, where the first device is the current device. The first wake-up event broadcast unit 93 is configured to broadcast a first wake-up event to a second device in a network group, where the second device is a device other than the first device in the network group where the current device is located. The first wake-up event includes at least the first energy value. The second wake-up event receiving unit 94 is configured to receive a second wake-up event broadcasted by each second device, where the second wake-up event includes at least a second energy value, where the second energy value is the energy value of the target voice signal relative to each second device. The decision result acquisition unit 95 is configured to determine a wake-up decision result based on the first energy value and the second energy value, where the wake-up decision result includes wake-up or not wake-up.

[0189] The embodiment of the present invention obtains a target voice signal, determines a first energy value of the target voice signal relative to a first device, broadcasts a first wake-up event to a second device in a network group, wherein the second device is a device other than the first device in the network group where the current device is located, and the wake-up event includes at least the first energy value. The embodiment receives a second wake-up event broadcast by each second device, wherein the second wake-up event includes at least two energy values of the target voice signal relative to each second device. The wake-up decision result is determined based on the first energy value and the second energy value, and the wake-up decision result includes wake-up and non-wake-up. In this way, the situation where multiple devices are woken up by voice at the same time can be reduced, confusing responses can be reduced, and the accuracy of the device's response to voice and the user experience can be improved.

[0190] exist Figure 3 In the device shown, the device includes at least one processor 11; a memory 12 communicatively connected to at least one processor 11; and a communication component 13 communicatively connected to a scanning device, wherein the communication component 13 receives and sends data under the control of the processor 11; wherein the memory 12 stores instructions that can be executed by at least one processor 11, and the instructions are executed by at least one processor 11 to implement the above-mentioned voice wake-up method.

[0191] Specifically, the device includes: one or more processors 11 and a memory 12, Figure 3 A processor 11 is taken as an example. The processor 11 and the memory 12 may be connected via a bus or other means. Figure 3 In the example above, a bus connection is used. Memory 12, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. Processor 11 executes the non-volatile software programs, instructions, and modules stored in memory 12 to execute various functional applications and data processing of the device, thereby implementing the aforementioned voice wake-up method.

[0192] The memory 12 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store a list of options, etc. In addition, the memory 12 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 12 may optionally include a memory remotely located relative to the processor 11, and these remote memories may be connected to an external device via a network. Examples of the aforementioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0193] One or more modules are stored in the memory 12 and, when executed by one or more processors 11, perform the voice wake-up method in any of the above method embodiments.

[0194] The above-mentioned product can execute the method provided in the embodiment of this application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of this application.

[0195] The embodiment of the present invention obtains a target voice signal, determines a first energy value of the target voice signal relative to a first device, broadcasts a first wake-up event to a second device in a network group, wherein the second device is a device other than the first device in the network group where the current device is located, and the wake-up event includes at least the first energy value. The embodiment receives a second wake-up event broadcast by each second device, wherein the second wake-up event includes at least two energy values of the target voice signal relative to each second device. The wake-up decision result is determined based on the first energy value and the second energy value, and the wake-up decision result includes wake-up and non-wake-up. In this way, the situation where multiple devices are woken up by voice at the same time can be reduced, confusing responses can be reduced, and the accuracy of the device's response to voice and the user experience can be improved.

[0196] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, wherein the computer-readable program is used to enable a computer to execute part or all of the above method embodiments.

[0197] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0198] The foregoing is merely a preferred embodiment of the present application and is not intended to limit the present application. Persons skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application are intended to be within the scope of protection of the present application.

Claims

1. A voice wake-up method, characterized in that: The method comprises: Acquire target speech signal; Determine a first energy value corresponding to the target speech signal, where the first energy value is an energy value of the target speech signal relative to a first device, where the first device is a current device; Broadcasting a first wake-up event to a second device in a network group, where the second device is a device other than the first device in the network group where the current device is located, and the first wake-up event includes at least the first energy value; receiving a second wake-up event broadcast by each second device, where the second wake-up event includes at least a second energy value, where the second energy value is an energy value of the target voice signal relative to each second device; A wake-up decision result is determined according to the first energy value and the second energy value, where the wake-up decision result includes wake-up and not wake-up.

2. The method according to claim 1, characterized in that The target voice signal is a predetermined wake-up voice.

3. The method according to claim 1, characterized in that Determining the first energy value corresponding to the target speech signal includes: Obtaining an audio sample corresponding to the target speech signal; performing a square operation on each of the audio samples to obtain a first intermediate value; performing an averaging operation on each of the first intermediate values to obtain a second intermediate value; A square root operation is performed on the second intermediate value to obtain the first energy value.

4. The method according to claim 1, wherein The first device is connected to the second device in the network group via one or more of Wi-Fi, power line communication (PLC), and Bluetooth mesh.

5. The method according to claim 4, characterized in that Broadcasting the wake-up event to the second device in the network group includes: The wake-up event is broadcast to the second device in the network group via one or more of Wi-Fi, power line communication (PLC), and Bluetooth mesh.

6. The method according to claim 1, characterized in that The first wake-up event also includes a unique identifier of the first device.

7. The method according to claim 1, characterized in that The determining the wake-up decision result according to the first energy value and the second energy value includes: comparing the received second energy value with the first energy value; In response to the second energy value being greater than the first energy value, a wake-up decision result is determined to be not wake-up.

8. The method according to claim 7, characterized in that The determining the wake-up decision result according to the first energy value and the second energy value further includes: In response to the second energy value being less than the first energy value, continuing to wait for receiving a second wake-up event until a predetermined stop condition is satisfied; The predetermined stop condition is that a predetermined waiting time is reached or the number of received second wake-up events meets a predetermined number, and the predetermined number is determined according to the total number of devices in the network group.

9. The method according to claim 8, characterized in that The determining the wake-up decision result according to the first energy value and the second energy value further includes: In response to a predetermined stop condition being met, there is no second energy value greater than the first energy value, and there is a second energy value equal to the first energy value, and a wake-up decision result is determined according to the historical wake-up weight.

10. A voice wake-up device, characterized in that: The device comprises: A speech signal acquisition unit, configured to acquire a target speech signal; a first energy value determining unit, configured to determine a first energy value corresponding to the target speech signal, where the first energy value is an energy value of the target speech signal relative to a first device, where the first device is a current device; a first wake-up event broadcasting unit, configured to broadcast a first wake-up event to a second device in a network group, where the second device is a device other than the first device in the network group where the current device is located, and the first wake-up event includes at least the first energy value; a second wake-up event receiving unit, configured to receive a second wake-up event broadcast by each second device, where the second wake-up event includes at least a second energy value, where the second energy value is an energy value of the target voice signal relative to each second device; A decision result acquiring unit is configured to determine a wake-up decision result according to the first energy value and the second energy value, where the wake-up decision result includes wake-up and not wake-up.

11. A voice wake-up system, characterized in that: The system includes multiple devices, which are connected to each other via one or more of Wi-Fi, power line communication (PLC), and Bluetooth mesh. Each device includes a memory and a processor, and the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 9.

12. An electronic device comprising a memory, a processor and a communication component, characterized in that: The communication component includes one or more of a Wi-Fi module, a power line communication PLC module, and a Bluetooth mesh module, and the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that When the computer program is run on a computer, the computer executes the method according to any one of claims 1 to 9.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: The computer program instructions implement the method according to any one of claims 1 to 9 when executed by a processor.

Images