Multi-device cooperative control method, terminal device, and computer-readable storage medium

Abstract

The application is suitable for the terminal technical field, and particularly relates to a multi-device cooperative control method, a terminal device and a computer readable storage medium. The method can acquire a display picture of a first device and device information of a second device, and can divide a display interface of the first device according to the device information of the second device to obtain each first partition corresponding to the display picture. Then, each second partition corresponding to each first partition in the second device can be determined, and first pixel information corresponding to each first partition can be acquired to cooperatively control each second partition in the second device according to the first pixel information corresponding to each first partition, so that the effect mode of the second device is matched with the display picture of the first device in a partition manner, the display picture of the first device is extended in a partition manner, the display picture achieves the best immersive effect, the viewing effect of the picture is improved, and the user experience is improved.

Description

Technical Field

[0001] This application belongs to the field of terminal technology, and in particular relates to a multi-device collaborative control method, terminal equipment, and computer-readable storage medium. Background Technology

[0002] Multi-device collaboration refers to the coordinated achievement of a specific goal by two or more devices. With the rapid development of smart TVs, smart screens, and other display devices, people have higher demands for display quality. Currently, multi-device collaboration can be achieved by using ambient lights with different effect modes in conjunction with display devices. This allows the ambient light effects to adapt to the currently playing content, thus extending the image on the display and enhancing the viewing experience. However, current multi-device collaboration requires users to manually switch ambient light effect modes, i.e., manually select the appropriate effect mode from the menu, which is inconvenient and results in a poor user experience. Summary of the Invention

[0003] This application provides a multi-device collaborative control method, a terminal device, and a computer-readable storage medium, which can solve the problem that current multi-device collaboration requires manual operation by the user, which is inconvenient and results in a poor user experience.

[0004] In a first aspect, embodiments of this application provide a multi-device collaborative control method, which may include:

[0005] Obtain the display screen of the first device and the device information of the second device;

[0006] Based on the device information of the second device, the display interface of the first device is divided into regions to obtain each first partition corresponding to the display screen;

[0007] Determine the second partition in the second device that corresponds to each of the first partitions;

[0008] Obtain the first pixel information corresponding to each of the first partitions;

[0009] Based on the first pixel information corresponding to each of the first partitions, each of the second partitions in the second device is controlled collaboratively.

[0010] The multi-device collaborative control method described above can acquire the display screen of the first device and the device information of the second device. Based on the device information of the second device, the display interface of the first device can be divided into regions to obtain the corresponding first partitions of the display screen. Subsequently, the corresponding second partitions in the second device can be determined, and the pixel information of each first partition can be acquired. Based on the pixel information of each first partition, collaborative control can be performed on each second partition in the second device, enabling the effect mode of the second device to achieve partition matching with the display screen of the first device. This partition extension of the display screen of the first device achieves optimal immersive effect, improves the viewing experience, and enhances the user experience.

[0011] For example, the device information includes the number of partitions corresponding to the second device, and the step of dividing the display interface of the first device into regions according to the device information of the second device to obtain each first partition corresponding to the display screen may include:

[0012] Obtain the positional relationship between the second device and the first device;

[0013] The target area corresponding to the display interface is determined based on the positional relationship between the second device and the first device;

[0014] The target area is divided according to the number of partitions corresponding to the second device to obtain each first partition corresponding to the target area.

[0015] In the multi-device collaborative control method provided by this scheme, the target area can be determined according to the positional relationship between the second device and the first device, and the target area can be divided according to the number of partitions of the second device, so that the number of the first partitions obtained by the division matches the number of partitions of the second device. Thus, the second device can be partitioned and collaboratively controlled according to the pixel information of each first partition in the target area, thereby improving the collaborative control effect.

[0016] The size of the target area can be determined based on the characteristics of the human eye's visual range. For example, the target areas corresponding to the left and right sides of the display interface can be determined to be larger than the target areas corresponding to the top and bottom of the display interface based on the human eye attention reference model, so that the extension of the ambient light conforms to the characteristics of human eye vision and improves the extension effect of the ambient light.

[0017] Optionally, obtaining the positional relationship between the second device and the first device may include:

[0018] Obtain pairing information of NFC tags at each preset location in the first device;

[0019] The location relationship between the second device and the first device is determined based on the pairing information of the NFC tag.

[0020] In the multi-device collaborative control method provided by this solution, the positional relationship between the second device and the first device can be determined directly based on the pairing information of the NFC tag at a preset location in the first device, without requiring the user to manually input the positional relationship, thus improving the efficiency of positional relationship determination.

[0021] In one possible implementation of the first aspect, dividing the target region according to the number of partitions corresponding to the second device to obtain each of the first partitions corresponding to the target region may include:

[0022] The first number of the first partition is determined based on the number of partitions corresponding to the second device;

[0023] The target region is divided according to the first number of the first partitions to obtain each of the first partitions corresponding to the target region.

[0024] In another possible implementation of the first aspect, dividing the target region according to the number of partitions corresponding to the second device to obtain each of the first partitions corresponding to the target region may include:

[0025] Obtain the second pixel information corresponding to the target region;

[0026] The target region is divided into regions based on the second pixel information corresponding to the target region to obtain the third partition corresponding to the target region;

[0027] The first number of the first partition is determined based on the second number of the third partition and the number of partitions corresponding to the second device;

[0028] The first partitions corresponding to the target region are determined based on the first number of the first partitions.

[0029] In the multi-device collaborative control method provided by this solution, the target area can be divided according to the pixel information of the target area and the number of partitions corresponding to the ambient light. The pixel information can include the color of each pixel. That is, the target area can be partitioned according to the color of each pixel, resulting in a third partition corresponding to the color. The third partition consists of adjacent pixels with the same or similar colors. Then, the first number of the first partition can be determined based on the third number of the third partition and the number of partitions corresponding to the ambient light, and the target area is divided according to the first number of the first partition. For example, when the third number of the third partition is less than or equal to the number of partitions corresponding to the ambient light, the third number of the third partition can be determined as the first number of the first partition; when the third number of the third partition is greater than the number of partitions corresponding to the ambient light, the number of partitions corresponding to the ambient light can be determined as the first number of the first partition. In other words, when the color of the target area is relatively uniform, the number of partitions in the target area can be reduced, thereby reducing the consumption of CPU and other resources; when the color of the target area is more complex, the number of partitions in the target area can be increased to improve the extension effect of the ambient light on the screen.

[0030] Optionally, the first pixel information includes pixel color, and the step of performing coordinated control on each of the second partitions in the second device according to the first pixel information corresponding to each of the first partitions may include:

[0031] The color of each second partition in the second device is determined based on the pixel color corresponding to each first partition;

[0032] Each of the second partitions is controlled collaboratively based on its color.

[0033] In the multi-device collaborative control method provided by this solution, the colors of each second partition in the second device can be collaboratively controlled according to the pixel color of the first partition, so that the color of each second partition matches the pixel color of the corresponding first partition, thereby improving the extension effect of the ambient light.

[0034] For example, the method may further include:

[0035] Obtain the sampling frequency of the displayed image and the refresh frequency of the second device;

[0036] The first pixel information corresponding to each of the first partitions is interpolated based on the sampling frequency of the display interface and the refresh frequency of the second device.

[0037] In the multi-device collaborative control method provided by this scheme, to reduce the CPU and other resource consumption during pixel information extraction, the display image can be acquired at a lower sampling frequency, thus allowing for pixel information extraction at a lower sampling frequency. Then, the extracted pixel information can be interpolated according to the refresh rate of the second device to increase the frequency to match the second device's refresh rate, thereby meeting the control requirements of the ambient light.

[0038] In one possible implementation of the first aspect, the first pixel information includes pixel brightness, and the method may further include:

[0039] The brightness of each second partition is controlled collaboratively based on the pixel brightness corresponding to each first partition.

[0040] In the multi-device collaborative control method provided by this solution, the brightness of each second partition can also be collaboratively controlled according to the pixel brightness of each first partition, so that the brightness of each second partition matches the pixel brightness of the corresponding first partition, thereby ensuring that the second device has a good extension effect on the display screen.

[0041] In another possible implementation of the first aspect, the method may further include:

[0042] The ambient brightness of the environment in which the first device is located is obtained through the ambient light sensor of the first device;

[0043] The brightness of each second partition is controlled collaboratively based on the pixel brightness corresponding to each first partition and the ambient brightness.

[0044] In the multi-device collaborative control method provided by this solution, the brightness of each second zone can also be collaboratively controlled in combination with the ambient brightness of the environment where the first device is located, so that the brightness of each second zone can also adapt to the ambient brightness and improve the extension effect of the second device on the display screen.

[0045] Secondly, embodiments of this application provide a multi-device collaborative control device, which may include:

[0046] The screen acquisition module is used to acquire the display screen of the first device and the device information of the second device.

[0047] The region division module is used to divide the display interface of the first device into regions according to the device information of the second device, so as to obtain each first partition corresponding to the display screen;

[0048] A partition determination module is used to determine the second partitions in the second device that correspond to each of the first partitions;

[0049] A pixel acquisition module is used to acquire the first pixel information corresponding to each of the first partitions;

[0050] The collaborative control module is used to perform collaborative control on each of the second partitions in the second device according to the first pixel information corresponding to each of the first partitions.

[0051] For example, the device information includes the number of partitions corresponding to the second device, and the region division module may include:

[0052] A location relationship acquisition unit is used to acquire the location relationship between the second device and the first device;

[0053] The target area determination unit is used to determine the target area corresponding to the display interface based on the positional relationship between the second device and the first device;

[0054] The region division unit is used to divide the target region according to the number of partitions corresponding to the second device, so as to obtain each of the first partitions corresponding to the target region.

[0055] Optionally, the location relationship acquisition unit may include:

[0056] The pairing information acquisition unit is used to acquire pairing information of NFC tags at each preset location in the first device;

[0057] The location relationship acquisition unit is used to determine the location relationship between the second device and the first device based on the pairing information of the NFC tag.

[0058] In one possible implementation of the second aspect, the region partitioning unit may include:

[0059] The first quantity determination unit is used to determine the first quantity of the first partition based on the number of partitions corresponding to the second device;

[0060] The first region division unit is used to divide the target region according to the first number of the first partitions, so as to obtain each of the first partitions corresponding to the target region.

[0061] In another possible implementation of the second aspect, the region partitioning unit may include:

[0062] An information acquisition subunit is used to acquire the second pixel information corresponding to the target region;

[0063] The second region division unit is used to divide the target region according to the second pixel information corresponding to the target region to obtain the third partition corresponding to the target region.

[0064] The second quantity determination unit is used to determine the first quantity of the first partition based on the second quantity of the third partition and the number of partitions corresponding to the second device;

[0065] The first partition is a sub-unit used to determine each of the first partitions corresponding to the target area based on the first number of the first partitions.

[0066] Optionally, the first pixel information includes pixel color, and the collaborative control module may include:

[0067] A color determination unit is used to determine the color of each second partition in the second device based on the pixel color corresponding to each first partition.

[0068] The collaborative control unit is used to perform collaborative control on each of the second partitions according to the color of each of the second partitions.

[0069] For example, the device may further include:

[0070] A frequency acquisition module is used to acquire the sampling frequency of the displayed screen and the refresh frequency of the second device;

[0071] The interpolation processing module is used to perform interpolation processing on the first pixel information corresponding to each of the first partitions based on the sampling frequency of the display interface and the refresh frequency of the second device.

[0072] In one possible implementation of the second aspect, the first pixel information includes pixel brightness, and the device may further include:

[0073] The first brightness control module is used to coordinately control the brightness of each second partition according to the pixel brightness corresponding to each first partition.

[0074] In another possible implementation of the second aspect, the apparatus may further include:

[0075] A brightness acquisition module is used to acquire the ambient brightness of the environment in which the first device is located through the ambient light sensor of the first device.

[0076] The second brightness control module is used to coordinately control the brightness of each second partition based on the pixel brightness corresponding to each first partition and the ambient brightness.

[0077] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it enables the terminal device to implement the multi-device collaborative control method described in any one of the first aspects above.

[0078] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a computer, enables the computer to implement the multi-device collaborative control method described in any one of the first aspects above.

[0079] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the multi-device collaborative control method described in any one of the first aspects.

[0080] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0081] Figure 1 This is a schematic diagram of the structure of the terminal device to which the multi-device collaborative control method provided in one embodiment of this application is applicable;

[0082] Figure 2 This is a schematic diagram of the software architecture to which the multi-device collaborative control method provided in one embodiment of this application is applicable;

[0083] Figure 3 This is a schematic diagram illustrating an application scenario of the multi-device collaborative control method provided in an embodiment of this application;

[0084] Figure 4 This is an example diagram of a human eye attention reference model in a display interface;

[0085] Figure 5 This is an example diagram of the target area provided in an embodiment of this application;

[0086] Figure 6 This is a schematic diagram of an application scenario for dividing a target region according to an embodiment of this application;

[0087] Figure 7 This is a schematic diagram of a scenario provided in an embodiment of this application;

[0088] Figure 8 This is an illustration of an application scenario where black borders appear in the display interface;

[0089] Figure 9 This is a schematic flowchart of a multi-device collaborative control method provided in an embodiment of this application;

[0090] Figure 10 This is a schematic diagram of the structure of the multi-device collaborative control device provided in the embodiments of this application. Detailed Implementation

[0091] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0092] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0093] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0094] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0095] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0096] Furthermore, the term "multiple" mentioned in the embodiments of this application should be interpreted as two or more.

[0097] The steps involved in the multi-device collaborative control method provided in this application are merely examples. Not all steps are mandatory, nor are all information or message contents required. They can be added or removed as needed during use. The same step or step or message with the same function in this application can be referenced and learned from each other in different embodiments.

[0098] The business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0099] Multi-device collaboration refers to the coordinated achievement of a specific goal by two or more devices. With the rapid development of display devices such as smart screens and smart TVs, people have higher demands for display effects. Currently, multi-device collaboration can be achieved by using ambient lights with different effect modes to adapt the lighting effects to the content being played, thereby extending the screen and enhancing the immersive viewing experience. However, current multi-device collaboration requires users to manually switch the ambient light effect modes, i.e., manually selecting the appropriate mode from the menu, which is inconvenient and results in a poor user experience. Furthermore, current effect modes typically adjust the ambient light as a whole. For example, when displaying a movie, the ambient light effect mode might be set to movie mode (assuming movie mode is a warm color mode), meaning the ambient light is adjusted to a warm color regardless of the movie being displayed. This indiscriminate, overall adjustment may not be compatible with the displayed content, resulting in poor screen extension and significantly impacting the viewing experience.

[0100] To address the aforementioned issues, this application provides a multi-device collaborative control method. This method acquires device information of a second device and divides the display interface of a first device into regions based on this information, obtaining first partitions corresponding to the display screen and acquiring first pixel information corresponding to each first partition. Subsequently, it determines each second partition in the second device that corresponds to each first partition, and determines control parameters for each second partition based on the first pixel information. This allows for collaborative control of each second partition in the second device according to its control parameters, enabling partition matching between the effect mode of the second device and the display screen of the first device. This partition extension of the first device's display screen achieves optimal immersive effects, improves viewing quality, enhances user experience, and demonstrates strong usability and practicality.

[0101] In this embodiment, the first device can be a terminal device with a display screen, such as a mobile phone, tablet computer, wearable device, in-vehicle device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), desktop computer, smart TV, smart screen, projector, etc. This embodiment does not limit the specific type of terminal device. The second device can be a smart light bulb, smart light strip, or other home appliance.

[0102] The following first describes the terminal device involved in the embodiments of this application. Please refer to... Figure 1 , Figure 1 A schematic diagram of a terminal device 100 is shown.

[0103] Terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0104] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0105] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0106] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0107] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0108] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0109] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0110] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0111] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0112] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0113] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to enable the display function of the terminal device 100.

[0114] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0115] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge terminal device 100, and can also be used for data transfer between terminal device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other terminal devices, such as AR devices.

[0116] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the terminal device 100. In other embodiments of this application, the terminal device 100 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0117] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 can also supply power to the terminal device via the power management module 141.

[0118] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0119] The wireless communication function of the terminal device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0120] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0121] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0122] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0123] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0124] In some embodiments, antenna 1 of terminal device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0125] Terminal device 100 implements display functions through a GPU, display screen 194, and application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0126] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0127] Terminal device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0128] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0129] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0130] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when terminal device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0131] Video codecs are used to compress or decompress digital video. Terminal device 100 may support one or more video codecs. Thus, terminal device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0132] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0133] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0134] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of terminal device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of terminal device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0135] Terminal device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0136] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0137] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device 100 can listen to music or make hands-free calls through the speaker 170A.

[0138] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal device 100 answers a phone call or voice message, the receiver 170B can be brought close to the listener's ear to receive the voice message.

[0139] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Terminal device 100 may be equipped with at least one microphone 170C. In some embodiments, terminal device 100 may be equipped with two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 100 may be equipped with three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0140] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0141] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Terminal device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, terminal device 100 detects the intensity of the touch operation based on pressure sensor 180A. Terminal device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0142] The gyroscope sensor 180B can be used to determine the motion attitude of the terminal device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the terminal device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the terminal device 100 through reverse movement, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0143] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0144] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0145] The 180E accelerometer can detect the magnitude of acceleration of the terminal device 100 in various directions (typically three axes). When the terminal device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the attitude of the terminal device, and can be applied to applications such as landscape / portrait switching and pedometers.

[0146] A distance sensor 180F is used to measure distance. The terminal device 100 can measure distance via infrared or laser. In some embodiments, during a shooting scene, the terminal device 100 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0147] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The terminal device 100 emits infrared light outward through the LED. The terminal device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 may use the proximity sensor 180G to detect when a user holds the terminal device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and screen locking.

[0148] The ambient light sensor 180L is used to sense the ambient light intensity. The terminal device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light intensity. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the terminal device 100 is in a pocket to prevent accidental touches.

[0149] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0150] Temperature sensor 180J is used to detect temperature. In some embodiments, terminal device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, terminal device 100 reduces the performance of the processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, terminal device 100 heats battery 142 to prevent abnormal shutdown of terminal device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, terminal device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0151] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of terminal device 100, in a different position than display screen 194.

[0152] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0153] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Terminal device 100 can receive button input and generate key signal inputs related to user settings and function control of terminal device 100.

[0154] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0155] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0156] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the terminal device 100. The terminal device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0157] The software system of terminal device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of terminal device 100.

[0158] Figure 2 This is a software structure block diagram of the terminal device 100 according to an embodiment of this application.

[0159] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0160] The application layer can include a series of application packages.

[0161] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0162] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0163] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0164] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0165] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0166] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0167] The phone manager is used to provide communication functions for terminal device 100. For example, it manages call status (including connection, hang-up, etc.).

[0168] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0169] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating the device, and flashing indicator lights.

[0170] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0171] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0172] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0173] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0174] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0175] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0176] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0177] A 2D graphics engine is a graphics engine for 2D drawing.

[0178] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0179] The multi-device collaborative control method provided in this application embodiment will be described in detail below with reference to the accompanying drawings and specific application scenarios.

[0180] Please see Figure 3 , Figure 3 This diagram illustrates an application scenario of the multi-device collaborative control method provided in an embodiment of this application. For example... Figure 3 As shown, this application scenario may include a first device 301 and a second device 302. The first device 301 can communicate with the second device 302 via wireless communication methods such as Bluetooth and WiFi, or via wired communication methods such as a data cable. The first device 301 can be a terminal device with a display screen, such as a smart TV, smart screen, or projector. The second device 302 can be a home appliance such as an ambient light, such as a smart bulb or smart light strip.

[0181] It should be understood that the positional relationship between the first device 301 and the second device 302 can be determined according to the actual situation. For example, the second device 302 can be set to the left of the first device 301, or to the right of the first device 301, or to the top of the first device 301, or to the bottom of the first device 301, etc. Figure 3 The example shown is that the second device 302 is located on the left and right sides of the first device 301.

[0182] In this embodiment, the ambient lights such as smart bulbs and smart light strips support zoned adjustment of lighting effects. That is, the ambient lights such as smart bulbs and smart light strips can be divided into multiple zones according to specific circumstances, and the lighting effects of each zone can be adjusted separately. Therefore, the lighting effects of each zone of the ambient lights such as smart bulbs and smart light strips can be controlled separately according to the display screen of the first device 301, so that the lighting effects of the ambient lights such as smart bulbs and smart light strips are matched with the display screen of the first device 301 in a zoned manner. This allows the display screen of the first device 301 to be extended in zones through the ambient lights such as smart bulbs and smart light strips, achieving the best immersive effect, improving the viewing experience, and enhancing the user experience.

[0183] It should be understood that the lighting effect of ambient lights such as smart bulbs and smart light strips can be composed of color and / or brightness. For example, ambient lights such as smart bulbs and smart light strips can be LED light groups, and LED lights can be light-emitting modules composed of three primary color LEDs. By adjusting the light intensity of the three primary color LEDs, different colors and brightness of light can be emitted.

[0184] It should be noted that the executing entity of the multi-device collaborative control method provided in this application embodiment can be the first device 301 with a display screen, or a third device such as a cloud server that is communicatively connected to the first device 301, or a second device 302.

[0185] For example, when the first device 301 displays a screen, the first device 301 can obtain the device information of the second device 302, and divide the display screen of the first device 301 into regions according to the device information of the second device 302 to obtain each first partition corresponding to the display screen. Then, the first device 301 can determine each second partition in the second device 302 that corresponds to each first partition. At the same time, the first device 301 can obtain the pixel information (e.g., pixel color, brightness, etc.) of each first partition from the display screen, and determine the control parameters (e.g., color, brightness, etc.) of each second partition according to the pixel information of each first partition, so as to coordinately control the second device 302 according to the control parameters of each second partition, so that the lighting effects of each second partition in the second device 302 match the first partition in the display screen of the first device 301, thereby improving the immersive effect of the screen of the first device 301.

[0186] For example, when the first device 301 displays a screen, a third device such as a cloud server can obtain the device information of the second device 302 and the display screen of the first device 301. Based on the device information of the second device 302, the third device can divide the display interface of the first device 301 into regions, obtaining each first partition corresponding to the display screen. Then, the third device such as the cloud server can determine each second partition in the second device 302 corresponding to each first partition. Simultaneously, the third device such as the cloud server can obtain the pixel information of each first partition based on the display screen of the first device 301, and determine the control parameters of each second partition based on the pixel information of each first partition. This allows for coordinated control of the second device 302 based on the control parameters of each second partition, ensuring that the lighting effects of each second partition in the second device 302 match those of each first partition in the display screen of the first device 301, thereby improving the immersive effect of the first device 301's screen.

[0187] Specifically, the first device 301 / cloud server or other third devices can send control commands to the second device 302. These control commands instruct the second device 302 to coordinately control each second partition according to its control parameters. In other words, after receiving the control commands, the second device 302 can control the color, brightness, etc., of each second partition according to the control parameters contained in the commands. Alternatively, the first device 301 can directly send the control parameters of each second partition to the second device 302. Upon receiving these parameters, the second device 302 can then independently control the color, brightness, etc., of each second partition according to its own parameters.

[0188] For example, when the first device 301 displays a screen, the second device 302 can also acquire the display screen of the first device 301 and divide the display screen of the first device 301 into regions according to its own device information to obtain each first partition corresponding to the display screen. Then, the second device 302 can determine each second partition in the second device 302 that corresponds to each first partition. At the same time, the second device 302 can acquire the pixel information of each first partition from the display screen and determine the control parameters of each second partition according to the pixel information of each first partition. Finally, the second device 302 can control each second partition according to the control parameters of each second partition so that the lighting effects of each second partition in the second device 302 match those of each first partition in the display screen of the first device 301, thereby improving the immersive effect of the screen of the first device 301.

[0189] It should be noted that, in addition to being controlled by multiple devices in a coordinated manner, ambient lighting and other home appliances can also be controlled through the terminal devices corresponding to these appliances. In other words, users can directly adjust and control the lighting effects of ambient lighting and other home appliances through the terminal devices corresponding to these appliances.

[0190] The following example illustrates the multi-device collaborative control method using a first device 301, which is a smart TV, and a second device 302, which is an ambient light.

[0191] In this embodiment, the smart TV can acquire streaming media content through a set-top box, digital television (DTV) antenna, etc., and display the acquired streaming media content on the display interface. Alternatively, the smart TV can directly acquire online live streaming media content such as Internet television (OTT), and display the acquired streaming media content on the display interface. During the process of the smart TV displaying streaming media content, when the smart TV determines that there is an ambient light in the environment where the smart TV is located that is connected to the smart TV, the smart TV can obtain the device information of the ambient light and the current display screen of the smart TV. Then, the smart TV can divide the display screen of the smart TV into regions according to the device information of the ambient light to obtain each first zone corresponding to the display screen. Then, the smart TV can determine each second zone in the ambient light that corresponds to each first zone. Simultaneously, the smart TV can obtain pixel information (such as pixel color and brightness) for each first zone from the displayed screen, and determine the control parameters (such as color and brightness) for each second zone based on the pixel information of each first zone. This allows for coordinated control of the second zones of the ambient light, ensuring that the lighting effects of each second zone match the images of each first zone on the smart TV's display, thereby improving the immersive experience and enhancing the user experience. For example, coordinated control between the smart TV and the ambient light can be achieved through the Hilink platform. Both the smart TV and the ambient light can connect to the Hilink platform. The smart TV can obtain its current display interface and, based on the Hilink protocol, acquire device information for the second device. This allows it to determine the control parameters for each second zone on the second device based on the display interface and the device information, and then coordinately control the second zones based on the Hilink service and these control parameters.

[0192] After acquiring the display image, the smart TV needs to use its Central Processing Unit (CPU) and other resources to extract pixel information from the image, thereby determining the control parameters of the ambient light based on this pixel information. In this embodiment, to reduce the CPU and other resources occupied by pixel information extraction, the smart TV can acquire the display image at a lower sampling frequency, thus allowing for pixel information extraction at a lower sampling frequency. Then, the smart TV can acquire the refresh rate of the ambient light (which can be included in the device information of the ambient light), and interpolate the extracted pixel information based on the ambient light's refresh rate to increase the frequency to match the ambient light's refresh rate, thereby meeting the control requirements of the ambient light. The refresh rate of the ambient light refers to the frequency of the light source in the ambient light, such as 60Hz, 120Hz, etc.

[0193] For example, a smart TV can use a smoothing algorithm to perform interpolation processing based on the most recently extracted pixel information. For instance, when the smart TV's sampling frequency is 30Hz and the ambient light's refresh rate is 60Hz, the smart TV can extract pixel information for each sampled frame of the displayed image, and then use a smoothing algorithm to perform interpolation processing based on the previously obtained pixel information to interpolate the pixel information frequency to 60Hz, thereby meeting the ambient light control requirements. For example, the smart TV can obtain predicted pixel information based on D = 0.1*A + 0.3*B + 0.6*C, where the predicted pixel information is the pixel information to be interpolated, D is the predicted pixel information at time t, A is the pixel information actually extracted at time t-1, B is the predicted pixel information at time t-1, and C is the pixel information actually extracted at time t. It is understood that the foregoing formulas are only illustrative and should not be construed as limiting the embodiments of this application. In the embodiments of this application, the smart TV can predict pixel information using any existing smoothing algorithm.

[0194] It's understandable that dividing the display screen of a smart TV involves dividing the target area of ​​the display screen. The target area of ​​the display interface can be at least one of the left, right, top, and bottom areas. Specifically, the smart TV can determine the target area based on the positional relationship between the ambient light and the TV itself. When the ambient light is on the left side of the smart TV, the target area is the left side; when it's on the right, the right side; when it's at the top, the top; and when it's at the bottom, the bottom.

[0195] For example, the device information of the ambient light may include the positional relationship between the ambient light and the smart TV; that is, the smart TV can determine the positional relationship between the ambient light and the smart TV based on the device information of the ambient light. It should be understood that the embodiments of this application can manage and control the ambient light through an application. Therefore, after placing the ambient light, the user can set the positional relationship between the ambient light and the smart TV in the application according to the placement location of the ambient light. The application can then record the positional relationship between the ambient light and the smart TV set by the user in the device information of the ambient light.

[0196] For example, the positional relationship between the ambient light and the smart TV can also be determined based on the pairing status of NFC tags located at each edge of the smart TV. That is, NFC tags are placed at each edge of the smart TV, and NFC tags can also be placed in the ambient light. When the ambient light is located on the left side of the smart TV, the NFC tag on the left edge of the smart TV will pair and connect with the ambient light; when the ambient light is located on the right side of the smart TV, the NFC tag on the right edge of the smart TV will pair and connect with the ambient light; when the ambient light is located on the top of the smart TV, the NFC tag on the top edge of the smart TV will pair and connect with the ambient light; and when the ambient light is located on the bottom of the smart TV, the NFC tag on the bottom edge of the smart TV will pair and connect with the ambient light. Therefore, the smart TV can determine the positional relationship between the ambient light and the smart TV based on the pairing status of the NFC tags at each edge of the smart TV.

[0197] For example, the positional relationship between the ambient light and the smart TV can also be determined based on user input. That is, after determining that there is an ambient light communicating with the smart TV in its environment, the smart TV can display a window on its own display interface or on the display interface of the terminal device containing the application corresponding to the ambient light. This window can display a prompt message asking the user to input the positional relationship between the ambient light and the smart TV. For example, the smart TV can display a prompt message on its display interface saying "Please enter the positional relationship between the ambient light and the smart TV," and the user can then input the positional relationship based on this prompt. The smart TV can then determine the positional relationship between the ambient light and the smart TV based on the user's input.

[0198] It is understood that the above-described method of determining the positional relationship between the ambient light and the smart TV based on device information, NFC tags, and user input is merely illustrative and should not be construed as a limitation on the embodiments of this application. Of course, other methods can also be used to determine the positional relationship between the ambient light and the smart TV in the embodiments of this application. For example, the smart TV can acquire an image containing both the ambient light and the smart TV through a camera device, and then determine the positional relationship between the ambient light and the smart TV by analyzing the image. For example, both the smart TV and the ambient light have GPS positioning capabilities. The smart TV can obtain the coordinates of the smart TV and the ambient light respectively, and determine the positional relationship between the ambient light and the smart TV based on the coordinates of the smart TV and the ambient light, and so on.

[0199] Please see Figure 4 , Figure 4 An example diagram of a human eye attention reference model in a display interface is shown. Figure 4As shown, research has found that when viewing a display interface, the maximum horizontal visual field of the human eye is the area within 60 degrees to the left and 60 degrees to the right of the standard line of sight, while the maximum vertical visual field is the area within 30 degrees above and 40 degrees below the standard line of sight. That is, when viewing a display interface, the human eye can receive more information horizontally than vertically, and it is easier to obtain detailed information on the left, right, and bottom sides of the display screen, while detail recognition at the top is generally weaker. In this embodiment, the smart TV can determine the size of the target area based on this characteristic of the human eye's visual range. When dividing the target area into zones, it can refine the extension of the ambient lighting effect based on the zones, thereby improving the extension effect of the ambient lighting and enhancing the immersive effect of the picture. For example, the smart TV can determine the size of the target area corresponding to the left and right sides of the display interface > the size of the target area corresponding to the bottom of the display interface > the size of the target area corresponding to the top of the display interface based on the human eye attention reference model. The size of the target area can be determined according to the actual situation. For example, based on empirical research, the target area corresponding to the left and right sides can be determined to be 60 columns of pixels, the target area corresponding to the bottom can be determined to be 30 rows of pixels, the target area corresponding to the top can be determined to be 20 rows of pixels, and so on.

[0200] Please see Figure 5 , Figure 5 An example diagram of the target area provided in an embodiment of this application is shown. Figure 5 As shown in (a), when the target area is the left or right side of the display interface, the smart TV can determine that the target area contains a large number of pixels, for example, it can determine that the target area contains 60 columns of pixels. Figure 5 As shown in (b), when the target area is the bottom area of ​​the display interface, the smart TV can determine that the target area contains fewer pixels, for example, it can determine that the target area contains 30 rows of pixels; when the target area is the top area of ​​the display interface, the smart TV can determine that the target area contains 20 rows of pixels, and so on.

[0201] The target area can include pixels located at the edges of the display interface. This means that the edges of the display interface can be extended using ambient light to visually extend the display. Specifically, when the target area is the left side of the display interface, the 60 columns of pixels represent the 60 columns closest to the left edge; when the target area is the right side, the 60 columns represent the 60 columns closest to the right edge; when the target area is the bottom, the 30 rows of pixels represent the 30 rows closest to the bottom edge; and when the target area is the top, the 20 rows of pixels represent the 20 rows closest to the top edge. It is understood that the 60 columns of pixels, 30 rows of pixels, and 20 rows of pixels mentioned above are only illustrative and should not be construed as limiting the embodiments of this application. In the embodiments of this application, the smart TV can also determine the size of the target area in combination with the actual scene. For example, it can be determined that the target area contains 40 columns of pixels, or it can be determined that the target area contains 25 rows of pixels, or it can be determined that the target area contains 15 rows of pixels, and so on.

[0202] In one possible implementation, the smart TV can also determine the number of pixels contained in the target area based on the resolution of the display interface. Specifically, when the resolution of the display interface is high, the smart TV can determine that the target area contains more pixels, for example, it may contain 70 columns of pixels closest to the left edge of the display interface or 40 rows of pixels closest to the bottom edge of the display interface, and so on. When the resolution of the display interface is low, the smart TV can determine that the target area contains fewer pixels, for example, it may contain 40 columns of pixels closest to the left edge of the display interface or 15 rows of pixels closest to the top edge of the display interface, and so on.

[0203] The process of dividing the target area will be described in detail below.

[0204] In this embodiment, the device information of the ambient light may further include the number of second partitions supported by the ambient light. It should be understood that the number of second partitions supported by the ambient light is an integer greater than or equal to 1. In one possible implementation, the smart TV can divide the target area according to the number of second partitions supported by the ambient light to obtain the first partition corresponding to the target area. For example, the smart TV can determine the first number of first partitions corresponding to the target area based on the number of second partitions supported by the ambient light, and can divide the target area of ​​the display screen according to the first number of first partitions to obtain the first partition corresponding to the target area. Specifically, the smart TV can directly determine the number of second partitions supported by the ambient light as the first number of first partitions, and then divide the target area according to a preset division method to obtain the first number of first partitions, so as to maximize the utilization of the ambient light's partition adjustment capability and improve the extension effect of the ambient light on the display screen. The preset division method can be set by default by the smart TV or customized by the user.

[0205] In this embodiment of the application, the smart TV can also be based on... Figure 4 The human eye attention reference model shown is used to determine the first number of the first partition. For example, when the target area is the top area of ​​the display interface, since the human eye generally has limited ability to recognize details at the top of the display screen, the smart TV can reduce the first number of the first partition to reduce the consumption of CPU and other resources on the smart TV. For instance, the smart TV can determine the first number of the first partition to be any value less than the number of partitions in the second partition.

[0206] It should be noted that when the target area is the left or right side of the display interface, the smart TV divides the target area by height. That is, the sum of the heights of the resulting first partitions equals the height of the target area, and all first partitions have the same width, which is also the width of the target area. When the target area is the top or bottom of the display interface, the smart TV divides the target area by width. That is, the sum of the widths of the resulting first partitions equals the width of the target area, and all first partitions have the same height, which is also the height of the target area.

[0207] For example, a smart TV can default to an equal pixel partitioning method. See also... Figure 6 , Figure 6 This illustration shows an application scenario diagram of the target region division provided by an embodiment of this application. For example... Figure 6 As shown in (a), when the target area is the left or right side of the display interface, the smart TV can divide the target area into multiple (a) using a preset partitioning method. Figure 6Taking four as an example, the first partition with the same height, i.e. Figure 6 A1, A2, A3, and A4 are shown in (a) in the diagram. For example... Figure 6 As shown in (b), when the target area is the top or bottom area of ​​the display interface, the smart TV can divide the target area into four first partitions with the same width using a preset partitioning method. Figure 6 C1, C2, C3 and C4 are shown in (b) in the figure.

[0208] For example, a smart TV can be based on Figure 4 The human eye attention reference model shown uses the default settings for preset partitioning. Specifically, when the target area is the left or right side of the display interface, the smart TV can default to setting the height of the first partition closest to the top and the first partition closest to the bottom to be less than the height of the other first partitions. The height of the other first partitions can be obtained through equal pixel partitioning, meaning the first partition closest to the top and the first partition closest to the bottom have fewer pixels, while the other first partitions have more pixels, to achieve the desired effect. Figure 6 The first partitions are shown in (c) in the diagram. The heights of the first partition closest to the top and the first partition closest to the bottom can be determined based on the actual situation.

[0209] In another possible implementation, the smart TV can divide the target area based on the pixel information of the target area and the number of second partitions supported by the ambient light, thus obtaining a first partition corresponding to the target area. The pixel information can include the color of each pixel. Specifically, the smart TV can partition the target area based on the color of each pixel, obtaining a third partition corresponding to the color. The third partition consists of adjacent pixels with the same or similar colors. Then, the smart TV can determine a first number of the first partition based on the second number of the third partition and the number of second partitions supported by the ambient light, and divide the target area based on the first number of the first partition. For example, when the second number of the third partition is less than or equal to the number of second partitions supported by the ambient light, the smart TV can determine the second number of the third partition as the first number of the first partition; when the second number of the third partition is greater than the number of second partitions supported by the ambient light, the smart TV can determine the number of second partitions supported by the ambient light as the first number of the first partition. In other words, when the target area has a relatively uniform color, a smart TV can reduce the number of partitions in that area, thereby reducing the CPU and other resources required by the smart TV; when the target area has a relatively complex color, a smart TV can increase the number of partitions in that area to improve the effect of ambient lighting on the image.

[0210] It should be understood that when the number of second partitions in the third partition is less than or equal to the number of second partitions supported by the ambient light, the smart TV can designate the third partition as the first partition. When the number of second partitions in the third partition is greater than the number of second partitions supported by the ambient light, the smart TV can merge the third partitions to obtain the required number of third partitions, and can designate each of the merged third partitions as the first partition. For example, the smart TV can merge the third partitions based on their colors, where merging refers to merging adjacent third partitions. The color of the third partition can be determined based on the color of each pixel in the third partition; for example, the average color of all pixels in the third partition can be used to determine the color of that third partition, and so on.

[0211] In this embodiment, the user can also customize the first number and size of the first partition according to actual needs. The smart TV can divide the target area according to the user-defined first number and size of the first partition to obtain the first partition corresponding to the target area. Here, the user-defined settings can be preset or set during the process of extending the ambient light effect.

[0212] In this process, after obtaining the first partition corresponding to the target area, the smart TV can determine the second partition in the ambient light that corresponds to each of the first partitions. Thus, it can control the lighting effects of the corresponding second partitions according to the pixel information of each first partition. This allows the display interface to be extended by partitioning through the lighting effects of each second partition in the ambient light, thereby improving the extension effect of the ambient light on the display screen and enhancing the user experience.

[0213] For example, when the number of first partitions is the same as the number of second partitions supported by the ambient light, and the ambient light is located on the left or right side of the display interface, the smart TV can determine the second partitions corresponding to each first partition according to the following order: the topmost first partition in the target area corresponds to the topmost second partition in the ambient light, and the bottommost first partition in the target area corresponds to the bottommost second partition in the ambient light. For example, please refer to... Figure 7 , Figure 7 This illustration shows a scenario diagram provided by an embodiment of this application. When the target area contains four first zones, and these four first zones are ordered A1, A2, A3, and A4 from top to bottom, while the four second zones in the ambient light are ordered B1, B2, B3, and B4 from top to bottom, as follows: Figure 7As shown in (a), the smart TV can determine the correspondence between A1 and B1, A2 and B2, A3 and B3, and A4 and B4. Therefore, the smart TV can control the lighting effect of B1 based on the pixel information of A1, control the lighting effect of B2 based on the pixel information of A2, control the lighting effect of B3 based on the pixel information of A3, and control the lighting effect of B4 based on the pixel information of A4, resulting in the following... Figure 7 The lighting effect diagram shown in (b) is as follows. Among them, Figure 7 The colors of the second partitions in (b) are for illustrative purposes only and are not actual colors. When the number of first partitions is the same as the number of second partitions supported by the ambient light, and the ambient light is located at the top or bottom of the display interface, the smart TV can determine the corresponding second partitions for each first partition according to the following order: the leftmost first partition in the target area corresponds to the leftmost second partition in the ambient light, and the rightmost first partition in the target area corresponds to the rightmost second partition in the ambient light. For example, when the target area contains four first partitions, and the four first partitions are C1, C2, C3, and C4 in order from left to right, while the four second partitions in the ambient light are D1, D2, D3, and D4 in order from left to right, the smart TV can determine that C1 corresponds to D1, C2 corresponds to D2, C3 corresponds to D3, and C4 corresponds to D4.

[0214] For example, when the number of first partitions is less than the number of second partitions supported by the ambient light, the smart TV can merge the second partitions of the ambient light using a preset merging method. This means merging adjacent second partitions using a preset merging method to adjust the number of second partitions to the same number as the first partitions. Then, the smart TV can determine the second partitions corresponding to each first partition according to the determination method described above. The preset merging method can be specifically determined by the smart TV based on the actual situation, or it can be customized by the user. For example, the smart TV can merge the second partitions based on the size of each first partition, so that a larger first partition corresponds to a larger second partition (a larger second partition means that the second partition has more light-emitting modules). Alternatively, the smart TV can randomly merge adjacent second partitions, and so on.

[0215] Please see Figure 8 , Figure 8 This illustration shows an application scenario where black borders appear in the display interface. For example... Figure 8As shown, when playing a 21:9 aspect ratio movie on a mainstream 16:9 smart TV, black bars appear at the top and / or bottom of the screen. If an ambient light is present at the top and / or bottom of this screen, extending the image beyond these black bars creates a visual abrupt change (i.e., a jump from the displayed image to the black bar, and then from the black bar to the ambient light effect), significantly impacting the user experience. In this embodiment, the smart TV can detect the presence of black bars in the display interface in real time. When a black bar is identified, and an ambient light is present in the target area of ​​the black bar for image extension, the smart TV can turn off the ambient light effect to avoid extending the display image beyond the black bar, reducing the visual abrupt change perceived by the human eye and improving the user experience. Alternatively, the smart TV can turn off the lighting effect of the second zone corresponding to the black bar in the ambient light to reduce the extension of the black bar by the ambient light. For example, when there is a black border at the top of the display interface, and there are ambient lights on the left and top of the display interface to extend the screen, the smart TV can turn off the lighting effect of the top ambient light and turn off the lighting effect of the second zone of the left ambient light corresponding to the black border, thereby reducing the extension of the black border, avoiding the visual effect of screen jumps, and improving the user experience.

[0216] In this embodiment, the smart TV can also acquire the color temperature of the smart TV and the brightness of each first zone in the display screen. It can adjust the color temperature of the ambient light based on the smart TV's color temperature, and adjust the brightness of each second zone in the ambient light based on the brightness of each first zone in the display screen, so that the color temperature of the ambient light matches the color temperature of the smart TV. Simultaneously, it can match the brightness of each second zone in the ambient light with the brightness of each first zone in the display screen, thus avoiding overly bright ambient light and improving the extension effect of the ambient light on the display screen. Furthermore, the smart TV can acquire the ambient brightness through its ambient light sensor and adjust the brightness of each second zone in the ambient light based on the brightness of each first zone and the ambient brightness, so that the brightness of the ambient light matches the ambient brightness and the brightness of the display screen, further improving the extension effect of the ambient light.

[0217] In this embodiment, when the brightness difference between two adjacent second zones in the ambient light exceeds a preset brightness threshold, the smart TV can further set a threshold to filter the boundary between the two adjacent second zones to avoid flickering at the boundary, thereby reducing the unpleasant experience caused by flickering and improving the user experience. The preset brightness threshold can be set according to actual conditions.

[0218] It should be noted that smart TVs can also be configured with a mapping between program modes and ambient light effect modes. When the ambient light needs to extend the display screen, the smart TV can obtain the program mode corresponding to the display screen and, based on the mapping between the program mode and the lighting effect mode, determine the ambient light effect mode, thereby controlling the ambient light to extend the smart TV's display screen according to that lighting effect mode. Specifically, the smart TV can determine the program mode corresponding to the display screen based on its source, and then determine the ambient light effect mode accordingly. For example, when the display screen is transmitted from a game console to the smart TV, the smart TV can determine that the program mode corresponding to the display screen is game mode, and thus determine that the ambient light effect mode is game mode. For example, when the display screen is transmitted from a set-top box or digital TV antenna to the smart TV, the smart TV can identify the channel logo on the display screen to determine the corresponding TV station. Then, the smart TV can determine the program mode corresponding to the display screen based on online electronic program guide (EPG) data. The EPG data can include program information from various TV stations within a certain time period, such as "CCTV5, 9:00-12:30 sports events," "CCTV6, 14:00-16:00 movies," "CCTV1, 19:00-19:30 news broadcast," and so on. That is, when the displayed TV station is CCTV5 and the current time is 9:00-12:30, the smart TV can determine that the program mode is sports mode, and thus determine the ambient lighting effect mode to be sports mode; when the displayed TV station is CCTV6 and the current time is 14:00-16:00, the smart TV can determine that the program mode is movie mode, and thus determine the ambient lighting effect mode to be movie mode; when the displayed TV station is CCTV1 and the current time is 19:00-19:30, the smart TV can determine that the program mode is news mode, and thus determine the ambient lighting effect mode to be news mode, and so on.

[0219] For example, when the display screen is transmitted to a smart TV via an internet TV, the smart TV can determine the program content corresponding to the display screen based on preset program category tags, thereby determining the corresponding program mode. For instance, if the program content is determined to be a sports event, the smart TV can determine the program mode as sports mode; if it's a movie, the smart TV can determine the program mode as movie mode; if it's news, the smart TV can determine the program mode as news mode, and so on.

[0220] Based on the same principle, the multi-device collaborative control provided in this application embodiment can also be applied to auditory multi-device collaborative control. For example, it can be used to collaboratively control a smart TV and an external speaker, so that the sound effect mode of the external speaker matches the content played on the smart TV, improving the playback effect. Specifically, the correspondence between the type of playback content and the sound effect mode of the external speaker can be preset. Therefore, when the smart TV plays content, it can identify the type of the content being played. Then, the smart TV can determine the speaker's sound effect mode (e.g., news sound effect mode, music sound effect mode, movie sound effect mode, etc.) based on the correspondence between the type of playback content and the sound effect mode, and collaboratively control the speaker according to the sound effect mode.

[0221] Based on the above description, the multi-device collaborative control method provided in the embodiments of this application will be briefly described below. Please refer to... Figure 9 , Figure 9 A schematic flowchart of a multi-device collaborative control method provided in an embodiment of this application is shown. Figure 9 As shown, the method may include:

[0222] S901, Obtain the display screen of the first device and the device information of the second device;

[0223] S902. Divide the display interface of the first device into regions according to the device information of the second device to obtain each first partition corresponding to the display screen;

[0224] S903. Determine the second partition in the second device that corresponds to each of the first partitions;

[0225] S904. Obtain the first pixel information corresponding to each of the first partitions;

[0226] S905. Perform coordinated control on each of the second partitions in the second device according to the first pixel information corresponding to each of the first partitions.

[0227] The multi-device collaborative control method provided in this application can acquire the display screen of a first device and the device information of a second device. Based on the device information of the second device, the display interface of the first device can be divided into regions to obtain each first partition corresponding to the display screen. Subsequently, each second partition in the second device corresponding to each first partition can be determined, and the pixel information of each first partition can be acquired. Based on the pixel information of each first partition, collaborative control can be performed on each second partition in the second device, enabling the effect mode of the second device to achieve partition matching with the display screen of the first device. This partition extension of the display screen of the first device achieves optimal immersive effect, improves the viewing experience, and enhances the user experience.

[0228] The device information may include the number of partitions corresponding to the second device, and the first pixel information may include pixel color and brightness, etc. That is, the display interface can be divided according to the number of partitions corresponding to the second device so that the number of first partitions matches the number of partitions corresponding to the second device. Thus, the color and / or brightness of each second partition can be coordinated and controlled according to the first pixel information (i.e., pixel color and brightness) corresponding to each first partition, thereby improving the extension effect of the second device on the display screen.

[0229] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0230] Corresponding to the multi-device collaborative control method described in the above embodiments, Figure 10 A structural block diagram of a multi-device collaborative control device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0231] Reference Figure 10 The device may include:

[0232] The screen acquisition module 1001 is used to acquire the display screen of the first device and the device information of the second device;

[0233] The region division module 1002 is used to divide the display interface of the first device into regions according to the device information of the second device, so as to obtain each first partition corresponding to the display screen.

[0234] The partition determination module 1003 is used to determine the second partitions in the second device that correspond to each of the first partitions;

[0235] The pixel acquisition module 1004 is used to acquire the first pixel information corresponding to each of the first partitions;

[0236] The collaborative control module 1005 is used to perform collaborative control on each of the second partitions in the second device according to the first pixel information corresponding to each of the first partitions.

[0237] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0238] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0239] This application also provides a terminal device, which includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, it causes the terminal device to implement the steps in any of the above-described method embodiments. For example, the structure of the terminal device can be as follows: Figure 1 As shown.

[0240] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a computer, causes the computer to perform the steps in any of the above method embodiments.

[0241] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps in any of the above method embodiments.

[0242] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable storage media cannot be electrical carrier signals or telecommunication signals.

[0243] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0244] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0245] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0246] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0247] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method of multi-device cooperative control, the method comprising: include: Acquire the display screen of the first device and the device information of the second device; the device information includes the number of partitions corresponding to the second device. Based on the number of partitions corresponding to the second device, the target area of ​​the display screen of the first device is divided into regions to obtain each first partition corresponding to the target area of ​​the display screen. Determine the second partition in the second device that corresponds to each of the first partitions; Obtain the first pixel information corresponding to each of the first partitions; Based on the first pixel information corresponding to each of the first partitions, each of the second partitions in the second device is controlled collaboratively. The step of dividing the target area of ​​the display screen of the first device according to the number of partitions corresponding to the second device to obtain each first partition corresponding to the target area of ​​the display screen includes: Obtain the second pixel information corresponding to the target region; The target region is divided into regions based on the second pixel information corresponding to the target region to obtain the third partition corresponding to the target region; The first number of the first partition is determined based on the second number of the third partition and the number of partitions corresponding to the second device; The first partitions corresponding to the target area are determined based on the first number of the first partitions and the third partition.

2. The method of claim 1, wherein, The step of dividing the target area of ​​the display screen of the first device according to the number of partitions corresponding to the second device to obtain each first partition corresponding to the target area of ​​the display screen further includes: Obtain the positional relationship between the second device and the first device; The target area corresponding to the display screen is determined based on the positional relationship between the second device and the first device; The target area is divided according to the number of partitions corresponding to the second device to obtain each first partition corresponding to the target area.

3. The method of claim 2, wherein, The step of obtaining the positional relationship between the second device and the first device includes: Obtain pairing information of NFC tags at each preset location in the first device; The location relationship between the second device and the first device is determined based on the pairing information of the NFC tag.

4. The method according to any one of claims 1 to 3, characterized in that, The first pixel information includes pixel color, and the step of coordinating control of each second partition in the second device according to the first pixel information corresponding to each first partition includes: The color of each second partition in the second device is determined based on the pixel color corresponding to each first partition; Each of the second partitions is controlled collaboratively based on its color.

5. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Obtain the sampling frequency of the displayed image and the refresh frequency of the second device; The first pixel information corresponding to each of the first partitions is interpolated based on the sampling frequency of the displayed screen and the refresh frequency of the second device.

6. The method according to any one of claims 1 to 3, characterized in that, The first pixel information includes pixel brightness, and the method further includes: The brightness of each second partition is controlled collaboratively based on the pixel brightness corresponding to each first partition.

7. The method according to claim 6, characterized in that, The method further includes: The ambient brightness of the environment in which the first device is located is obtained through the ambient light sensor of the first device; The brightness of each second partition is controlled collaboratively based on the pixel brightness corresponding to each first partition and the ambient brightness.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the terminal device to implement the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the computer, it causes the computer to implement the method as described in any one of claims 1 to 7.

Images