Communication method, storage medium and terminal equipment

Abstract

The invention relates to the technical field of network communication, and discloses a communication method, a storage medium and terminal equipment, and the method comprises the following steps: in real-time communication, a terminal obtains equipment parameters of the terminal in real time, determines the parameter change state of the equipment parameters, and determines the scene type of the current environment of the terminal based on the parameter change state. And then, the terminal predicts the network state of the current environment where the terminal is located based on the scene type, and adjusts the transmission bandwidth of the terminal based on the predicted network state, so that the terminal performs data transmission of real-time communication through the transmission bandwidth matched with the network state of the current environment where the terminal is located. Thus, the terminal does not need to wait for feedback information of another terminal, the network state of the current environment where the terminal is located can be predicted directly through the device parameters of the terminal, and it is ensured that the terminal can adjust the transmission bandwidth of data transmission in real-time communication in time along with switching of the network state.

Description

Technical Field

[0001] The present invention relates to the field of network communication technology, and in particular to a communication method, a storage medium and a terminal device. Background Art

[0002] Terminal devices can establish real-time communications such as voice calls and video calls with other terminal devices through the network. During the process of establishing real-time communication, the terminal device will adjust the data transmission bandwidth during the real-time communication process accordingly as the network status switches. For example, when a user uses a terminal device such as a mobile phone to make a video call with a friend through the friend's mobile phone, the user carries the mobile phone from a strong network environment such as outside a subway car to a weak network environment such as inside a subway car. The network status of the user's mobile phone may switch from a high-quality network state (such as a higher network transmission rate) to a low-quality network state (such as a lower network transmission rate). The user's mobile phone should then adjust from the wider transmission bandwidth (such as a higher video frame rate) corresponding to the high-quality network state to the narrower transmission bandwidth (such as a lower video frame rate) corresponding to the low-quality network state, so that the user's mobile phone and the friend's mobile phone can transmit data at a lower video frame rate, ensuring the quality of the video call.

[0003] However, when a user's phone moves from a strong network environment to a weak network environment, for example, the user's phone may not be able to detect the network status change in time, making it unable to adjust the transmission bandwidth for real-time communication data transmission accordingly. As a result, the user's phone may still transmit data to a friend's phone at a high video frame rate even when the network transmission speed is low. This may cause data delays and data loss during the data transmission process, resulting in video call lag and interruptions, affecting the user experience. Summary of the Invention

[0004] Embodiments of the present invention provide a communication method, a storage medium, and a terminal device.

[0005] In a first aspect, the present invention provides a communication method, applied to a first device, wherein the first device establishes a real-time communication connection with a second device, the method comprising: determining a parameter change state of the first device from a first device parameter to a second device parameter; based on the parameter change state, determining that the first device changes from a first scenario corresponding to the first device parameter to a second scenario corresponding to the second device parameter; adjusting the transmission bandwidth of the first device from a first bandwidth corresponding to the first scenario to a second bandwidth corresponding to the second scenario; and the first device transmitting data to the second device based on the second bandwidth.

[0006] In an embodiment of the present invention, during the process of establishing a real-time communication connection between a first device and a second device, the first device obtains the device parameters of the first device itself in real time, determines the parameter change state of the device parameters, and determines the scenario type (second scenario) of the environment in which the first device is currently located based on the parameter change state. Subsequently, the first device predicts the network status of the environment in which the first device is currently located based on the scenario type, and adjusts the transmission bandwidth of the first device based on the predicted network status, so that the first device transmits data for real-time communication through the transmission bandwidth (second bandwidth) matched by the network status of the current environment. In this way, the first device does not need to wait for feedback information from the second device to determine the network status of the first device, and can directly predict the network status of the environment in which the first device is currently located through the device parameters of the first device itself, ensuring that the first device can adjust the transmission bandwidth of data transmission in real-time communication in a timely manner as the network status switches.

[0007] In a possible implementation of the first aspect above, the above-mentioned determination of the change of the first device from the first scene corresponding to the first device parameters to the second scene corresponding to the second device parameters based on the parameter change state includes: inputting the parameter change state into a preset target model, wherein the target model is determined based on the motion parameters, environmental parameters and communication parameters of the first device as training samples, the motion parameters are used to represent the motion state of the first device, the environmental parameters are used to represent the environment and position of the first device, and the communication parameters are used to represent the communication state of the first device; based on the output result of the target model, determining the first scene type of the first scene corresponding to the first device parameters and the second scene type of the second scene corresponding to the second device parameters; based on the first scene type and the second scene type, determining that the first device changes from the first scene to the second scene.

[0008] In a possible implementation of the first aspect above, after determining that the first device changes from the first scene to the second scene, it also includes: the first device sends a notification message to the second device, where the notification message is used to indicate that the first device changes from the first scene type to the second scene type.

[0009] In a possible implementation of the first aspect above, the notification message includes type information of the second scenario type, the type information is used by the second device to determine the second bandwidth, and the notification message is used to instruct the second device to transmit data to the first device based on the second bandwidth.

[0010] In a possible implementation of the first aspect above, the motion parameters include one or more of the following: acceleration sensor parameters, magnetic sensor parameters, gyroscope sensor parameters, direction sensor parameters, and gravity sensor parameters; the environmental parameters include one or more of the following: ambient light sensor parameters, latitude and longitude parameters, and altitude parameters; and the communication parameters include one or more of the following: cellular cell parameters and WIFI signal parameters.

[0011] In a possible implementation of the first aspect above, the first scene type includes any one of the following: subway platform, subway car, high-speed rail platform, high-speed rail car, low-speed driving, and high-speed driving; the second scene type includes any one of the following: subway platform, subway car, high-speed rail platform, high-speed rail car, low-speed driving, and high-speed driving.

[0012] In a possible implementation of the first aspect, when the network transmission rate of the first scenario is greater than the network transmission rate of the second scenario, the first bandwidth is greater than the second bandwidth; or, when the network transmission rate of the first scenario is less than the network transmission rate of the second scenario, the first bandwidth is less than the second bandwidth.

[0013] In a second aspect, the present invention provides a communication method, applied to a first device, wherein the first device establishes a real-time communication connection with a second device, and the method includes: determining the second device parameters of the first device in a second scene; inputting the second device parameters into a preset target model, wherein the target model is determined based on the motion parameters, environmental parameters and communication parameters of the first device as training samples, the motion parameters are used to represent the motion state of the first device, the environmental parameters are used to represent the environment and position of the first device, and the communication parameters are used to represent the communication state of the first device; based on the output result of the target model, determining the second scene type of the second scene; based on the second scene type, correspondingly setting the transmission bandwidth of the first device; the first device transmits data to the second device based on the set transmission bandwidth.

[0014] In a third aspect, the present invention provides a communication method, applied to a second device, wherein the second device establishes a real-time communication connection with a first device, the method comprising: receiving a notification message sent by the first device, wherein the notification message is used to indicate that the first device changes from a first scene type corresponding to the first scene to a second scene type corresponding to the second scene; based on the second scene type, performing corresponding setting on the transmission bandwidth of the second device; and the second device transmits data to the first device based on the set transmission bandwidth.

[0015] In a fourth aspect, a communication method is applied to a first device and a second device, wherein the first device establishes a real-time communication connection with the second device, and the method includes: the first device determines a parameter change state of the first device from a first device parameter to a second device parameter; the first device inputs the parameter change state into a preset target model, wherein the target model is determined based on motion parameters, environmental parameters and communication parameters of the first device as training samples, the motion parameters are used to represent the motion state of the first device, the environmental parameters are used to represent the environment and position of the first device, and the communication parameters are used to represent the communication state of the first device; the first device determines a first scene corresponding to the first device parameter based on an output result of the target model. The first device determines that the first device changes from the first scene to the second scene based on the first scene type and the second scene type; the first device adjusts the transmission bandwidth of the first device from the first bandwidth corresponding to the first scene to the second bandwidth corresponding to the second scene, and the first device transmits data to the second device based on the second bandwidth; the first device sends a notification message to the second device, where the notification message is used to indicate that the first device changes from the first scene type to the second scene type; the second device receives the communication information; the second device sets the transmission bandwidth of the second device to the second bandwidth based on the second scene type, and the second device transmits data to the first device based on the second bandwidth.

[0016] In the implementation of this application, after the first device determines the scene type of the environment in which the first device is currently located, the first device can send the type information of the determined scene type to the second device in the form of a notification message, so that the second device can predict the network status of the environment in which the first device is currently located. Then, the second device can timely adjust the transmission bandwidth of the second device's data transmission based on the predicted network status of the first device. In this way, when the second device transmits data as a sender to the receiver of the first device, the second device can perform real-time communication data transmission based on the transmission bandwidth matched by the network status of the environment in which the first device is currently located, thereby ensuring the quality of real-time communication.

[0017] In the fifth aspect, an embodiment of the present invention provides a readable storage medium, which stores instructions. When the instructions are executed on a terminal device, the terminal device implements any one of the communication methods provided by the first to fourth aspects and various possible implementations of the first to fourth aspects.

[0018] In the sixth aspect, an embodiment of the present invention provides a terminal device, which includes: a memory for storing instructions executed by one or more processors of the terminal device; and a processor, which is one of the processors of the terminal device, for executing the instructions stored in the memory to implement any one of the communication methods provided by the above-mentioned first to fourth aspects and various possible implementations of the above-mentioned first to fourth aspects.

[0019] In the seventh aspect, an embodiment of the present invention provides a computer program product, which includes instructions, and when the instructions are executed by a terminal device, the terminal device can implement any one of the communication methods provided by the first to fourth aspects and various possible implementations of the first to fourth aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 According to some embodiments of the present invention, a schematic diagram of a video call real-time communication scenario is shown;

[0021] Figure 2 According to some embodiments of the present invention, a schematic diagram of some device parameters is shown;

[0022] Figure 3A According to some embodiments of the present invention, a schematic diagram of training a target model is shown;

[0023] Figure 3B According to some embodiments of the present invention, some schematic diagrams of scene division are shown;

[0024] Figure 4 According to some embodiments of the present invention, a schematic diagram of a decision-making process of a mobile phone is shown;

[0025] Figure 5A According to some embodiments of the present invention, a flow chart of a communication method is shown;

[0026] Figure 5B According to some embodiments of the present invention, a schematic diagram of a mobile phone interface is shown;

[0027] Figure 5C According to some embodiments of the present invention, another schematic diagram of a mobile phone interface is shown;

[0028] Figure 6 According to some embodiments of the present invention, an architecture diagram of a mobile phone is shown;

[0029] Figure 7 According to some embodiments of the present invention, a structural diagram of a mobile phone is shown. DETAILED DESCRIPTION

[0030] Illustrative embodiments of the present invention include, but are not limited to, a communication method, a storage medium, and a terminal device.

[0031] The following is combined with Figures 1 to 7 The technical solution of the present invention is introduced.

[0032] When using a mobile phone or other terminal device, a user can make a video call with a friend who is also using a mobile phone through the network. During the video call, the user and the friend can see each other's real-time video through the display of each other's mobile phone, realizing "face-to-face" communication. For example, Figure 1 As shown, user A initiates a video call request to user B's mobile phone 20 through the video call function of mobile phone 10. When user B accepts the video call request through mobile phone 20, user A and user B can conduct a video call through mobile phone 10 and mobile phone 20 respectively, realizing "face-to-face" communication.

[0033] In some embodiments, Figure 1 In the video call between user A and user B shown in the figure, user A is in a subway station preparing to board the subway. Subsequently, when user A enters the subway car from the subway platform (i.e., outside the subway car) carrying mobile phone 10, the network status of mobile phone 10 may switch from a high-quality network state to a low-quality network state. When switching from a high-quality network state to a low-quality network state, mobile phone 10 should adjust the transmission bandwidth from the wider transmission bandwidth corresponding to the high-quality network state to the narrower transmission bandwidth corresponding to the low-quality network state. This allows mobile phone 10 to transmit data with mobile phone 20 using the narrower transmission bandwidth in the low-quality network state, ensuring the quality of the video call.

[0034] However, when user A enters the subway car from the subway platform carrying mobile phone 10, mobile phone 10 may not be able to detect the network status change in time, making it unable to adjust the transmission bandwidth for data transmission during the video call in response to the network status change. As a result, mobile phone 10 continues to transmit data to mobile phone 20 using a wider transmission bandwidth even in a low-quality network state, which may cause data delays and data loss during the data transmission process, resulting in video call freezes and interruptions, affecting the user experience.

[0035] Understandably, Figure 1 As shown, during the process of establishing a video call between mobile phone 10 and mobile phone 20, when mobile phone 10 transmits data to mobile phone 20, for example, when mobile phone 10 sends a video stream to mobile phone 20, mobile phone 10 acts as a transmitter and mobile phone 20 acts as a receiver; when mobile phone 20 transmits data to mobile phone 10, for example, when mobile phone 20 sends a video stream to mobile phone 10, mobile phone 20 acts as a transmitter and mobile phone 10 acts as a receiver.

[0036] In the following, mobile phone 10 is used as the sending end and mobile phone 20 is used as the receiving end. Figure 1 The real-time communication scenario example of the video call is shown to illustrate some embodiments of the present invention.

[0037] In some embodiments, during a video call between mobile phone 10 and mobile phone 20, mobile phone 10 uses its camera to capture multiple frames of real-time images of the environment surrounding mobile phone 10, forming a video stream. Mobile phone 10 encodes the video stream and sends the encoded video stream to mobile phone 20. Subsequently, after receiving the encoded video stream, mobile phone 20 decodes the encoded video stream to obtain multiple frames of real-time images captured by the camera of mobile phone 10, and displays the multiple frames of real-time images on the display screen. Thus, during the video call, the display screen of mobile phone 20 can display the multiple frames of real-time images captured by the camera of mobile phone 10, forming a video image, allowing users A and B to communicate "face to face" through mobile phones 10 and 20.

[0038] In some embodiments, upon receiving a coded video stream from mobile phone 10, mobile phone 20 may send feedback information regarding the coded video stream reception status to mobile phone 10. For example, mobile phone 20 may send feedback information such as the coded video stream reception response time and received data size to mobile phone 10 via a receive report (RR). After receiving the feedback information, mobile phone 10 uses the feedback information to evaluate the network status of mobile phone 10 and adjusts the frame rate or bit rate of the coded video stream transmitted from mobile phone 10 to mobile phone 20 based on the evaluation result of the network status. For example, if mobile phone 10 determines based on the feedback information that the network status evaluation result is a high-quality network status, mobile phone 10 may increase the frame rate of the coded video stream transmitted to mobile phone 20 to improve the clarity of the video image during the video call. If mobile phone 10 determines based on the feedback information that the network status evaluation result is a low-quality network status, mobile phone 10 may reduce the frame rate of the coded video stream transmitted to mobile phone 20 to ensure the smoothness of the video image during the video call.

[0039] It is understood that the process of mobile phone 10 sending the encoded video stream to mobile phone 20, and mobile phone 20 receiving the encoded video stream and sending feedback information to mobile phone 10 requires at least one round trip time (RTT). In other words, mobile phone 10 needs at least one RTT to receive the feedback information from mobile phone 20, that is, the feedback information received by mobile phone 10 is not instant information.

[0040] As previously mentioned, when user A, carrying mobile phone 10, enters a subway car from a subway platform, the network status of mobile phone 10 may switch from a high-quality network to a low-quality network. In a low-quality network, the RTT may increase, making it more difficult for mobile phone 10 to receive timely feedback from mobile phone 20 and promptly detect the network status change. As a result, mobile phone 10 cannot adjust the bitrate or frame rate of the encoded video stream transmitted to mobile phone 10 in real time according to the real-time changes in network status during a video call, affecting the quality of the video call.

[0041] It is understood that the above scenario of entering a subway car from a subway platform is merely an example scenario of entering a weak network environment with a low-quality network state from a strong network environment with a high-quality network state. In other embodiments, the scenario of entering a high-speed rail car from a high-speed rail platform, changing from a low-speed driving to a high-speed driving, and any other scenario of changing from a strong network environment to a weak network environment may also be included, without specific limitation. In other embodiments, the scenario of entering a strong network environment with a high-quality network state from a weak network environment may also be included, such as entering a subway platform from a subway car, entering a high-speed rail platform from a high-speed rail car, changing from a high-speed driving to a low-speed driving, and other scenarios, without specific limitation.

[0042] I understand. Figure 1 The video call scenario shown is only an example of a real-time communication scenario. In other embodiments, it may also include any scenario that requires real-time data transmission, such as voice calls, real-time location sharing, real-time game interaction, etc., without specific limitation.

[0043] To this end, the present invention proposes a communication method in which, during the process of establishing real-time communication, mobile phone 10 obtains its own device parameters in real time, determines the parameter change status of the device parameters, and determines the environmental change status of the environment in which mobile phone 10 is located based on the parameter change status. Subsequently, mobile phone 10 determines the scene type of the environment in which mobile phone 10 is currently located based on the environmental change status, and predicts the network status of the environment in which mobile phone 10 is currently located based on the determined scene type. Based on the predicted network status, mobile phone 10 adjusts the transmission bandwidth of mobile phone 10 so that mobile phone 10 can transmit data for real-time communication using the transmission bandwidth that matches the network status of the environment in which it is currently located, thereby ensuring the quality of real-time communication. In this way, during the real-time communication process, mobile phone 10 does not need to wait for feedback information from the other end device (such as mobile phone 20) to determine the network status of mobile phone 10. Instead, the network status of mobile phone 10 can be predicted directly based on its own device parameters. This ensures that when the environment in which mobile phone 10 is currently located changes and switches to a different network state, mobile phone 10 can promptly adjust the transmission bandwidth for data transmission in real-time communication as the network state switches, thereby improving the quality of real-time communication.

[0044] For example, when mobile phone 10 changes from a subway platform environment to a subway car environment, mobile phone 10 can determine that it has changed from a subway platform environment to a subway car environment based on the parameter change status of mobile phone 10's own device parameters. Mobile phone 10 can then predict the network status of the subway car environment, allowing mobile phone 10 to transmit data for real-time communication using a transmission bandwidth that matches the network status of the subway car environment currently in which mobile phone 10 is located, thereby ensuring the communication quality of mobile phone 10 during real-time communication.

[0045] In some embodiments, after mobile phone 10 determines the scene type of its current environment, mobile phone 10 may also send information about the determined scene type to mobile phone 20, so that mobile phone 20 can predict the network status of its current environment. Mobile phone 20 can then adjust the transmission bandwidth for its data transmission based on the predicted network status of mobile phone 10. In this way, when mobile phone 20, acting as a transmitter, transmits data to a receiver of mobile phone 10, mobile phone 20 can perform real-time communication data transmission based on the transmission bandwidth that matches the network status of its current environment, thereby ensuring the quality of real-time communication.

[0046] It can be understood that in order to ensure the real-time communication quality between mobile phone 10 and mobile phone 20, real-time communication data transmission is usually carried out with a transmission bandwidth that matches the network status of the environment where the device with poor network status is located. For example, when the network transmission rate of the environment where mobile phone 10 is located is lower than the network transmission rate of the environment where mobile phone 20 is located, mobile phone 10 and mobile phone 20 usually carry out real-time communication data transmission with the transmission bandwidth that matches the network status of the environment where mobile phone 10 is located to ensure the quality of data transmission between mobile phone 10 and mobile phone 20.

[0047] It can be understood that the above-mentioned numbering of mobile phone 10 or mobile phone 20 does not limit the functions of mobile phone 10 or mobile phone 20. In other embodiments, mobile phone 10 can be replaced by mobile phone 20, or mobile phone 20 can be replaced by mobile phone 10, and the functions of mobile phone 10 and mobile phone 20 remain unchanged.

[0048] It can be understood that the above-mentioned mobile phone 10 and mobile phone 20 are only examples of terminal devices at both ends in a real-time communication scenario. In other embodiments, the terminal device may include a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) or a dedicated camera (such as a SLR camera, a card camera), etc., and the terminal devices at both ends in the real-time communication scenario can be the same type of device, such as mobile phone 10 and mobile phone 20, or can be different types of devices, without specific limitation.

[0049] Some embodiments of the present invention are explained below using a mobile phone as an example of a terminal device.

[0050] In some embodiments, after obtaining the device parameters of the mobile phone, the mobile phone can input the obtained device parameters into a target model (such as a convolutional neural network model, a recurrent neural network model, etc.) pre-deployed in the mobile phone, and determine the scene type corresponding to the device parameters obtained by the mobile phone through the output results of the target model, thereby determining the scene type of the environment in which the mobile phone is located when obtaining the device parameters. For example, Figure 2 As shown, the device parameters of a mobile phone may include motion parameters such as acceleration sensor parameters, magnetic sensor parameters, gyroscope sensor parameters, orientation sensor parameters, and gravity sensor parameters used to represent the mobile phone's motion state; environmental parameters such as ambient light sensor parameters, latitude and longitude parameters, and altitude parameters used to represent the mobile phone's environment and location; and communication parameters such as cellular cell parameters and wireless fidelity (WIFI) signal parameters used to represent the mobile phone's communication state. During the process of establishing real-time communication, the mobile phone uses the device parameters acquired in real time as online data. After data processing such as data denoising and classification, the data is input into a target model deployed in the mobile phone. The output of the target model is used to determine the scene type of the mobile phone's current environment.

[0051] In some embodiments, as Figure 2 As shown in the figure, before deploying the target model in a mobile phone, it is usually necessary to train the target model offline to ensure the accuracy of the target model output results. For example, before offline training of the target model, it is necessary to first collect the historical device parameters of the mobile phone in different environments (such as subway platforms, subway cars, high-speed rail platforms, high-speed rail cars, low-speed driving, high-speed driving, etc.) as offline data. After data processing such as data denoising and classification, offline training is performed through a neural network. Through offline training, the input parameters and processing methods of the target model are continuously adjusted to ensure that the output results of the trained target model meet the requirements, that is, the accuracy of the output results meets the preset requirements.

[0052] Figure 3A According to some embodiments of the present invention, the training process of the target model is shown. Figure 3A As shown in Figure 2, the training process mainly includes:

[0053] 1. Scene division.

[0054] In some embodiments, the scene division can be performed based on the different environments where the mobile phone is located when the historical device parameters are collected. Figure 3BAs shown, environments such as subway platforms and high-speed rail platforms can be divided into indoor low-speed scenes, environments such as low-speed driving can be divided into outdoor low-speed scenes, environments such as high-speed rail cars and high-speed driving can be divided into outdoor high-speed scenes, and environments such as subway cars can be divided into indoor high-speed scenes, etc.; alternatively, environments such as subway platforms, subway cars, high-speed rail platforms, high-speed rail cars, low-speed driving, and high-speed driving can be divided into subway platforms, subway cars, high-speed rail platforms, high-speed rail cars, low-speed driving, and high-speed driving scenes respectively, without specific restrictions.

[0055] 2. Feature selection.

[0056] In some embodiments, the contribution value of the historical device parameters can be used as the corresponding change degree of the mobile phone's historical device parameters when the mobile phone changes to different environments. The collected historical device parameters are sorted based on the contribution value, and features are selected according to the sorting results to be sequentially input into the neural network for feature training. For example, when a mobile phone enters a subway car from a subway platform, enters a high-speed rail car from a high-speed rail platform, or changes from low-speed driving to high-speed driving, the corresponding acceleration sensor parameters, magnetic sensor parameters, gyroscope sensor parameters, and direction sensor parameters have the largest parameter changes, followed by gravity sensor parameters, ambient light sensor parameters, longitude and latitude parameters, altitude parameters, cellular cell parameters, Wi-Fi signal parameters, etc. In this way, when performing feature training, the acceleration sensor parameters, magnetic sensor parameters, and gyroscope sensor parameters can be input first, and then the gravity sensor parameters, ambient light sensor parameters, longitude and latitude parameters, altitude parameters, cellular cell parameters, Wi-Fi signal parameters, etc. can be input in sequence in subsequent optimization training to improve training accuracy.

[0057] 3. Data processing.

[0058] In some embodiments, different types of historical device parameters can be treated as individual one-dimensional vectors, and based on scene division, multiple types of historical device parameters in the corresponding scene can be formed into a multimodal vector, with each multimodal vector corresponding to a scene. For example, in a subway platform scene, the accelerometer sensor parameters, magnetic sensor parameters, gyroscope sensor parameters, orientation sensor parameters, gravity sensor parameters, ambient light sensor parameters, latitude and longitude parameters, altitude parameters, cellular cell parameters, and Wi-Fi signal parameters of a mobile phone can be treated as individual one-dimensional vectors, and a multimodal vector for the subway platform scene can be formed based on each one-dimensional vector in the subway platform scene.

[0059] 4. Neural network training: Input the multimodal vector into the neural network and use the neural network to train the model.

[0060] 5. Correct the results.

[0061] In some embodiments, after inputting a multimodal vector into a neural network to obtain preliminary neural network training results, the preliminary results can be modified based on prior knowledge to obtain a modified result, thereby optimizing neural network training and improving the accuracy of the neural network training results. For example, based on prior knowledge, when the previous scene type of a mobile phone is a subway platform, the next scene type is typically a subway car. Therefore, after outputting the subway platform scene, the probability of outputting the subway car scene in the subsequent output scene can be relatively increased to modify and optimize the model output results. Simultaneously, based on the results of previous preprocessing steps such as feature selection and data processing, the neural network can continue to be trained and optimized by adjusting network parameters, data dimensions, number of features, and scene sets. For example, during neural network training, neural network training can first be performed based on a multimodal vector formed by acceleration sensor parameters, magnetic sensor parameters, gyroscope sensor parameters, and orientation sensor parameters. Then, based on the neural network output results, orientation sensor parameters, gravity sensor parameters, ambient light sensor parameters, latitude and longitude parameters, altitude parameters, cellular cell parameters, and Wi-Fi signal parameters can be sequentially input for training optimization to adjust the number of parameters or vector dimensions during optimization, thereby improving the accuracy of the output results. Finally, during the optimization process, when the accuracy of the output results reaches the preset requirements, the offline training of the target model is completed. The target model can then be deployed in the corresponding mobile phone so that the mobile phone can subsequently use the target model to judge the scene type of the current environment.

[0062] In the following, some embodiments of the present invention are described by taking mobile phone 10 as a transmitting end and mobile phone 20 as a receiving end as an example.

[0063] In some embodiments, as Figure 4As shown, after the target model is deployed to the mobile phone 10 (i.e., the sending end), during the real-time communication between the mobile phone 10 and the mobile phone 20 (i.e., the receiving end), the mobile phone 10 inputs the device data obtained in real time by the mobile phone 10 into the target model after data processing such as data denoising and classification by the microcontroller unit (MCU) in the mobile phone 10, and performs terminal communication scene recognition to determine the scene type of the current environment of the mobile phone 10 based on the output result of the target model. Subsequently, the mobile phone 10 can use the network performance prediction algorithm to predict the network status such as network bandwidth, delay, packet loss rate, jitter, throughput, congestion level, etc. corresponding to the scene type. In some embodiments, the mobile phone 10 can use network performance prediction algorithms including network statistical algorithms (average value, variance, trend analysis, etc.), machine learning algorithms (regression, decision tree, etc.) to predict the network status based on the scene type. After obtaining the predicted network status, mobile phone 10 implements a source-channel joint transmission strategy decision to determine the transmission bandwidth of mobile phone 10 in the predicted network status based on the transmission data and transmission channel in the real-time communication of mobile phone 10. Finally, mobile phone 10 makes a dynamic bandwidth selection (adaptive rate selection, ARS) decision and adjusts the transmission bandwidth of mobile phone 10 in real-time communication to mobile phone 20 according to the switching of different network statuses in real time.

[0064] In some embodiments, after mobile phone 10 determines the scene type of its current environment based on the output of the target model, mobile phone 10 transmits the determined scene type information to mobile phone 20, performing a scene notification to inform mobile phone 20 of the scene type of its current environment. Mobile phone 20 receives the type information sent by mobile phone 10 and, based on the type information, uses a network performance prediction algorithm to predict downlink network performance for network conditions corresponding to the type information, such as network bandwidth, latency, packet loss rate, jitter, throughput, and congestion level. In some embodiments, mobile phone 20 may use network performance prediction algorithms, including network statistics algorithms (average, variance, trend analysis, etc.) and machine learning algorithms (regression, decision trees, etc.), to predict network conditions based on the type information. After obtaining the predicted network condition, mobile phone 20 implements a source-channel joint transmission strategy decision to determine the transmission bandwidth of mobile phone 20 in the predicted network condition based on the transmitted data and transmission channel in mobile phone 20's real-time communication. Finally, mobile phone 20 uses dynamic bandwidth selection to adjust the transmission bandwidth of data transmitted from mobile phone 20 to mobile phone 10 in real-time communication in real time according to the switching between different network conditions.

[0065] In some embodiments, a target model may also be deployed in the mobile phone 20 so that when the mobile phone 20 acts as a transmitter, the mobile phone 20 can input the device data obtained by the mobile phone 20 in real time into the target model to perform terminal communication scene recognition, so as to determine the scene type of the current environment of the mobile phone 20 based on the output result of the target model.

[0066] It is understood that when a mobile phone establishes real-time communication, it inputs its device parameters, acquired in real time, into a target model. Based on the output of the target model, the mobile phone can determine the scene type of the environment in which the mobile phone was located when acquiring the corresponding device parameters. For example, the mobile phone inputs the device parameters acquired while located in a subway platform environment into the target model. Based on the output of the target model, the mobile phone can determine that the scene type of the mobile phone's current environment is a subway platform. In this way, based on the determined subway platform scene type, the mobile phone can adjust its transmission bandwidth to a transmission bandwidth that matches the subway platform environment to enable real-time communication data transmission.

[0067] It can be understood that during the process of establishing real-time communication, the mobile phone inputs its device parameters, acquired in real time, into the target model. Based on the changes in the output of the target model, the mobile phone can determine the environmental change state of the mobile phone's environment. For example, the mobile phone inputs the device parameters acquired during the process of transitioning from a subway platform environment to a subway car environment into the target model in the order in which the parameters were acquired. If the device parameters input into the target model by the mobile phone are those acquired while in the subway platform environment, the target model outputs a subway platform scene type. If the device parameters input into the target model by the mobile phone are those acquired while in the subway car environment, the target model outputs a subway car scene type. Based on the changes in the output of the target model, the mobile phone can determine that the scene type of the mobile phone's environment has changed from a subway platform to a subway car. In this way, based on the determined change in scene type from a subway platform to a subway car environment, the mobile phone can adjust its transmission bandwidth from a transmission bandwidth that matches the subway platform environment to a transmission bandwidth that matches the subway car environment, so that when the mobile phone is in the subway car environment, it can transmit data for real-time communication using the transmission bandwidth that matches the subway car environment.

[0068] The following Figure 1 The video call scenario shown uses mobile phone 10 as a sending end and mobile phone 20 as a receiving end to illustrate some embodiments of the present invention.

[0069] Figure 5A According to some embodiments of the present invention, a flow chart of a communication method is shown. Figure 5A As shown, the process steps include:

[0070] S500: Mobile phone 10 establishes a real-time communication connection with mobile phone 20.

[0071] In some embodiments, as Figure 1 As shown, user A initiates a video call request to user B's phone 20 using the video call function of mobile phone 10. When user B accepts the video call request through mobile phone 20, user A and user B can conduct a video call through mobile phone 10 and mobile phone 20 respectively. At this point, mobile phone 10 and mobile phone 20 establish a real-time communication connection.

[0072] S511: The mobile phone 10 obtains device parameters of the mobile phone 10.

[0073] In some embodiments, as Figure 1 As shown, during a video call between user A and user B, mobile phone 10 acquires device parameters of mobile phone 10 in real time. For example, mobile phone 10 may use parameters of sensors such as an accelerometer, a magnetic sensor, a gyroscope, an orientation sensor, a gravity sensor, and an ambient light sensor collected during real-time communication, as well as location parameters such as the mobile phone 10's geographic location information such as latitude, longitude, and altitude, and network location information such as cellular cell information and Wi-Fi signal information as device parameters.

[0074] S512: The mobile phone 10 determines the scene type of the environment in which the mobile phone 10 is currently located based on the device parameters.

[0075] In some embodiments, the mobile phone 10 inputs the device parameters acquired in real time into a target model pre-deployed in the mobile phone 10, and determines the scene type of the environment in which the mobile phone 10 is currently located based on the output of the target model. The process of determining the scene type by the target model can be found in the description of the target model above and is not further elaborated here.

[0076] S513: The mobile phone 10 predicts the network status of the mobile phone 10 based on the determined scenario type.

[0077] In some embodiments, after the mobile phone 10 determines the scene type of the environment in which the mobile phone 10 is currently located through the target model, the mobile phone 10 can predict the network status of the environment in which the mobile phone 10 is currently located through network performance prediction algorithms such as network statistical algorithms of mean value, variance, and trend analysis, or machine learning algorithms of regression and decision trees.

[0078] S514: The mobile phone 10 adjusts the transmission bandwidth of the mobile phone 10 based on the predicted network status.

[0079] In some embodiments, the mobile phone 10 can make a joint transmission strategy decision through the source channel, determine the transmission bandwidth in the predicted network state based on the transmission data and transmission channel in the real-time communication between the mobile phone 10 and the mobile phone 20, and then adjust the transmission bandwidth of the mobile phone 10 in real-time communication in real time according to the switching of different network states based on the dynamic bandwidth selection decision.

[0080] S515: The mobile phone 10 transmits data to the mobile phone 20 based on the adjusted transmission bandwidth.

[0081] In some embodiments, mobile phone 10 can transmit data for real-time communication using a transmission bandwidth that matches the network status of the current environment, thereby ensuring the quality of real-time communication. For example, during a real-time video call between mobile phone 10 and mobile phone 20, when the network status of mobile phone 10 is determined to be a high-quality network status or to have changed from a low-quality network status to a high-quality network status, mobile phone 10 can increase the frame rate of the encoded video stream sent to mobile phone 20 to improve the clarity of the video image during the video call; when the network status of mobile phone 10 is determined to be a low-quality network status or to have changed from a high-quality network status to a low-quality network status, mobile phone 10 can reduce the frame rate of the encoded video stream sent to mobile phone 20 to ensure the smoothness of the video image during the video call.

[0082] In this way, during the real-time communication process with mobile phone 10 as the sending end, mobile phone 10 does not need to wait for feedback information from mobile phone 20 to determine the network status of mobile phone 10 itself. Mobile phone 10 can predict the network status of the current environment of mobile phone 10 through its own device parameters, ensuring that when mobile phone 10 switches to a different network status due to environmental changes, mobile phone 10 can adjust the transmission bandwidth of data transmission in real-time communication in time with the switching of network status, thereby improving the quality of real-time communication.

[0083] In some embodiments, when the mobile phone 10 recognizes that the scene type of the environment in which the mobile phone 10 is located has changed and the transmission bandwidth needs to be adjusted, the mobile phone 10 can display a pop-up window through the display screen of the mobile phone 10.

[0084] For example Figure 5B As shown, when the mobile phone 10 changes from the subway platform environment to the subway car environment, the network status of the mobile phone 10 changes from the high-quality network status to the low-quality network status, and the mobile phone 10 needs to reduce the transmission bandwidth of the mobile phone 10 in real-time communication. Figure 5B A pop-up window stating "You are taking the subway, the network quality is about to degrade" is displayed to remind user A that the video resolution of video call may be reduced.

[0085] For example Figure 5CAs shown, when the mobile phone 10 changes from the subway car environment to the subway platform environment, the network state of the mobile phone 10 changes from the low-quality network state to the high-quality network state, and the mobile phone 10 can improve the transmission bandwidth of the mobile phone 10 in real-time communication. Figure 5C The pop-up window "Network Quality Improved" is displayed to remind user A that the video resolution of the video call will be improved, or to enable user A to perform other sharing that requires large bandwidth support during the video call, or even perform other high-traffic operations such as online downloading at the same time.

[0086] I understand. Figure 5B and Figure 5C The pop-up window shown can be displayed in the middle of the screen of the mobile phone 10, or at the top, or at any other position, without any specific limitation.

[0087] S516: The mobile phone 10 sends the type information of the determined scene type to the mobile phone 20.

[0088] In some embodiments, after mobile phone 10 obtains the scene type of the environment in which mobile phone 10 is currently located through the target model, it synchronously sends the acquired scene type information to mobile phone 20 that has established a video call with mobile phone 10, so that mobile phone 20 can subsequently adjust the frame rate or bit rate of the video stream transmitted from mobile phone 20 to mobile phone 10 based on the network status corresponding to the scene type of mobile phone 10. It should be noted that when mobile phone 20 transmits a video stream to mobile phone 10, mobile phone 20 acts as the transmitter and mobile phone 10 acts as the receiver.

[0089] It is understood that step S516 can be performed before or after any step in the process of steps S513 to S515, or simultaneously with any step. For example, step S516 can be performed before step S513, after step S513, or simultaneously with step S513, without any specific limitation.

[0090] S521 : The mobile phone 20 receives the type information sent from the mobile phone 10 , and predicts the network status of the mobile phone 10 based on the type information.

[0091] In some embodiments, after mobile phone 10 determines its own scenario type at step S512, mobile phone 10 sends type information of the determined scenario type to mobile phone 20. Subsequently, after receiving the type information sent from mobile phone 10, mobile phone 20 may predict the network status of the environment currently located by mobile phone 10 based on the type information. For example, mobile phone 20 may also predict the network status of the environment currently located by mobile phone 10 using a network performance prediction algorithm such as a network statistical algorithm using mean, variance, or trend analysis, or a machine learning algorithm using regression or decision trees.

[0092] S522: The mobile phone 20 adjusts the transmission bandwidth of the mobile phone 20 based on the predicted network status.

[0093] In some embodiments, the mobile phone 20 can make a joint transmission strategy decision through the source channel, determine the transmission bandwidth in the predicted network state based on the transmission data and transmission channel in the real-time communication between the mobile phone 20 and the mobile phone 10, and then adjust the transmission bandwidth of the mobile phone 20 in real-time communication in real time according to the switching of different network states based on the dynamic bandwidth selection decision.

[0094] S523: The mobile phone 20 transmits data to the mobile phone 10 based on the adjusted transmission bandwidth.

[0095] In some embodiments, when mobile phone 20 transmits data as a transmitter to mobile phone 10 as a receiver, mobile phone 20 may perform real-time communication data transmission based on a transmission bandwidth that matches the network status of mobile phone 10's current environment, thereby ensuring the quality of real-time communication. In some embodiments, when mobile phone 20 recognizes that the scene type of mobile phone 10's current environment has changed and that the transmission bandwidth needs to be adjusted, mobile phone 20 may display a pop-up window on the display screen of mobile phone 20, such as a pop-up window stating that "the other party's network quality is about to degrade," to remind user B that the video resolution of the video call may be reduced.

[0096] It can be understood that in step S521, the mobile phone 10 can send the type information of the scene type determined by the mobile phone 10 to the mobile phone 20 in the form of a notification message. In some embodiments, the notification message can be used to trigger the adjustment of the transmission bandwidth of the data transmitted by the mobile phone 20 during real-time communication. For example, in steps S522 to S523, the mobile phone 20 calls the source channel joint transmission strategy decision and the dynamic bandwidth selection decision based on the received notification message to adjust the transmission bandwidth. In some other embodiments, the notification message can also only serve as a prompt. For example, based on the received notification message, the mobile phone 20 displays a pop-up window on the display screen of the mobile phone 20 to prompt user B that the network status of the other user A has changed. User B can manually adjust the transmission bandwidth of the mobile phone 20 based on the prompt information, such as video resolution, etc., without specific limitation.

[0097] Based on the above Figure 5A From the description of the communication method shown, it can be seen that when mobile phone 10 is establishing real-time communication for a video call with mobile phone 20, mobile phone 10 as the sending end does not need to wait for feedback information from mobile phone 20 on the receiving end. Mobile phone 10 can directly predict the network status of the current environment of mobile phone 10 based on its own device parameters. Then, mobile phone 10 can timely adjust the transmission bandwidth of data transmission of mobile phone 10 during the real-time communication of the video call according to the predicted network status, thereby adjusting the video frame rate of the video stream transmitted by mobile phone 10 to mobile phone 20 to ensure the quality of real-time communication of the video call.

[0098] At the same time, after mobile phone 10 determines its own scenario type, mobile phone 10 sends the determined scenario type to mobile phone 20, so that mobile phone 20 can predict the network status of mobile phone 10's current environment. Mobile phone 20 can then adjust the transmission bandwidth of mobile phone 20's data transmission in a timely manner based on the predicted network status of mobile phone 10. In this way, when mobile phone 20, as the sender, transmits data to the receiver of mobile phone 10, mobile phone 20 can perform real-time communication data transmission based on the transmission bandwidth that matches the network status of mobile phone 10's current environment, ensuring the quality of real-time communication.

[0099] Figure 6 According to some embodiments of the present invention, a diagram of the architecture of a mobile phone 10 is shown. Figure 6 As shown, the layered architecture divides the operating system of the mobile phone 10 into four layers, namely, the application layer, the system service layer, the hardware abstraction layer (HAL) layer and the kernel layer from top to bottom, and the layers communicate with each other through software interfaces.

[0100] The application layer can include a series of application packages. For example, the application package may include applications such as camera, phone, information, weather, map, video, etc. The application layer and the system service layer transmit data through the application programming interface (API). The system service layer may include multiple functional modules. For example, the surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. The kernel layer is the layer between hardware (such as chips) and software. The kernel layer includes at least display driver, camera driver, audio driver, sensor driver, etc.

[0101] Figure 7 According to some embodiments of the present invention, a schematic diagram of the structure of a mobile phone 10 is shown. Figure 7As shown, the mobile phone 10 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

[0102] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0103] For example, in some embodiments, the processor 110 may execute instructions corresponding to the methods provided in the aforementioned embodiments.

[0104] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the instruction or data again, it can directly access the memory. This avoids repeated accesses, reduces processor 110 latency, and improves processing efficiency.

[0105] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface.

[0106] USB interface 130 is an interface that complies with USB standards and specifications, and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. USB interface 130 can be used to connect a charger to charge mobile phone 10, and can also be used to transfer data between mobile phone 10 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as AR devices.

[0107] It should be understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the mobile phone 10. In other embodiments of the present invention, the mobile phone 10 may also adopt different interface connection methods from those in the above embodiments, or a combination of multiple interface connection methods.

[0108] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the mobile phone 10. While charging the battery 142, the charging management module 140 can also provide power to the mobile phone 10 via the power management module 141.

[0109] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0110] The wireless communication functions of mobile phone 10 are implemented using antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, a modem processor, and a baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 10 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0111] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G / 6G applied on the mobile phone 10. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0112] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0113] The wireless communication module 160 can provide wireless communication solutions applied to the mobile phone 10, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0114] In some embodiments, the antenna 1 of the mobile phone 10 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone 10 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-CDMA), long term evolution (LTE), 5G and subsequent evolution standards, BT, GNSS, WLAN, NFC, FM, and / or IR technology. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS) and / or the satellite-based augmentation system (SBAS).

[0115] The mobile phone 10 implements display functions through a GPU, display screen 194, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0116] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the mobile phone 10 may include one or N display screens 194, where N is a positive integer greater than one.

[0117] The mobile phone 10 can implement shooting functions through the ISP, camera 193, video codec, GPU, display 194, and application processor. The ISP is used to process data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then transmitted to the ISP for processing and converted into an image visible to the naked eye. The ISP can also perform algorithmic optimization on image noise, brightness, and skin color. The ISP can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be set in the camera 193.

[0118] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the mobile phone 10 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0119] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the mobile phone 10 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0120] Video codecs are used to compress or decompress digital video. Mobile phone 10 may support one or more video codecs. This allows mobile phone 10 to play or record videos in a variety of encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0121] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU enables intelligent cognitive applications on the phone 10, such as image recognition, face recognition, speech recognition, and text comprehension.

[0122] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 10. The external memory card communicates with the processor 110 through the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0123] The internal memory 121 can be used to store computer executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile phone 10 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the mobile phone 10 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor. For example, in some embodiments, the internal memory 121 may be used to temporarily store instructions of the methods provided in the aforementioned embodiments.

[0124] The mobile phone 10 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0125] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the mobile phone 10 by inserting or removing it from the SIM card interface 195. The mobile phone 10 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The mobile phone 10 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the mobile phone 10 uses an eSIM, or embedded SIM card. The eSIM card can be embedded in the mobile phone 10 and cannot be separated from the mobile phone 10.

[0126] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the mobile phone 10. In other embodiments of the present invention, the mobile phone 10 may include more or fewer components than shown, or may combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0127] In some embodiments, an embodiment of the present invention further provides a computer-readable medium, which stores program code. When the computer program code runs on a computer, the computer executes the methods in the above aspects.

[0128] In some embodiments, an embodiment of the present invention further provides a computer program product, which includes: computer program code, which enables the computer to execute the methods in the above aspects when the computer program code is run on a computer.

[0129] In the accompanying drawings, some structural or method features may be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order may not be required. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of a structural or method feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such features may not be included or may be combined with other features.

[0130] It should be noted that the various units / modules mentioned in the various device embodiments of the present invention are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems raised by the present invention. In addition, in order to highlight the innovative part of the present invention, the above-mentioned device embodiments of the present invention do not introduce units / modules that are not closely related to solving the technical problems raised by the present invention. This does not mean that other units / modules do not exist in the above-mentioned device embodiments.

[0131] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0132] While the present invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention.

Claims

1. A communication method, applied to a first device, wherein: The first device establishes a real-time communication connection with the second device, wherein the method comprises: determining a parameter change state of the first device from a first device parameter to a second device parameter; Based on the parameter change state, determining that the first device changes from a first scene corresponding to the first device parameter to a second scene corresponding to the second device parameter; Adjusting the transmission bandwidth of the first device from a first bandwidth corresponding to the first scenario to a second bandwidth corresponding to the second scenario; The first device transmits data to the second device based on the second bandwidth.

2. The method according to claim 1, characterized in that The determining, based on the parameter change state, that the first device changes from a first scene corresponding to the first device parameter to a second scene corresponding to the second device parameter includes: Inputting the parameter change state into a preset target model, wherein the target model is determined based on motion parameters, environmental parameters, and communication parameters of the first device as training samples, the motion parameters are used to represent the motion state of the first device, the environmental parameters are used to represent the environment and position of the first device, and the communication parameters are used to represent the communication state of the first device; Determining, based on an output result of the target model, a first scene type of a first scene corresponding to the first device parameter, and a second scene type of a second scene corresponding to the second device parameter; Based on the first scene type and the second scene type, it is determined that the first device changes from the first scene to the second scene.

3. The method according to claim 2, after determining that the first device changes from the first scene to the second scene, the method further comprises: The first device sends a notification message to the second device, where the notification message is used to indicate that the first device changes from the first scene type to the second scene type.

4. The method according to claim 3, characterized in that: The notification message includes type information of the second scenario type, where the type information is used by the second device to determine the second bandwidth, and the notification message is used to instruct the second device to transmit data to the first device based on the second bandwidth.

5. The method according to claim 2, characterized in that: The motion parameters include one or more of the following: acceleration sensor parameters, magnetic sensor parameters, gyroscope sensor parameters, direction sensor parameters, and gravity sensor parameters; The environmental parameters include one or more of the following: ambient light sensor parameters, longitude and latitude parameters, and altitude parameters; The communication parameters include one or more of the following: cellular cell parameters, WIFI signal parameters.

6. The method according to claim 2, characterized in that The first scene type includes any one of the following: subway platform, subway car, high-speed rail platform, high-speed rail car, low-speed driving, high-speed driving; The second scene type includes any one of the following: a subway platform, a subway car, a high-speed rail platform, a high-speed rail car, low-speed driving, and high-speed driving.

7. The method according to claim 1, characterized in that When the network transmission rate of the first scenario is greater than the network transmission rate of the second scenario, the first bandwidth is greater than the second bandwidth; or, When the network transmission rate of the first scenario is lower than the network transmission rate of the second scenario, the first bandwidth is lower than the second bandwidth.

8. A communication method, applied to a first device, wherein: The first device establishes a real-time communication connection with the second device, wherein the method comprises: Determining a second device parameter of the first device in a second scenario; Inputting the second device parameters into a preset target model, wherein the target model is determined based on motion parameters, environmental parameters, and communication parameters of the first device as training samples, the motion parameters are used to represent the motion state of the first device, the environmental parameters are used to represent the environment and position of the first device, and the communication parameters are used to represent the communication state of the first device; determining a second scene type of the second scene based on an output result of the target model; Based on the second scenario type, a transmission bandwidth of the first device is correspondingly set; The first device transmits data to the second device based on the set transmission bandwidth.

9. A communication method, applied to a second device, wherein: The second device establishes a real-time communication connection with the first device, wherein the method comprises: receiving a notification message sent by the first device, wherein the notification message is used to instruct the first device to change from a first scene type corresponding to a first scene to a second scene type corresponding to a second scene; Based on the second scenario type, a transmission bandwidth of the second device is correspondingly set; The second device transmits data to the first device based on the set transmission bandwidth.

10. A communication method, applied to a first device and a second device, wherein: The first device establishes a real-time communication connection with the second device, wherein the method comprises: The first device determines a parameter change state of the first device from a first device parameter to a second device parameter; The first device inputs the parameter change state into a preset target model, wherein the target model is determined based on motion parameters, environmental parameters, and communication parameters of the first device as training samples, the motion parameters are used to represent the motion state of the first device, the environmental parameters are used to represent the environment and position of the first device, and the communication parameters are used to represent the communication state of the first device; The first device determines, based on an output result of the target model, a first scene type of a first scene corresponding to the first device parameter and a second scene type of a second scene corresponding to the second device parameter; The first device determines, based on the first scene type and the second scene type, that the first device changes from the first scene to the second scene; The first device adjusts the transmission bandwidth of the first device from a first bandwidth corresponding to the first scenario to a second bandwidth corresponding to the second scenario; The first device transmits data to the second device based on the second bandwidth; The first device sends a notification message to the second device, where the notification message is used to indicate that the first device changes from the first scene type to the second scene type; The second device receives the communication information; The second device sets the transmission bandwidth of the second device to the second bandwidth based on the second scenario type; The second device transmits data to the first device based on the second bandwidth.

11. A computer-readable storage medium, characterized in that: The readable storage medium stores instructions, which, when executed on a terminal device, enable the terminal device to implement the method according to any one of claims 1 to 10.

12. A terminal device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the terminal device; And a processor, which is one of the processors of the terminal device, configured to execute instructions stored in the memory to implement the method described in any one of claims 1 to 10.

Images