A data processing method and device, computer equipment and storage medium

Abstract

Embodiments of the present application disclose a data processing method and device, computer equipment and a storage medium. The method comprises: when a service state of a controlled device is a first state, determining a first type module and a second type module in a communication module of the controlled device; a priority of a first link corresponding to the first type module is higher than a priority of a second link corresponding to the second type module; obtaining a first multimedia data stream and a second multimedia data stream for transmission to a control device; sending the first multimedia data stream to the control device through the first link, and sending the second multimedia data stream to the control device through the second link, so that the control device determines control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream; and when the controlled device receives the control signaling, performing a control operation indicated by the control signaling. The embodiments of the present application can improve the efficiency of data transmission.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to the technical field of 5G (5th Generation mobile network), and more particularly to a data processing method and device, a computer device and a storage medium. BACKGROUND

[0002] With the digital transformation of various industries and the improvement of network capabilities, remote is gradually becoming a trend, especially in combination with automation and unmanned, which can effectively improve work efficiency, improve working environment and reduce labor costs. Therefore, the development of 5G communication technology is particularly important. However, in the existing 5G communication system, the controlled device deployed in the 5G communication system can only provide differentiated quality of service for data streams with different IP addresses. However, for the remote control application running on the same controlled device, all multimedia data streams need to be sent to the target application (for example, the control application) with the same IP address, which will result in that the traditional controlled device cannot provide differentiated data guarantee capability, so as to increase the time delay of data transmission, thereby reducing the efficiency of data transmission. SUMMARY

[0003] The embodiments of the present application provide a data processing method and device, a computer device and a storage medium, which can improve the efficiency of data transmission.

[0004] In one aspect, the embodiments of the present application provide a data processing method, comprising:

[0005] When the service state of the controlled device is a first state, determining a first type module and a second type module in the communication module of the controlled device; the first type module and the second type module are determined based on the network state parameter of the geographical location where the controlled device is located; the priority of the first link corresponding to the first type module is higher than the priority of the second link corresponding to the second type module;

[0006] Obtaining a first multimedia data stream and a second multimedia data stream for transmission to the control device; the first multimedia data stream is determined by the key data frame in the multimedia data collected by the controlled device; the second multimedia data stream is determined by the auxiliary data frame in the multimedia data;

[0007] The first multimedia data stream is sent to the control device through the first link, and the second multimedia data stream is sent to the control device through the second link, so that the control device determines the control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream;

[0008] When the controlled device receives the control signaling, a control operation indicated by the control signaling is performed.

[0009] The embodiment of the application provides a data processing device, comprising:

[0010] A module type determination module is configured to determine a first type module and a second type module in a communication module of the controlled device when a service state of the controlled device is a first state; the first type module and the second type module are determined based on a network state parameter of a geographic location where the controlled device is located; a priority of a first link corresponding to the first type module is higher than a priority of a second link corresponding to the second type module.

[0011] A multimedia data stream acquisition module is configured to acquire a first multimedia data stream and a second multimedia data stream for transmission to the control device; the first multimedia data stream is determined by a key data frame in the multimedia data collected by the controlled device; the second multimedia data stream is determined by an auxiliary data frame in the multimedia data.

[0012] A first data stream sending module is configured to send the first multimedia data stream to the control device through the first link and send the second multimedia data stream to the control device through the second link, so that the control device determines control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream.

[0013] A control operation execution module is configured to perform a control operation indicated by the control signaling when the controlled device receives the control signaling.

[0014] The device further comprises:

[0015] A signal strength acquisition module is configured to acquire a network state parameter of each module in the communication module of the controlled device.

[0016] A heartbeat state detection module is configured to detect a heartbeat state of each module between a remote control application in the controlled device and a control application in the control device.

[0017] A first service condition acquisition module is configured to acquire a service condition associated with the controlled device, and determine a first network state threshold and a first service detection duration in the first service condition from the service condition.

[0018] A first state switching module is configured to determine that the controlled device satisfies the first service condition when the network state parameter of each module is greater than the first network state threshold within the first service detection duration and the heartbeat state of each module is in a normal state, and switch the service state of the controlled device to the first state; the normal state is determined when the remote control application periodically sends a heartbeat packet to the control application.

[0019] The communication module includes N modules; N is a positive integer greater than 1; the first type of module in the N modules includes module U1; the second type of module in the N modules includes module U2;

[0020] The apparatus further includes:

[0021] The parameter determination module is configured to determine a network status parameter of the module U1 as a first network status parameter, a heartbeat status of the module U1 as a first heartbeat status, a network status parameter of the module U2 as a second network status parameter, and a heartbeat status of the module U2 as a second heartbeat status.

[0022] The second service condition acquisition module is configured to acquire a second service condition from a service condition associated with the controlled device.

[0023] The second state switching module is configured to switch the service state of the controlled device from the first state to the second state when it is determined that the controlled device satisfies the second service condition based on the first network status parameter, the second network status parameter, the first heartbeat status, and the second heartbeat status.

[0024] The second service condition includes a first sub-condition; the first sub-condition includes a second network status threshold and a second service detection duration.

[0025] The second state switching module includes:

[0026] The first switching unit is configured to determine that the controlled device satisfies the first sub-condition and switch the service state of the controlled device from the first state to the second state if the first network status parameter and the second network status parameter are both less than the second network status threshold within the second service detection duration, and the first heartbeat status and the second heartbeat status are both in a normal state.

[0027] The second service condition includes a second sub-condition; the second sub-condition includes a first network status threshold, a second network status threshold, and a third service detection duration; the second network status threshold is less than the first network status threshold.

[0028] The second state switching module further includes:

[0029] The second switching unit is configured to determine that the controlled device satisfies the second sub-condition and switch the service state of the controlled device from the first state to the second state if the first network status parameter is greater than the first network status threshold within the third service detection duration, the second network status parameter is less than the second network status threshold within the third service detection duration, and the first heartbeat status and the second heartbeat status are both in a normal state.

[0030] The first type module includes a module U1 in the communication module; the second type module includes a module U2 in the communication module; the access area associated with the module U1 is a first area, and the access area associated with the module U2 is a second area; the first area is different from the second area; the first link refers to a communication link corresponding to the module U1; the module type of the module U1 is the first type module associated with the first area;

[0031] The apparatus further includes:

[0032] The area locking module is configured to lock the first area associated with the module U1 when the service state of the controlled device is the second state.

[0033] The parameter configuration notification sending module is configured to send, through a remote control application running on the controlled device, a parameter configuration notification associated with the module U2 to a remote control management server corresponding to the remote control application, so that the remote control management server invokes a communication management system to perform parameter configuration on the second area associated with the module U2; the communication management system is configured to generate module update information; the module update information is generated when the module type corresponding to the module U2 is changed from the second type module to the first type module based on the second area associated with the module U2.

[0034] The module update information receiving module is configured to receive the module update information returned by the remote control management server, and determine the communication link corresponding to the module U2 as a third link based on the module update information and the first type module associated with the second area.

[0035] The network aggregation module is configured to perform network aggregation on the first link and the third link through the remote control application to obtain a first aggregated link; the first aggregated link is configured to transmit the first multimedia data stream and the second multimedia data stream.

[0036] The communication module includes N modules; N is a positive integer greater than 1; the N modules include the module U1 and the module U2.

[0037] The apparatus further includes:

[0038] The target state determining module is configured to determine the first state as a target state, or determine the second state as the target state when the first state of the controlled device is changed to the second state.

[0039] The third service condition obtaining module is configured to obtain a third service condition from a service condition associated with the controlled device; the third service condition includes a first network state threshold, a second network state threshold, and a fourth service detection time length; the second network state threshold is less than the first network state threshold.

[0040] The third state switching module is configured to determine that the controlled device satisfies a third service condition, and switch the service state of the controlled device from the target state to a third state, if the network state parameter of the module U1 is less than a second network state threshold within a fourth service detection duration, the network state parameter of the module U2 is greater than a first network state threshold within the fourth service detection duration, and the heartbeat state of the module U1 and the heartbeat state of the module U2 are both in a normal state.

[0041] The access area associated with the module U1 is a first area, and the access area associated with the module U2 is a second area; the first area is different from the second area.

[0042] The device further includes:

[0043] The communication module type changing module is configured to determine the module type corresponding to the module U2 when the service state of the controlled device is the third state, and change the module type corresponding to the module U2 from the second type module to the first type module based on the second area associated with the module U2 when the module type corresponding to the module U2 is the second type module.

[0044] The area switching notification sending module is configured to send an area switching notification to a remote control management server corresponding to a remote control application in the controlled device through the remote control application; the area switching notification is used to instruct the remote control management server to configure the access area associated with the module U1 as the second area associated with the module U2 through a communication management system; and the communication management system is used to generate area switching information associated with the module U1 when the access area associated with the module U1 is successfully switched to the second area.

[0045] The area switching information receiving module is configured to receive the area switching information returned by the remote control management server, and change the module type corresponding to the module U1 from the first type module to the second type module based on the second area associated with the module U1.

[0046] The device further includes:

[0047] The fourth service condition obtaining module is configured to obtain a fourth service condition from service conditions associated with the controlled device; the fourth service condition includes a fifth service detection duration.

[0048] The fault module determining module is configured to determine a module in a fault state as a fault module and switch the service state of the controlled device to a fourth state if there is a module in a fault state in the module U1 and the module U2; the fault state refers to a state in which a network state parameter is not reported within a fifth service detection duration or a heartbeat state is in an abnormal state.

[0049] The normal module determination module is configured to determine, in the module U1 and the module U2, the module other than the fault module as the normal module;

[0050] The normal module type changing module is configured to change the module type corresponding to the normal module from the second type module to the first type module based on the access area associated with the normal module if the module type corresponding to the normal module is the second type module.

[0051] The communication module includes N modules; N is a positive integer greater than 1;

[0052] The module type determination module includes:

[0053] The module auxiliary information determination unit is configured to determine N module auxiliary information corresponding to the N modules when the service state of the controlled device is the first state; one module corresponds to one module auxiliary information, and the network state parameter in one module auxiliary information is determined by the network state of the access area associated with the corresponding module; the network state of the access area is detected based on the geographical position of the controlled device;

[0054] The module auxiliary information sending unit is configured to send the N module auxiliary information to the remote control management server corresponding to the remote control application through the remote control application running on the controlled device; the remote control management server is configured to obtain the module establishment information through the communication management system; the module establishment information is generated when the communication management system successfully establishes the first type module based on the network state parameter in each module auxiliary information in the N module auxiliary information; the first type module is the module corresponding to the module auxiliary information with the maximum network state parameter;

[0055] The module establishment information receiving unit is configured to receive the module establishment information through the remote control application, determine the first type module based on the module establishment information, and determine the module other than the first type module in the N modules as the second type module.

[0056] The multimedia data stream obtaining module includes:

[0057] The multimedia data acquisition unit is configured to acquire the multimedia data of the geographical position of the controlled device through the data acquisition component of the controlled device;

[0058] The encoding configuration policy obtaining unit is configured to obtain the encoding configuration policy associated with the multimedia data, determine the key data frame in the multimedia data and the auxiliary data frame in the multimedia data based on the encoding configuration policy;

[0059] The key frame encoding processing unit is configured to perform encoding processing on the key data frame based on the encoding configuration policy to obtain the first multimedia data stream for transmission to the control device associated with the controlled device;

[0060] An auxiliary frame encoding processing unit is configured to encode and process the auxiliary data frame based on the encoding configuration strategy to obtain a second multimedia data stream for transmission to the control device.

[0061] The apparatus further comprises:

[0062] A network aggregation condition obtaining module is configured to obtain a network aggregation condition associated with the controlled device, and when the controlled device satisfies the network aggregation condition, perform network aggregation on the first link and the second link through a remote control application running on the controlled device to obtain a second aggregation link.

[0063] A second data stream sending module is configured to send the first multimedia data stream and the second multimedia data stream to the control device through the second aggregation link.

[0064] The network aggregation condition comprises a first aggregation condition or a second aggregation condition; the first aggregation condition refers to that the network state parameter of the first type module or the second type module decreases, and the decreased network state parameter is not less than a second network state threshold; and the second aggregation condition refers to that the network load of a geographic location where the controlled device is located is greater than a load threshold.

[0065] The control operation execution module comprises:

[0066] A first signaling receiving unit is configured to receive, through the first link, a first control signaling returned by the control device to the remote control application of the controlled device.

[0067] A second signaling receiving unit is configured to receive, through the second link, a second control signaling returned by the control device to the remote control application; the second control signaling is obtained by copying the first control signaling by a control application in the control device.

[0068] A signaling deduplication processing unit is configured to perform deduplication processing on the first control signaling and the second control signaling through the remote control application to obtain a target control signaling, and execute a control operation indicated by the target control signaling; the target control instruction is the first control signaling or the second control signaling.

[0069] The embodiment of the present application provides a computer device, which comprises a processor and a memory.

[0070] The processor is connected with the memory, and the memory is configured to store a computer program; when the computer program is executed by the processor, the computer device executes the method provided by the embodiment of the present application.

[0071] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0072] The embodiment of the present application provides a computer program product or a computer program, and the computer program product or the computer program comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided by the embodiment of the present application.

[0073] In the embodiment of the present application, since the priority of the first link corresponding to the first type module is higher than the priority of the second link corresponding to the second type module, when the controlled device is in the first state, the first multimedia data stream determined by the key data frame can be transmitted to the control device according to the first link corresponding to the first type module determined by the controlled device. At the same time, the second multimedia data stream determined by the auxiliary data frame can be transmitted to the control device according to the second link corresponding to the second type module determined by the controlled device, so that the multimedia data corresponding to the multimedia data stream (the first multimedia data stream and the second multimedia data stream) is transmitted on the first link and the second link, so as to provide differentiated data guarantee capability, thereby reducing the time delay during data transmission, and further improving the efficiency during data transmission. BRIEF DESCRIPTION OF DRAWINGS

[0074] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0075] Figure 1 is a network architecture schematic diagram provided by the embodiment of the present application;

[0076] Figure 2 is a scene schematic diagram for data interaction provided by the embodiment of the present application;

[0077] Figure 3 is a flow schematic diagram of a data processing method provided by the embodiment of the present application;

[0078] Figure 4 is a flow schematic diagram of a controlled device in a first state provided by the embodiment of the present application;

[0079] Figure 5 is a state switching schematic diagram for a controlled device provided by an embodiment of the present application;

[0080] Figure 6 is a flowchart corresponding to when the controlled device is in the second state provided by an embodiment of the present application;

[0081] Figure 7 is a flowchart corresponding to when the controlled device is in the third state provided by an embodiment of the present application;

[0082] Figure 8 is a structural schematic diagram of a data processing apparatus provided by an embodiment of the present application;

[0083] Figure 9 is a schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0084] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0085] Please refer to Figure 1 , Figure 1 is a network architecture schematic diagram provided by an embodiment of the present application. As Figure 1 indicated, the communication management system 1X in the network architecture can include but is not limited to a 5G private network management system, which will not be limited here. When the communication management system 1X is a 5G private network management system, the communication network corresponding to the communication management system 1X can be a 5G private network. It should be understood that Figure 1 the network architecture shown can also include a controlled device 110B, a control device 120B, and a remote control management server 10F.

[0086] Among them, the controlled device 110B can include a data acquisition component, a remote control gateway running a remote control application, and a terminal device 10Z (for example, a 5G customized industry terminal). As Figure 1As shown, the data collection component here can be configured to collect multimedia data (i.e., including audio data and video data) in real time, which can include an audio data collection component (e.g., a microphone) and a video data collection component (e.g., a camera). It can be understood that the remote control gateway and the terminal device 10Z in the embodiments of the present application can be two independent devices or one integrated device, which will not be limited here. The remote control application (e.g., an industry remote control audio and video real-time transmission software) here can be configured to encode the multimedia data collected by the data collection component to obtain a multimedia data stream corresponding to the multimedia data. The terminal device 10Z here is customized by the terminal chip without modifying the terminal chip, so as to save the labor cost and simplify the implementation complexity. The communication module (e.g., a 5G module) of the terminal device 10Z can be considered as a user terminal (User Equipment, UE) in a 5G private network, and the number of the communication modules can be N, N being a positive integer greater than 1, and the N modules can be jointly optimized with the remote control application. As shown in Figure 1 The N modules can specifically include a module U1, a module U2, …, and a module UN. N .

[0087] It can be understood that when each module respectively and independently performs the selection, access and registration procedures of the access area (for example, a 5G area) according to a standard procedure, the application embodiment can refer to the service state of the controlled device 110B as an initial state. It should be understood that the access area associated with each module can be the same area or different areas, which will not be limited here. Among them, the application embodiment can refer to the module that establishes the first bearer (for example, a dedicated bearer) and the second bearer (for example, a default bearer) with the access area as a first type module, and refer to the module that only establishes the second bearer with the access area as a second type module. Among them, the bearer (Bearer) here can be divided into the following two categories according to the QCI (QoS class identifier, that is, a parameter for identifying the transmission characteristics of service data packets): GBR (Guaranteed Bit Rate) type bearer and Non-GBR type bearer. It should be understood that the dedicated bearer here is used to provide quality of service (Quality of Service, QoS) guarantee for specific services (for example, audio and video call, control signaling transmission and other services), which can belong to the GBR type bearer or the Non-GBR type bearer, which will not be limited here. The default bearer here is used to guarantee the basic service requirements of the user, and each user has at least one default bearer, which generally belongs to the Non-GBR type bearer. The priority of the communication link corresponding to the default bearer is lower than that of the communication link corresponding to the dedicated bearer.

[0088] If Figure 1The illustrated communication module takes 2 as an example, specifically including module U1 and module U2, then when the first type module is module U1 in N modules, and the second type module is module U2 in N modules, it means that module U1 can establish a first bearer (for example, a dedicated bearer) and a second bearer (for example, a default bearer) with the access area (that is, the first area) associated with module U1 at the same time, while module U2 can only establish a second bearer with the access area (that is, the second area) associated with module U2. It is worth noting that the data stream sent by module U1 to the target address (that is, the Internet Protocol Address, IP address of the control application of the control device 120B) will be sent through the communication link corresponding to the first bearer of module U1, while the data stream sent by module U1 to other addresses except the target address will still be sent through the communication link corresponding to the second bearer of module U1. Among them, the communication link corresponding to the first bearer established by module U1 and the first area can be referred to as the first link, and the communication link corresponding to the second bearer established by module U2 and the second area can be referred to as the second link. Wherein, the communication link here refers to the link established between the remote control application and the control application through the communication air interface (for example, 5G air interface) and access routing of a certain module in the communication network (for example, 5G private network).

[0089] Wherein, Figure 1 The control device 120B and the controlled device 110B illustrated can be two terminal devices deployed in the same vehicle device (for example, a vehicle device in an automatic driving scene), for example, the controlled device 110B (for example, a wheel, a mechanical arm, etc.) is deployed on the vehicle device, and the control device 120B can be deployed in the cockpit of the vehicle device. Optionally, the controlled device 110B can be deployed on the vehicle device, and the control device 120B can be deployed in a location area (for example, a control room) having a location association relationship with the vehicle device, which will not be limited here. As Figure 1 The control device 120B illustrated can include an access routing, a control application, and a data output component. The access routing here can include a data stream receiving function and a signaling sending function. The data stream receiving function is used to receive the multimedia data stream corresponding to the multimedia data sent by the remote control application, and the signaling sending function is used to send the control signaling associated with the multimedia data determined by the control application. The control application here can be used to decode and render the received multimedia data stream, etc., and output the recovered multimedia data on Figure 1 The data output component illustrated can include a video data output component (for example, a display) and an audio data output component (for example, a microphone).

[0090] wherein, Figure 1 The remote control management server 10F shown can be a server corresponding to the remote control application. The remote control management server 10F can be a standalone physical server, a server cluster or a distributed system formed by multiple physical servers, or a cloud server providing cloud computing services. Cloud computing is a computing mode that distributes computing tasks on a resource pool formed by a large number of computing devices, so that various application systems can obtain computing power, storage space, and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" can be infinitely expanded in the eyes of the user, and can be obtained at any time, used on demand, expanded at any time, and paid according to use. As a basic capability provider of cloud computing, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) is established, and a plurality of types of virtual resources are deployed in the resource pool for external customers to select and use. The cloud computing resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0091] For ease of understanding, further, Figure 2 Figure 2 is a scene diagram for data interaction provided by an embodiment of the present application. It should be understood that the controlled device 210B in the embodiment of the present application can be deployed on a vehicle device, and the controlled device 210B can be the controlled device 110B in the embodiment corresponding to the above Figure 1 , for ease of description, the number of communication modules in the controlled device 210B can be taken as two, which can specifically include Figure 2 modules U1 and U2 shown. The control device 220B in the embodiment of the present application can be deployed on the cockpit of the vehicle device where the controlled device 210B is located, or can be deployed in a location area (for example, a remote control room) having a location association relationship with the vehicle device, which will not be limited here. The control device 220B can be a terminal device having a network connection relationship with the controlled device 210B, and the control device 220B can be the controlled device 120B in the embodiment corresponding to the above Figure 1 .

[0092] It should be understood that when both modules of the controlled device 210B independently perform the selection, access, and registration procedures of the access area (for example, a 5G area) according to the standard procedure, the business state of the controlled device 210B can be referred to as an initial state. The access area associated with the module U1 can be a first area, and the access area associated with the module U2 can be a second area. The first area and the second area here can be the same area or different areas, which will not be limited here.​

[0093] When the service state of the controlled device 210B is the first state (for example, the normal transmission state), the controlled device 210B can determine the first type module and the second type module in the built-in communication module. The first type module and the second type module here are determined by the remote control management server through the communication management system based on the network state parameters of the geographical location of the controlled device 210B. The network state parameters here may include signal strength parameters (Reference Signal Received Power, referred to as RSRP), signal to noise ratio parameters (Signal to Interferenceplus Noise Ratio, referred to as SINR) and other network state measurement values, and the network state parameters will not be limited here. The remote control management server can be the server corresponding to the remote control application in the controlled device 210B, and the remote control management server can be the above-mentioned Figure 1 The remote control management server 10F in the corresponding embodiment. For example, Figure 2 As shown, the first type module determined by the controlled device 210B can be module U1, and the second type module can be module U2. This means that module U1 can simultaneously establish a first bearer (e.g., a dedicated bearer) and a second bearer (e.g., a default bearer) with the first area, while module U2 can only establish a second bearer with the second area. It is understandable that the communication link (i.e., the first link) corresponding to the establishment of the first bearer by module U1 with the first area has a higher priority than the communication link (i.e., the second link) corresponding to the establishment of the second bearer by module U2 with the second area.

[0094] like Figure 2 As shown, when the controlled device 210B obtains multimedia data 20S (including audio data and video data) collected in real time, it can obtain key data frames S1 and auxiliary data frames S2 from the multimedia data 20S according to the importance of the data frames indicated by the encoding configuration policy. The key data frames S1 may include I-frame data or retransmitted frame data determined due to frame transmission errors, and the auxiliary data frames S2 may include P-frame data. Furthermore, the controlled device 210B can use the encoding configuration policy to determine the multimedia data stream L1 (i.e., the first multimedia data stream) corresponding to the key data frame S1 and the multimedia data stream L2 (i.e., the second multimedia data stream) corresponding to the auxiliary data frame S2.

[0095] At this time, the controlled device 210B can send the multimedia data stream L1 to the manipulation device 220B through the first link, and send the multimedia data stream L2 to the manipulation device 220B through the second link. It can be understood that the manipulation device 220B can decode and process the multimedia data stream L1 and the multimedia data stream L2, and render the recovered multimedia data 20S to the data output component of the manipulation device 220B, so that the manipulation device 220B determines the control signaling associated with the multimedia data 20S by responding to the trigger operation performed by the manipulation object (for example, the manipulation user) on the multimedia data 20S. Here, the trigger operation can include touch operations such as clicking and long pressing, and can also include non-contact operations such as voice and gestures, which will not be limited here. For example, the control signaling can instruct the controlled device 210B to turn the vehicle (for example, right turn).

[0096] It should be understood that in order to improve the transmission efficiency of the control signaling, the manipulation device 220B can directly return the control signaling to the controlled device 210B through the first link. Optionally, since the data packet of the control signaling is relatively small, in order to ensure the reliability of the control signaling during data transmission, the manipulation device 220B can also copy the control signaling when determining the control signaling, so as to obtain two control signaling, for example, Figure 2 The control signaling K1 and the control signaling K2 are shown. At this time, the manipulation device 220B can return any one of the two control signaling (for example, the control signaling K1) to the controlled device 210B through the first link, and return the other control signaling (for example, the control signaling K2) to the controlled device 210B through the second link.

[0097] It can be understood that when the controlled device 210B receives the control signaling (the control signaling K1 or the control signaling K2) returned by the manipulation device, the controlled device 210B can perform the control operation indicated by the control signaling. For example, the controlled device 210B controls the vehicle device to turn right.

[0098] It can be seen that, since the priority of the first link corresponding to the first type module (i.e. the module U1 which establishes the first and second bearers with the first area at the same time) is higher than the priority of the second link corresponding to the second type module (i.e. the module U2 which establishes only the second bearer with the second area), when the controlled device 210B is in the first state, the multimedia data stream L1 determined by the key data frame S1 can be transmitted to the control device 220B according to the first link corresponding to the first type module determined by the controlled device 210B. At the same time, the controlled device 210B can also transmit the multimedia data stream L2 determined by the auxiliary data frame S2 to the control device 220B according to the second link corresponding to the second type module determined by the controlled device 220B, so as to reduce the time delay during data transmission, thereby improving the efficiency of data transmission.

[0099] In the embodiments of the present application, when the controlled device determines two types of modules with different priorities (i.e. the first type module and the second type module), the specific implementation manner in which the controlled device transmits the first and second multimedia data streams to the same control device according to the priorities of the links corresponding to the modules can be referred to the following Figures 3-7 Embodiments.

[0100] Further, please refer to Figure 3 , Figure 3 is a flowchart of a data processing method provided by an embodiment of the present application. As Figure 3 shown, the method can be executed by a controlled device, which can be deployed on a vehicle device, for example, the controlled device can be the controlled device 110B in the above-mentioned Figure 1 Embodiments. The method can at least include the following steps S101-S104:

[0101] Step S101, when the service state of the controlled device is in a first state, determining a first type module and a second type module in the communication module of the controlled device.

[0102] In the embodiment, the communication module of the controlled device can include N modules, where N is a positive integer greater than 1. Specifically, when the service state of the controlled device is in the first state, the controlled device can determine N module assistance information corresponding to the N modules. One module corresponds to one module assistance information, and the network state parameter in one module assistance information is determined by the network state of the access area associated with the corresponding module. The network state of the access area is detected based on the geographical location of the controlled device. Further, the controlled device can send the N module assistance information to the remote control management server corresponding to the remote control application through the remote control application running on the controlled device. The remote control management server is used to obtain module establishment information through the communication management system. The module establishment information is generated when the communication management system successfully establishes a first type module based on the network state parameter in each of the N module assistance information. The first type module can be the module corresponding to the module assistance information with the largest network state parameter. Further, the controlled device can receive the module establishment information returned by the remote control management server through the remote control application, and then determine the first type module based on the module establishment information, and determine the modules other than the first type module in the N modules as the second type module. The first type module and the second type module are determined based on the network state parameter (such as signal strength parameter or signal-to-noise ratio parameter) of the geographical location of the controlled device.

[0103] As shown in Figure 1 , the N modules built in the terminal device 10Z in the controlled device 110B can include module U1, module U2, …, and module U N . For ease of description, the number of communication modules in the embodiment can be taken as an example of 2, which can include module U1 and module U2. In the embodiment, the first network state parameter detected by module U1 in the associated access area (for example, the first area) can be referred to as the first network state parameter, and the second network state parameter detected by module U2 in the associated access area (for example, the second area) can be referred to as the second network state parameter. The first area and the second area can be the same area or different areas, which will not be limited here. When the service state of the controlled device 110B is in the first state (for example, the normal transmission state), each module can establish a second bearer with the access area associated with itself. For example, module U1 can establish a second bearer (for example, a default bearer) with the first area, and module U2 can establish a second bearer with the second area.

[0104] Further, each module can respectively send the corresponding module auxiliary information to the remote control application running on the controlled device 110B, so that the controlled device 110B can determine the two module auxiliary information corresponding to the two modules. Wherein, the module auxiliary information here can include network state parameters (for example, RSRP), region identifier of access region, address identifier of module (for example, IP address of module). For example, the module auxiliary information corresponding to the module U1 (i.e. the first module auxiliary information) can include RSRP1, region identifier 1 of the first region, and address identifier 1 of the module U1; the module auxiliary information corresponding to the module U2 can include RSRP2, region identifier 2 of the second region, and address identifier 2 of the module U2. Further, when the remote control management server 10F corresponding to the remote control application receives the two module auxiliary information sent by the controlled device 110B, it can call the module establishment information corresponding to the first type module. Figure 1 As shown in the communication management system 1X (for example, 5G private network management system) of the embodiment, a module is selected in the communication network 1W (for example, 5G private network) to establish a first bearer (for example, dedicated bearer), so as to determine the module establishing the first bearer as a first type module, and generate module establishment information associated with the first type module.

[0105] It can be understood that if the service state of the controlled device 110B is switched from the initial state to the first state, the communication management system 1X can filter the module corresponding to the module auxiliary information with the highest network state parameter in the communication network 1W based on the network state parameter of each module auxiliary information in the two module auxiliary information, so as to establish the first bearer between the filtered module and the associated access region, and then determine the module as the first type module when the establishment is successful, and generate the module establishment information. For example, if RSRP1 (for example, -90dbm) is greater than RSRP2 (for example, -98dbm), at this time, the communication management system can establish the first bearer between the module U1 and the first region, and then determine the module U1 as the first type module when the establishment is successful.

[0106] In this case, the communication management system 1X can ensure the network performance (e.g., bandwidth, latency, reliability, etc.) of the first bearer first, since the priority of the first bearer is higher than that of the second bearer. When the communication management system 1X determines the module U1 as the first type module, the communication management system 1X can configure the service quality parameter (QoS parameter) of the module U1 according to the service requirement configuration strategy. The service quality parameter can include bandwidth, latency, etc. Meanwhile, the communication management system 1X can also modify the 5G network parameter of the module U1 accordingly. For example, in order to reduce the latency, the communication management system 1X can reduce the SR (Scheduling Request, i.e., uplink scheduling request) period of the module U1; for another example, since the terminal device 10Z in the controlled device 110B does not need to consider the energy saving problem, the communication management system 1X can turn off the C-DRX (Connected-mode DRX, i.e., discontinuous reception in connected mode) function, etc.; and since the network status parameter of the module U1 is high, the inter-frequency measurement function of the neighboring cell can be turned off to further reduce the latency.

[0107] Optionally, if the service state of the controlled device 110B is switched from the switching state (e.g., the second state or the third state) to the first state, the communication management system 1X can identify the first type module in the switching state. It can be understood that, when the communication management system 1X identifies the first type module in the switching state as the module U1, it does not need to determine the first type module again, but can directly generate the module establishment information based on the first type module in the switching state. Optionally, when the communication management system 1X identifies the first type module in the switching state as the module U2, the second type module as the module U1, and the communication management system 1X continuously detects that the network status parameter of the module U1 is greater than that of the module U2 within the service detection time (e.g., 1 second), the communication management system 1X can first delete the first bearer established between the module U2 and the associated access area (e.g., the second area), and then establish the first bearer between the module U1 and the associated access area (e.g., the first area), thereby determining the module U1 as the first type module and generating the module establishment information. Optionally, when the communication management system 1X does not identify the first type module, it can also select the module with the highest network status parameter based on the two modules, establish the first bearer between the selected module and the associated access area, and then determine the selected module as the first type module and generate the module establishment information.

[0108] Further, Figure 1The remote control management server shown can obtain the module establishment information generated by the communication management system 1X, and then can send the module establishment information to the remote control application and the controlled application at the same time, so that the remote control application and the controlled application respectively determine the first type module and the second type module in the first state based on the module establishment information. Taking the remote control application as an example, when the controlled device receives the module establishment information through the remote control application, it can determine the first type module based on the module establishment information, and determine the module other than the first type module as the second type module.

[0109] In step S102, the first multimedia data stream and the second multimedia data stream for transmission to the control device are obtained.

[0110] Specifically, the controlled device can collect multimedia data in the geographical location where the controlled device is located in real time through the data collection component of the controlled device. The multimedia data here can include audio data and video data. Further, the controlled device can obtain an encoding configuration strategy associated with the multimedia data, and then determine key data frames in the multimedia data and auxiliary data frames in the multimedia data based on the encoding configuration strategy. At this time, the controlled device can encode the key data frames based on the encoding configuration strategy to obtain the first multimedia data stream for transmission to the control device associated with the controlled device, and encode the auxiliary data frames based on the encoding configuration strategy to obtain the second multimedia data stream for transmission to the control device.

[0111] The encoding configuration strategy here can include an encoding strategy and a service configuration strategy. For example, in the automatic driving scene, the encoding order and the importance of the audio data (such as the alarm sound of special vehicles such as ambulances and fire engines) and the data frame that has transmission errors (i.e. retransmission frame data), the service configuration strategy can indicate that the audio data, I frame data, and retransmission frame data are all key data frames. At this time, the controlled device can determine the I frame data, retransmission frame data, and audio data in the multimedia data as key data frames based on the encoding strategy and the service configuration strategy in the encoding configuration strategy, and then can encode the key data frames based on the encoding strategy in the encoding configuration strategy, thereby obtaining the first multimedia data stream. At the same time, the controlled device can take the data frames (such as P frame data) in the multimedia data other than the key data frames as auxiliary data frames, and then can encode the auxiliary data frames based on the encoding strategy in the encoding configuration strategy to obtain the second multimedia data stream. In this way, when frame transmission errors occur, retransmission frame data can be retransmitted as soon as possible, thereby minimizing the lag time.

[0112] Optionally, when the application scenario is an industrial construction scene, the business configuration policy indicates that the retransmission data is a key data frame, and the audio data (for example, construction noise data, etc.) is an auxiliary data frame. The controlled device can determine the I frame data and the retransmission frame data in the multimedia data as key data frames based on the encoding strategy in the encoding configuration policy and the business configuration policy, and then encode the key data frames based on the encoding strategy in the encoding configuration policy to obtain a first multimedia data stream. At the same time, the controlled device can determine the data frames (for example, P frame data and audio data) in the multimedia data other than the key data frames as auxiliary data frames, and then encode the auxiliary data frames based on the encoding strategy in the encoding configuration policy to obtain a second multimedia data stream.

[0113] As shown in Figure 2 The multimedia data stream L1 (i.e., the first multimedia data stream) is determined by the key data frame S1 in the multimedia data 20S collected by the controlled device 210B; and the multimedia data stream L2 (i.e., the second multimedia data stream) is determined by the auxiliary data frame S2 in the multimedia data 20S.

[0114] In step S103, the first multimedia data stream is sent to the control device through the first link, and the second multimedia data stream is sent to the control device through the second link, so that the control device determines the control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream.

[0115] The priority of the first link corresponding to the first type module is higher than the priority of the second link corresponding to the second type module. Specifically, the controlled device can obtain a network aggregation condition associated with the controlled device, where the network aggregation condition is used to indicate that when the first link corresponding to the first type module cannot meet the transmission demand, the first link and the second link corresponding to the second type module need to be aggregated to ensure the quality of data transmission. It can be understood that when the controlled device does not meet the network aggregation condition, the controlled device can directly send the first multimedia data stream to the control device through the first link, and send the second multimedia data stream to the control device through the second link. At this time, the control device can decode the first multimedia data stream and the second multimedia data stream respectively, and output the recovered multimedia data after decoding to the data output component. For example, the video data is output to the video output component (for example, a display screen) of the control device, and the audio data is output to the audio output component (for example, a microphone) of the control device, so that the control device can determine the control signaling associated with the multimedia data when responding to the trigger operation of the control object to the multimedia data.

[0116] As shown in Figure 2As shown, when the controlled device 210B does not satisfy the network aggregation condition, the controlled device 210B can directly send the multimedia data stream L1 to the control device 220B through the first link (i.e., the communication link corresponding to the first bearer established by the module U1 and the first region), and send the multimedia data stream L2 to the control device 220B through the second link (i.e., the communication link corresponding to the second bearer established by the module U1 and the second region). At this time, the control device 220B can respectively decode and process the multimedia data stream L1 and the multimedia data stream L2, and output the recovered multimedia data 20S after decoding processing to the data output component to determine the control signaling associated with the multimedia data 20S.

[0117] The network aggregation condition can include a first aggregation condition, a second aggregation condition, or a third aggregation condition. The first aggregation condition refers to a network state parameter of the first type module or the second type module decreasing, and the decreased network state parameter being greater than a second network state threshold (e.g., a signal strength parameter of -110 dbm) indicating poor network quality. The second aggregation condition refers to a network load of a geographical location where the controlled device is located being greater than a load threshold. The third aggregation condition refers to uplink interference on the base station side reaching an interference threshold. It can be understood that when the controlled device satisfies any one of the network aggregation conditions, it can be determined that the communication link corresponding to the module in the controlled device cannot meet the sending requirement, such as the network load being too heavy, the network state parameter decreasing but still being greater than the second network state threshold, or the uplink interference on the base station side being large.

[0118] Optionally, when the controlled device satisfies the network aggregation condition, the controlled device can perform network aggregation on the first link and the second link through a remote control application run by the controlled device to obtain an aggregated link. In this embodiment of the present application, the aggregated link obtained by the controlled device when the controlled device is in the first state and the first link and the second link are aggregated can be referred to as a second aggregated link. Further, the controlled device can send the first multimedia data stream and the second multimedia data stream to the control device through the second aggregated link, so that the control device determines the control signaling associated with the multimedia data. It can be understood that the control signaling determined by the control device can also be returned to the controlled device through the second aggregated link.

[0119] For example, when the controlled device meets the network aggregation condition, the controlled device can start the network aggregation function through the remote control application, and then no longer distinguish the transmission of the first multimedia data stream and the second multimedia data stream on the two communication links, but perform network aggregation on the two 5G links based on the real-time quality of the first link and the second link probed, to obtain a second aggregation link, thereby increasing the transmission bandwidth, and thus effectively ensuring the transmission quality of the multimedia data stream. Of course, due to the overhead of network aggregation, the aggregation bandwidth of the second aggregation link is less than the sum of the bandwidths of the two communication links. It can be understood that the smaller the aggregation overhead is, the closer the network aggregation efficiency is to 1. The network aggregation algorithm for aggregating the first link and the second link in the embodiments of the present application is not limited here. If the access areas associated with the first type module and the second type module are the same area, using the communication modules of the two module types at the same time will bring the essential reason of uplink gain. At this time, the embodiments of the present application can regard the communication modules of the two module types as a virtual terminal, so as to increase the uplink transmission power and the number of uplink transmission antennas.

[0120] Step S104, when the controlled device receives the control signaling, the control operation indicated by the control signaling is performed.

[0121] Specifically, since the data packet of the control signaling is small, in order to effectively ensure the efficiency and reliability of the control signaling, the control device in the embodiments of the present application can copy the control signaling associated with the multimedia data through the control application in the control device when generating the control signaling, to obtain two identical control signalings, for example, a first control signaling and a second control signaling. It should be understood that the controlled device can receive the first control signaling returned by the control device to the remote control application of the controlled device through the first link, and at the same time, the controlled device can also receive the second control signaling returned by the control device to the remote control application through the second link. Further, the controlled device can perform deduplication processing on the first control signaling and the second control signaling through the remote control application to obtain a target control signaling, and then perform the control operation indicated by the target control signaling. Wherein, the target control instruction here can be the first control signaling or the second control signaling.

[0122] As shown in Figure 2 When the control device 220B copies the control signaling, the control device 220B can send the first control signaling to the controlled device 220A through the first link, and send the second control signaling to the controlled device 220A through the second link. Figure 2When the control signaling K1 and the control signaling K2 are shown, the control device 220B can return any one of the two control signalings (i.e., the first control signaling, for example, the control signaling K1) to the controlled device 210B through the first link, and return the other one of the two control signalings (i.e., the second control signaling, for example, the control signaling K2) to the controlled device 210B through the second link. When the controlled device 210B receives the two control signalings (the control signaling K1 and the control signaling K2) returned by the control device, the controlled device 210B can perform deduplication processing on the control signaling K1 and the control signaling K2 through the remote control application to obtain a target control signaling (for example, the control signaling K1 or the control signaling K2), and then can execute the control operation indicated by the target control signaling. For example, control the vehicle device where the controlled device 210B is located to turn right.

[0123] In the embodiment of the present application, since the priority of the first link corresponding to the first type module is higher than the priority of the second link corresponding to the second type module, when the controlled device is in the first state, the first multimedia data stream determined by the key data frame and obtained can be sent to the control device according to the first link corresponding to the first type module determined by the controlled device. At the same time, the second multimedia data stream determined by the auxiliary data frame and obtained can also be sent to the control device according to the second link corresponding to the second type module determined by the controlled device, so as to realize the differentiated transmission of the multimedia data streams (the first multimedia data stream and the second multimedia data stream) corresponding to the multimedia data on the first link and the second link, so as to provide differentiated data guarantee capability, thereby reducing the time delay during data transmission, and further improving the efficiency of data transmission.

[0124] Further, please refer to Figure 4 , Figure 4 is a flow diagram corresponding to the controlled device in the first state provided by the embodiment of the present application. As Figure 4 shown, the method involves a controlled device, a control device and a remote control management server. The controlled device can be deployed on a vehicle device, for example, the controlled device can be the controlled device 110B in the embodiment corresponding to the above Figure 1 The control device can be deployed in the cockpit of the vehicle device where the controlled device is located, or can be deployed in a location area (for example, a remote control room) having a location association relationship with the vehicle device, which will not be limited here. The control device can be the controlled device 120B in the embodiment corresponding to the above Figure 1 The remote control management server can be a server corresponding to the remote control application operated by the controlled device, and the remote control management server can be the remote control management server 130B in the embodiment corresponding to the above Figure 1The remote control management server 10F in the corresponding embodiment. The method can at least include the following steps S201-S208:

[0125] Step S201, when the service state of the controlled device is the first state, the controlled device sends N module auxiliary information corresponding to N modules to the remote control management server.

[0126] Specifically, when the service state of the controlled device is the first state, the controlled device can obtain the module auxiliary information corresponding to each of the N modules in the communication module through the remote control application running on the controlled device. The module auxiliary information can include network state parameters, area identifier of the access area, and address identifier of the module (e.g., IP address of the module). N is a positive integer greater than 1. One module corresponds to one module auxiliary information, and the network state parameter in one module auxiliary information is determined by the network state of the access area associated with the corresponding module. The network state of the access area is detected based on the geographical location of the controlled device.

[0127] Step S202, the remote control management server obtains module establishment information by calling the communication management system.

[0128] Specifically, when the remote control management server corresponding to the remote control application receives the N module auxiliary information, it can call the communication management system to select a module and the access area associated with it to establish a first bearer in the communication network, and then determine the module with the first bearer as the first type module and generate module establishment information associated with the first type module. Further, the remote control management server can obtain the module establishment information generated by the communication management system.

[0129] Step S203, the remote control management server returns the module establishment information to the controlled device.

[0130] Specifically, the remote control management server can return the module establishment information to the remote control application in the controlled device.

[0131] Step S204, the controlled device determines the first type module and the second type module based on the module establishment information.

[0132] Specifically, the controlled device can determine the first type module based on the module establishment information, and determine the modules other than the first type module in the N modules as the second type module. The first type module and the second type module are determined based on the network state parameters of the geographical location of the controlled device.

[0133] Step S205, the controlled device obtains the first multimedia data stream and the second multimedia data stream for transmission to the control device.

[0134] Specifically, the controlled device can acquire multimedia data of a geographical location where the controlled device is located in real time through a data acquisition component of the controlled device. Here, the multimedia data can include audio data and video data. Further, the controlled device can obtain an encoding configuration policy associated with the multimedia data, and then determine key data frames in the multimedia data and auxiliary data frames in the multimedia data based on the encoding configuration policy. At this time, the controlled device can encode the key data frames based on the encoding configuration policy to obtain a first multimedia data stream for transmission to a control device associated with the controlled device, and encode the auxiliary data frames based on the encoding configuration policy to obtain a second multimedia data stream for transmission to the control device.

[0135] In step S206, the controlled device sends the first multimedia data stream to the control device through the first link and sends the second multimedia data stream to the control device through the second link.

[0136] Specifically, the priority of the first link corresponding to the first type of module is higher than the priority of the second link corresponding to the second type of module. Specifically, the controlled device can obtain a network aggregation condition associated with the controlled device, where the network aggregation condition is used to indicate that when the first link corresponding to the first type of module cannot meet the transmission demand, the first link and the second link corresponding to the second type of module need to be aggregated to ensure the quality of data transmission. It can be understood that when the controlled device does not meet the network aggregation condition, the controlled device can directly send the first multimedia data stream to the control device through the first link and send the second multimedia data stream to the control device through the second link.

[0137] In step S207, the control device determines control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream.

[0138] Specifically, the control device can decode the first multimedia data stream and the second multimedia data stream respectively, and output the recovered multimedia data after decoding to a data output component. For example, video data is output to a video output component (e.g., a display screen) of the control device, and audio data is output to an audio output component (e.g., a microphone) of the control device, so that the control device can determine the control signaling associated with the multimedia data in response to a trigger operation of the control object on the multimedia data.

[0139] In step S208, the control device returns the control signaling to the controlled device to make the controlled device perform a control operation indicated by the control information.

[0140] Specifically, since the data packet of the control signaling is relatively small, in order to effectively ensure the efficiency and reliability of the control signaling, the control device in the embodiments of the present application can copy the control signaling associated with the multimedia data by the control application in the control device when generating the control signaling, to obtain two identical control signalings, for example, a first control signaling and a second control signaling. It should be understood that the controlled device can receive the first control signaling returned by the control device to the remote control application of the controlled device through the first link, and at the same time, the controlled device can also receive the second control signaling returned by the control device to the remote control application through the second link. Further, the controlled device can perform deduplication processing on the first control signaling and the second control signaling by the remote control application to obtain a target control signaling, and then can execute the control operation indicated by the target control signaling. Wherein, the target control instruction here can be the first control signaling or the second control signaling.

[0141] The specific implementation of steps S201-S208 can be referred to the description of steps S101-S104 in the above Figure 3 The description of steps S101-S104 in the corresponding embodiments will not be repeated here.

[0142] It should be understood that the controlled device in the embodiments of the present application can obtain the network state parameters of each module in the communication module of the controlled device, and then can detect the heartbeat state between the remote control application in the controlled device and the control application in the control device for each module. Wherein, the remote control application and the control application periodically send heartbeat packets through each module, on the one hand, the periodic sending of the heartbeat packets can detect the communication state of each module in real time, and when the hardware or software failure causes the network to be disconnected, the disaster recovery processing can be performed in time, that is, entering the processing flow corresponding to the abnormal state (i.e. the fourth state); on the other hand, especially for the first type module in the current business state, the periodic sending of the heartbeat packets can effectively avoid the first type module from being released into the idle state by the network due to no data transmission, thereby reducing the transmission delay of the first type module when data suddenly needs to be transmitted.

[0143] Further, the controlled device can acquire a service condition associated with the controlled device. The service condition can include a first service condition, a second service condition, a third service condition, a fourth service condition, and a fifth service condition. The first service condition is used to indicate that the controlled device switches to a first state (e.g., a normal transmission state). For example, the first service condition is that the network state parameter of each module is greater than a first network state threshold (e.g., a signal strength parameter is -100dbm) within a first service detection time (e.g., 1 second), and the heartbeat state of each module is in a normal state. The normal state is determined by the remote control application when the heartbeat packet is periodically sent to the control application. The second service condition is used to indicate that the controlled device switches to a second state (e.g., a switching state H1). The second state is a state in which a first bearer between a certain module (e.g., a second type module) and an associated access area is re-established according to the network state parameter of each module. The second service condition can include a first sub-condition and a second sub-condition. For example, the first sub-condition is that the network state parameter of each module is less than a second network state threshold (e.g., a signal strength parameter is -110dbm) within a second service detection time, and the heartbeat state of each module is in a normal state. The second sub-condition is that the network state parameter of a current first type module in the communication module is greater than the first network state threshold within a third service detection time, the network state parameter of a current second type module is less than the second network state threshold within the third service detection time, and the heartbeat state of each module is in a normal state. The third service condition is used to indicate that the controlled device switches to a third state (e.g., a switching state H2). The third state is a state in which a certain module (e.g., a module with a network state parameter less than a second threshold) is associated with a reconfigured access area, and a second bearer between the module and the new access area is established according to the network state parameter of each module. For example, the third service condition is that the network state parameter of a certain module in the communication module is greater than the first threshold, while the network state parameter of other modules is less than the second threshold, and the heartbeat state of each module is in a normal state. The fourth service condition is used to indicate that the controlled device switches to a fourth state (e.g., an abnormal state). For example, the fourth service condition is that there is a module in a fault state in the communication module of the controlled device. The fault state is a state in which the network state parameter or the heartbeat state is reported as an abnormal state within a fifth service detection time. The fifth service condition is used to indicate that the controlled device switches from the fourth state to the first state. For example, the fifth service condition is that the module in the fault state resumes normal communication within a sixth service detection time, and the heartbeat state of the module is in a normal state.The first service detection duration, the second service detection duration, the third service detection duration, the fourth service detection duration, the fifth service detection duration, and the sixth service detection duration in the embodiments of the present application can be dynamically adjusted according to actual conditions, and can be the same as or different from each other, which will not be limited here. The second network state threshold (used to indicate the threshold of the poor network state) in the embodiments of the present application is less than the first network state threshold (used to indicate the threshold of the excellent network state), and both of the two network state thresholds can be dynamically adjusted according to actual conditions, which will not be limited here.

[0144] It can be understood that the controlled device can determine the first network state threshold in the first service condition and the first service detection duration from the service condition. When the network state parameter of each module is greater than the first network state threshold within the first service detection duration, and the heartbeat state of each module is in the normal state, it can be determined that the controlled device meets the first service condition, at this time, the service state of the controlled device can be switched to the first state, and then the processing flow corresponding to the first state can be executed, that is, the specific embodiments of steps S201-S208 or steps S101-S104. It can be understood that the service state of the controlled device in the embodiments of the present application is determined according to the network state parameter detected by each module at the geographical position of the controlled device and the service condition.

[0145] For ease of understanding, further, please refer to Figure 5 , Figure 5 is a state switching schematic diagram for a controlled device provided by the embodiments of the present application. As Figure 5 indicated, the communication module in the controlled device in the embodiments of the present application can include N modules, where N is a positive integer greater than 1. For ease of description, the first type module in the embodiments of the present application can be taken as an example of 2, which can specifically include module U1 and module U2. For example, the controlled device can be the controlled device 210B in the embodiment corresponding to Figure 2 It can be understood that the service condition obtained by the controlled device can be as shown in Figure 5 , which can specifically include condition 51c, condition 52c, and condition 54c, condition 53c, condition 55c, and condition 56c.

[0146] It should be understood that when the modules U1 and U2 of the controlled device independently perform the selection, access and registration procedures of the access area (for example, a 5G area) according to the standard procedure, the application embodiment can regard the service state of the controlled device 110B as an initial state. For example, the access area associated with the module U1 is a first area, and the access area associated with the module U2 is a second area. The first area and the second area can be the same area or different areas, which will not be limited here. It can be understood that the application embodiment can regard the network state parameter (for example, RSRP1) detected by the module U1 as a first network state parameter, the heartbeat state of the module U1 as a first heartbeat state, the network state parameter (for example, RSRP2) detected by the module U2 as a second network state parameter, and the heartbeat state of the module U2 as a second heartbeat state based on the geographical location of the controlled device.

[0147] It should be understood that the controlled device can obtain a first service condition (that is, condition 51c) from the service conditions associated with the controlled device. Figure 5 The first network state threshold in condition 51c) and the first service detection duration are shown. When the first network state parameter and the second network state parameter are both greater than the first network state threshold within the first service detection duration, and the first heartbeat state and the second heartbeat state are both in a normal state, the controlled device can determine that it satisfies condition 51c. At this time, the controlled device can switch its service state to a first state, and then can perform the processing procedure corresponding to the first state, that is, the specific embodiments of steps S201-S208 or steps S101-S104. If the service state of the controlled device is the first state, it can be understood that the current geographical location of the controlled device is closer to the first area.

[0148] For example, if the current service state of the controlled device is the initial state, and the controlled device satisfies condition 51c, the controlled device can switch its service state from the initial state to the first state. Optionally, if the current service state of the controlled device is the second state, and the controlled device satisfies condition 51c, the controlled device can switch its service state from the second state to the first state. Optionally, if the current service state of the controlled device is the third state, and the controlled device satisfies condition 51c, the controlled device can switch its service state from the third state to the first state. When the service state of the controlled device is in the first state, if the first network state parameter is greater than the second network state parameter, the first type module determined by the controlled device is the module U1, and the second type module is the module U2.

[0149] It can be understood that the controlled device can obtain a second service condition (for example, condition 51d) from the service conditions associated with the controlled device. Figure 5The condition 52c or the condition 54c) is shown, at this time, the controlled device can determine that the controlled device meets the second service condition based on the first network state parameter, the second network state parameter, the first heartbeat state and the second heartbeat state, switch the service state of the controlled device from the first state to the second state, and then can execute the processing flow corresponding to the second state. Wherein, if the service state of the controlled device is the second state, it can be understood that the current geographical position of the controlled device is located in the critical region between the first region and the second region.

[0150] For example, if the first network state parameter and the second network state parameter are less than the second network state threshold within the second service detection time length, and the first heartbeat state and the second heartbeat state are in the normal state, it can be determined that the controlled device meets the condition 52c (i.e. the first sub condition), at this time, the controlled device can switch the service state of itself from the first state to the second state. For another example, if the first network state parameter is greater than the first network state threshold within the third service detection time length, the second network state parameter is less than the second network state threshold within the third service detection time length, and the first heartbeat state and the second heartbeat state are in the normal state, the controlled device can determine that itself meets the condition 54c (i.e. the second sub condition), at this time, the controlled device can switch the service state of itself from the first state to the second state. Figure 5 It can be understood that the controlled device can obtain the third service condition (i.e. the condition 53c) from the service condition associated with the controlled device.

[0151] The condition 53c is shown, the condition 53c can include the first network state threshold, the second network state threshold and the fourth service detection time length; the second network state threshold is less than the first network state threshold. If the first network state parameter (i.e. the network state parameter of the module U1) is less than the second network state threshold within the fourth service detection time length, the second network state parameter (i.e. the network state parameter of the module U2) is greater than the first network state threshold within the fourth service detection time length, and the heartbeat state of the module U1 and the heartbeat state of the module U2 are in the normal state, the controlled device can determine that itself meets the condition 53c (i.e. the third service condition). Figure 5 The condition 53c is shown, the condition 53c can include the first network state threshold, the second network state threshold and the fourth service detection time length; the second network state threshold is less than the first network state threshold. If the first network state parameter (i.e. the network state parameter of the module U1) is less than the second network state threshold within the fourth service detection time length, the second network state parameter (i.e. the network state parameter of the module U2) is greater than the first network state threshold within the fourth service detection time length, and the heartbeat state of the module U1 and the heartbeat state of the module U2 are in the normal state, the controlled device can determine that itself meets the condition 53c (i.e. the third service condition). Figure 5 The condition 53c is shown, the condition 53c can include the first network state threshold, the second network state threshold and the fourth service detection time length; the second network state threshold is less than the first network state threshold. If the first network state parameter (i.e. the network state parameter of the module U1) is less than the second network state threshold within the fourth service detection time length, the second network state parameter (i.e. the network state parameter of the module U2) is greater than the first network state threshold within the fourth service detection time length, and the heartbeat state of the module U1 and the heartbeat state of the module U2 are in the normal state, the controlled device can determine that itself meets the condition 53c (i.e. the third service condition).

[0152] For example, the controlled device can determine the current service state as the target state, such as, when the service state of the controlled device is the first state, the controlled device can determine the first state as the target state, when the controlled device satisfies the condition 53c, the controlled device can directly switch the service state of the controlled device from the first state to the third state. Or when the first state of the controlled device is switched to the second state, the controlled device can determine the second state as the target state, when the controlled device satisfies the condition 53c, the controlled device can switch the service state of the controlled device from the second state to the third state.

[0153] It can be understood that the controlled device can obtain the fourth service condition (for example, the condition 55c shown in the figure) from the service conditions associated with the controlled device. Wherein, the condition 55c can include a fifth service detection time length. If there is a module in the fault state in the module U1 and the module U2, the controlled device can determine that the controlled device satisfies the fourth service condition, and determine the module in the fault state as the fault module, and switch the service state of the controlled device to the fourth state. Wherein, the fault state here refers to the state when the network state parameter is not reported within the fifth service detection time length or the heartbeat state is an abnormal state. Figure 5

[0154] For example, if the current service state of the controlled device is the first state, and the controlled device satisfies the condition 55c, the controlled device can switch the service state of the controlled device from the first state to the fourth state. Optionally, if the current service state of the controlled device is the second state, and the controlled device satisfies the condition 55c, the controlled device can switch the service state of the controlled device from the second state to the fourth state. Optionally, if the current service state of the controlled device is the third state, and the controlled device satisfies the condition 55c, the controlled device can switch the service state of the controlled device from the third state to the fourth state.

[0155] ​Furthermore, the controlled device can execute the processing flow corresponding to the fourth state. It can be understood that when the service state of the controlled device is the fourth state, the controlled device can use the modules other than the faulty module in module U1 and module U2 as normal modules. If the module type corresponding to the normal module is a first type module, that is, the normal module establishes the first bearer and the second bearer with the associated access area at the same time, then the controlled device does not need to repeatedly establish the first bearer between the normal module and the associated access area. Optionally, if the module type corresponding to the normal module is a second type module, the controlled device needs to change the module type corresponding to the normal module from the second type module to the first type module based on the access area associated with the normal module, that is, the normal module needs to establish the first bearer again with the associated access area on the basis of establishing the second bearer. In this way, all data to be transmitted can be transmitted through the communication link corresponding to the establishment of the first bearer by the normal module and the associated access area, thereby effectively ensuring that data transmission is not interrupted, thereby improving the reliability of data transmission.

[0156] It is understandable that the controlled device may also obtain the fifth service condition (eg, Figure 6 Condition 56c shown). Condition 56c may include a sixth service detection duration. If the faulty module determined by the controlled device in the fourth state resumes normal communication within the sixth service detection duration, and the heartbeat status of the faulty module that has resumed normal communication is normal, the faulty module that has resumed normal communication may establish a second bearer with the associated access zone. At this point, the controlled device may switch its service state from the fourth state to the first state.

[0157] For further understanding, please refer to Figure 6 , Figure 6 This is a flow chart corresponding to a controlled device in the second state provided by an embodiment of the present application. Figure 6 As shown, in the embodiment of the present application, the first type of module in the controlled device includes module U1 in the communication module; the second type of module includes module U2 in the communication module. Module U1 is associated with the first access area, while module U2 is associated with the second access area, and the first area is different from the second area. The first link refers to the communication link corresponding to module U1. Module U1 is of the first type, which is associated with the first area.

[0158] It should be understood that if the service state of the controlled device is the second state, it can be understood that the current geographical location of the controlled device is located in the critical region between the first region and the second region, that is, it is possible that the network state parameters (for example, signal strength parameters) of the module U1 and the module U2 are both too low due to the geographical location, so that the case of two modules simultaneously configuring the associated access region exists. In order to effectively prevent the occurrence of this case, when the service state of the controlled device is switched to the second state, the controlled device can execute Figure 3 As shown in step S61, the access region (that is, the first region) associated with the module U1 is locked. Further, the controlled device can execute step S62, and send a parameter configuration notification associated with the module U2 to the remote control management server corresponding to the remote control application running on the controlled device through the remote control application, so that the remote control management server executes step S63, invokes the communication management system, and forwards the parameter configuration notification to the communication management system.

[0159] At this time, the communication management system can execute step S64, and perform parameter configuration on the second region associated with the module U2 based on the parameter configuration notification, and start the handover measurement function. The handover measurement function here can include same frequency and different frequency measurement. For example, the communication management system can configure the related offset parameter to be an offset threshold (for example, 10db) based on the access region (that is, the second region) associated with the module U2, so that the module U2 will detect other regions to be 10db lower than the second region, which means that the module U2 is more likely to access the second region.

[0160] For example, the transmission power associated with the second region is -100dbm, and the related offset parameter for the second region is 10db, which means that the actual signal strength parameter detected by the module U2 in the second region is -90dbm. Based on this, the communication management system can establish a first bearer between the module U2 and the second region based on the second region associated with the module U2. The specific implementation of the communication management system configuring the quality of service parameter of the module U2 when establishing the first bearer of the module U2 can be referred to in the above Figure 7 The specific implementation of configuring the quality of service parameter of the module U1 will not be described here. Further, the communication management system can change the module type corresponding to the module U2 from the second type module to the first type module, and generate module update information when the establishment is successful. This means that when the controlled device is in the second state, the first type module in the communication module can include the module U1 and the module U2, that is, the module U1 and the first region simultaneously establish the first bearer and the second bearer, and the module U2 and the second region simultaneously establish the first bearer and the second bearer.

[0161] Further, the communication management system can perform step S65, return the module update information to the remote control management server, so that the remote control management server performs step S66, returns the module update information to the controlled device. At this time, the controlled device can perform step S67, based on the module update information and the first type module associated with the second area, determines the communication link corresponding to the module U2 as the third link, that is, the communication link corresponding to the module U2 when the module U2 establishes the first bearer with the second area. Since the network state parameters of the module U1 and the module U2 at this time are both too low, further causing the first link corresponding to the module U1 and the third link corresponding to the module U2 to be poor, at this time, the controlled device can perform step S68, through the remote control application, network aggregation is performed on the first link and the third link, so that an aggregated link can be obtained; wherein, the first aggregated link obtained by the controlled device after network aggregation is performed on the first link and the third link in the second state is referred to as the first aggregated link. The first aggregated link is used to transmit the first multimedia data stream, the second multimedia data stream, and the control signaling returned by the control device when the service state of the controlled device is the second state.

[0162] For ease of understanding, further, please refer to Figure 7 , Figure 7 is a flow diagram corresponding to the controlled device in the third state provided by an embodiment of the present application. As Figure 8 indicated, the communication module in the controlled device in the embodiment of the present application includes N modules; N is a positive integer greater than 1; the N modules include a module U1 and a module U2; wherein, the access area associated with the module U1 is a first area, and the access area associated with the module U2 is a second area; the first area is different from the second area.

[0163] It should be understood that if the service state of the controlled device is the third state, it can be understood that the current geographical position of the controlled device is closer to the second area. At this time, the controlled device can first determine the module type corresponding to the module U2 associated with the second area, and when the module type corresponding to the module U2 is the first type module (that is, the first bearer has been established with the second area), the controlled device needs to generate a region switching notification for the module U1. Optionally, when the module type corresponding to the module U2 is the second type module (that is, the first bearer has not been established with the second area), the controlled device can perform step S71, change the module type corresponding to the module U2, that is, based on the second area associated with the module U2, establish the first bearer of the module U2, and change the module type corresponding to the module U2 from the second type module to the first type module.

[0164] Further, the controlled device can determine whether the access area associated with the module U1 is locked. If the module U1 is locked with the first area, the access area associated with the module U1 needs to be unlocked for subsequent normal area switching. At this time, the controlled device can generate an area switching notification for the module U1 based on the access area (i.e. the second area) associated with the module U2, and then can perform step S72 to send the area switching notification to the remote control management server through the remote control application in the controlled device, so that the remote control management server performs step S73 to send the area switching notification to the communication management system.

[0165] At this time, the communication management system can perform step S74 to configure the access area associated with the module U1 from the first area to the second area associated with the module U2. For example, the communication management system can obtain the second area associated with the module U2 indicated by the area switching notification, configure the access parameters of the module U1 to assist the module U1 to switch from the first area to the second area faster, and establish a second bearer between the module U1 and the second area when the switching is successful, and generate area switching information associated with the module U1.

[0166] Further, the communication management system can perform step S75 to return the area switching information to the remote control management server, so that the remote control management server performs step S76 to return the area switching information to the controlled device. At this time, the controlled device can perform step S67 to change the module type corresponding to the module U1 from the first type module to the second type module based on the second area associated with the module U1 when receiving the area switching information. It can be understood that when the controlled device is in the third state, in the uplink direction (i.e. the direction from the controlled device to the controlled device), the remote control application can send the first multimedia data stream corresponding to the multimedia data collected by the controlled device to the control application through the third link of the module U2 (i.e. the communication link corresponding to the first bearer established between the module U2 and the second area), and send the second multimedia data stream corresponding to the multimedia data through the fourth link of the module U1 (i.e. the communication link corresponding to the second bearer established between the module U1 and the second area). In the downlink direction (i.e. the direction from the control device to the controlled device), the control application can copy the determined control signaling and return the copied control signaling to the remote control application through the third link and the fourth link respectively.

[0167] In the embodiment of the present application, since the priority of the first link corresponding to the first type module is higher than the priority of the second link corresponding to the second type module, when the controlled device is in the first state, the first multimedia data stream determined by the key data frame can be transmitted to the control device according to the first link corresponding to the first type module determined by the controlled device. At the same time, the second multimedia data stream determined by the auxiliary data frame can also be transmitted to the control device according to the second link corresponding to the second type module determined by the controlled device, so as to realize the differentiated transmission of the multimedia data streams (the first multimedia data stream and the second multimedia data stream) corresponding to the multimedia data on the first link and the second link, so as to provide differentiated data guarantee capability, thereby reducing the time delay during data transmission, and further improving the efficiency during data transmission. In addition, since the first multimedia data stream and the second multimedia data stream are both used to transmit to the control application in the control device, this will cause the remote control application in the controlled device to realize different quality of service guarantees for different multimedia data streams of the same IP address (i.e. the IP address of the control application), so as to improve the performance of the remote control application. Of course, since the controlled device in the embodiment of the present application can establish the first bearer between a certain module and the associated access area in the first state, the second state, the third state or even the fourth state (abnormal state), it means that when the geographical location of the controlled device changes, i.e. the communication module needs to switch the access area, the communication can be guaranteed not to be interrupted, and the video can be guaranteed not to be stuck, thereby greatly improving the reliability during data transmission.

[0168] Further, please refer to Figure 8 , Figure 8 is a structural schematic diagram of a data processing apparatus provided in an embodiment of the present application. As shown in Figure 8 , the data processing apparatus 1 can be a computer program (including program code) running in a computer device, for example, the data processing apparatus 1 is an application software; the data processing apparatus 1 can be used to execute the corresponding steps in the method provided in the embodiment of the present application. As shown in Figure 1 , the data processing apparatus 1 can run in a controlled device, which can be the above-mentioned Figure 3The controlled device 110B in the corresponding embodiment. The data processing apparatus 1 can comprise: a module type determination module 11, a multimedia data stream acquisition module 12, a first data stream sending module 13, a control operation execution module 14, a signal strength acquisition module 15, a heartbeat state detection module 16, a first service condition acquisition module 17, a first state switching module 18, a parameter determination module 19, a second service condition acquisition module 20, a second state switching module 21, a region locking module 22, a parameter configuration notification sending module 23, a module update information receiving module 24, a network aggregation module 25, a target state determination module 26, a third service condition acquisition module 27, a third state switching module 28, a communication module type changing module 29, a region switching notification sending module 30, a region switching information receiving module 31, a fourth service condition acquisition module 32, a faulty module determination module 33, a normal module determination module 34, a normal module type changing module 35, a network aggregation condition acquisition module 36, and a second data stream sending module 37.

[0169] The module type determination module 11 is configured to determine a first type module and a second type module in the communication module of the controlled device when the service state of the controlled device is in a first state; the first type module and the second type module are determined based on a network state parameter of a geographic location where the controlled device is located; and the priority of a first link corresponding to the first type module is higher than the priority of a second link corresponding to the second type module.

[0170] The communication module comprises N modules; N is a positive integer greater than 1.

[0171] The module type determination module 11 comprises a module auxiliary information determination unit 111, a module auxiliary information sending unit 112, and a module establishment information receiving unit 113.

[0172] The module auxiliary information determination unit 111 is configured to determine N module auxiliary information corresponding to N modules when the service state of the controlled device is in a first state; one module corresponds to one module auxiliary information, and the network state parameter in one module auxiliary information is determined by the network state of an access region associated with the corresponding module; and the network state of the access region is detected based on the geographic location where the controlled device is located.

[0173] The module auxiliary information sending unit 112 is configured to send the N module auxiliary information to a remote control management server corresponding to the remote control application by a remote control application run by the controlled device; the remote control management server is configured to acquire module establishment information by a communication management system; the module establishment information is generated by the communication management system when a first type module is successfully established based on a network state parameter in each of the N module auxiliary information; the first type module is a module corresponding to the module auxiliary information with the largest network state parameter;

[0174] The module establishment information receiving unit 113 is configured to receive the module establishment information by the remote control application, determine the first type module based on the module establishment information, and determine the modules other than the first type module in the N modules as the second type module.

[0175] The specific implementation of the module auxiliary information determining unit 111, the module auxiliary information sending unit 112, and the module establishment information receiving unit 113 can be referred to the description of the corresponding embodiments of the step S101, which will not be repeated here. Figure 3 The specific implementation of the module auxiliary information determining unit 111, the module auxiliary information sending unit 112, and the module establishment information receiving unit 113 can be referred to the description of the corresponding embodiments of the step S101, which will not be repeated here.

[0176] The multimedia data stream acquisition module 12 is configured to acquire a first multimedia data stream and a second multimedia data stream for transmission to a control device; the first multimedia data stream is determined by key data frames in multimedia data collected by the controlled device; and the second multimedia data stream is determined by auxiliary data frames in the multimedia data.

[0177] The multimedia data stream acquisition module 12 includes a multimedia data acquisition unit 121, an encoding configuration strategy acquisition unit 122, a key frame encoding processing unit 123, and an auxiliary frame encoding processing unit 124.

[0178] The multimedia data acquisition unit 121 is configured to acquire multimedia data of a geographical location where the controlled device is located by a data acquisition component of the controlled device.

[0179] The encoding configuration strategy acquisition unit 122 is configured to acquire an encoding configuration strategy associated with the multimedia data, and determine key data frames in the multimedia data and auxiliary data frames in the multimedia data based on the encoding configuration strategy.

[0180] The key frame encoding processing unit 123 is configured to perform encoding processing on the key data frames based on the encoding configuration strategy to obtain the first multimedia data stream for transmission to the control device associated with the controlled device.

[0181] The auxiliary frame encoding processing unit 124 is configured to perform encoding processing on the auxiliary data frames based on the encoding configuration strategy to obtain the second multimedia data stream for transmission to the control device.

[0182] The specific implementation of the multimedia data acquisition unit 121, the encoding configuration strategy acquisition unit 122, the key frame encoding processing unit 123, and the auxiliary frame encoding processing unit 124 can be referred to the above Figure 3 The description of step S102 in the corresponding embodiment will not be repeated here.

[0183] The first data stream sending module 13 is configured to send the first multimedia data stream to the control device through the first link and send the second multimedia data stream to the control device through the second link, so that the control device determines the control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream.

[0184] The control operation execution module 14 is configured to execute the control operation indicated by the control signaling when the controlled device receives the control signaling.

[0185] The control operation execution module 14 includes a first signaling receiving unit 141, a second signaling receiving unit 142, and a signaling deduplication processing unit 143.

[0186] The first signaling receiving unit 141 is configured to receive, through the first link, the first control signaling returned by the control device to the remote control application of the controlled device.

[0187] The second signaling receiving unit 142 is configured to receive, through the second link, the second control signaling returned by the control device to the remote control application; the second control signaling is obtained by copying the first control signaling by the control application in the control device.

[0188] The signaling deduplication processing unit 143 is configured to perform deduplication processing on the first control signaling and the second control signaling by the remote control application to obtain target control signaling, and execute the control operation indicated by the target control signaling; the target control instruction is the first control signaling or the second control signaling.

[0189] The specific implementation of the first signaling receiving unit 141, the second signaling receiving unit 142, and the signaling deduplication processing unit 143 can be referred to the above Figure 5 The description of step S104 in the corresponding embodiment will not be repeated here.

[0190] The signal strength acquisition module 15 is configured to acquire the network state parameter of each module in the communication module of the controlled device.

[0191] The heartbeat state detection module 16 is configured to detect the heartbeat state between the remote control application in the controlled device and the control application in the control device for each module, respectively.

[0192] The first service condition obtaining module 17 is configured to obtain a service condition associated with the controlled device, and determine a first network state threshold and a first service detection duration in the first service condition from the service condition.

[0193] The first state switching module 18 is configured to determine that the controlled device satisfies the first service condition when the network state parameters of each module are all greater than the first network state threshold within the first service detection duration, and the heartbeat states of each module are all in a normal state, and switch the service state of the controlled device to a first state; the normal state is determined by the remote control application when the heartbeat packet is periodically sent to the control application.

[0194] The communication module includes N modules; N is a positive integer greater than 1; the first type module in the N modules includes a module U1; the second type module in the N modules includes a module U2.

[0195] The parameter determination module 19 is configured to take the network state parameter of the module U1 as a first network state parameter, take the heartbeat state of the module U1 as a first heartbeat state, take the network state parameter of the module U2 as a second network state parameter, and take the heartbeat state of the module U2 as a second heartbeat state.

[0196] The second service condition obtaining module 20 is configured to obtain a second service condition from the service condition associated with the controlled device.

[0197] The second state switching module 21 is configured to switch the service state of the controlled device from the first state to a second state when it is determined that the controlled device satisfies the second service condition based on the first network state parameter, the second network state parameter, the first heartbeat state, and the second heartbeat state.

[0198] The second service condition includes a first sub-condition; the first sub-condition includes a second network state threshold and a second service detection duration.

[0199] The second state switching module 21 includes a first switching unit 211 and a second switching unit 212.

[0200] The first switching unit 211 is configured to determine that the controlled device satisfies the first sub-condition and switch the service state of the controlled device from the first state to the second state if the first network state parameter and the second network state parameter are both less than the second network state threshold within the second service detection duration, and the first heartbeat state and the second heartbeat state are both in the normal state.

[0201] The second service condition includes a second sub-condition; the second sub-condition includes the first network state threshold, the second network state threshold, and a third service detection duration; the second network state threshold is less than the first network state threshold.

[0202] The second switching unit 212 is configured to determine that the controlled device satisfies a second sub-condition if the first network status parameter is greater than the first network status threshold within the third service detection duration, the second network status parameter is less than the second network status threshold within the third service detection duration, and the first heartbeat status and the second heartbeat status are both in the normal state, and switch the service status of the controlled device from the first state to a second state.

[0203] The specific implementation of the first switching unit 211 and the second switching unit 212 can be referred to the above description of the first switching unit 211 and the second switching unit 212. Figure 4 The description of switching to the second state in the corresponding embodiment will not be repeated here.

[0204] The first type of module includes module U1 in the communication module; the second type of module includes module U2 in the communication module; the access area associated with module U1 is the first area, and the access area associated with module U2 is the second area; the first area is different from the second area; the first link refers to the communication link corresponding to module U1; the module type of module U1 is the first type of module associated with the first area;

[0205] The area locking module 22 is configured to lock the first area associated with module U1 when the service status of the controlled device is in the second state.

[0206] The parameter configuration notification sending module 23 is configured to send a parameter configuration notification associated with module U2 to a remote control management server corresponding to a remote control application running on the controlled device through the remote control application, so that the remote control management server calls a communication management system to perform parameter configuration on the second area associated with module U2; the communication management system is configured to generate module update information; the module update information is generated when the module type corresponding to module U2 is changed from the second type of module to the first type of module based on the second area associated with module U2.

[0207] The module update information receiving module 24 is configured to receive the module update information returned by the remote control management server, and determine the communication link corresponding to module U2 as a third link based on the module update information and the first type of module associated with the second area.

[0208] The network aggregation module 25 is configured to perform network aggregation on the first link and the third link through the remote control application to obtain a first aggregated link; the first aggregated link is used to transmit the first multimedia data stream and the second multimedia data stream.

[0209] The communication module includes N modules; N is a positive integer greater than 1; the N modules include module U1 and module U2.

[0210] The target state determining module 26 is configured to determine the first state as the target state, or determine the second state as the target state when the first state of the controlled device is changed to the second state.

[0211] The third service condition obtaining module 27 is configured to obtain a third service condition from the service condition associated with the controlled device; the third service condition comprises a first network state threshold, a second network state threshold, and a fourth service detection duration; the second network state threshold is less than the first network state threshold.

[0212] The third state switching module 28 is configured to determine that the controlled device satisfies the third service condition, and switch the service state of the controlled device from the target state to a third state, if the network state parameter of the module U1 is less than the second network state threshold within the fourth service detection duration, the network state parameter of the module U2 is greater than the first network state threshold within the fourth service detection duration, and the heartbeat state of the module U1 and the heartbeat state of the module U2 are both in the normal state.

[0213] The first area is different from the second area.

[0214] The communication module type changing module 29 is configured to determine the module type corresponding to the module U2 when the service state of the controlled device is the third state, and change the module type corresponding to the module U2 from the second type module to the first type module based on the second area associated with the module U2 when the module type corresponding to the module U2 is the second type module.

[0215] The area switching notification sending module 30 is configured to send an area switching notification to a remote control management server corresponding to a remote control application in the controlled device through the remote control application; the area switching notification is used to instruct the remote control management server to configure the access area associated with the module U1 as the second area associated with the module U2 through a communication management system; the communication management system is configured to generate area switching information associated with the module U1 when the access area associated with the module U1 is successfully switched to the second area.

[0216] The area switching information receiving module 31 is configured to receive the area switching information returned by the remote control management server, and change the module type corresponding to the module U1 from the first type module to the second type module based on the second area associated with the module U1.

[0217] The fourth service condition obtaining module 32 is configured to obtain a fourth service condition from the service condition associated with the controlled device; the fourth service condition comprises a fifth service detection duration.

[0218] The fault module determination module 33 is configured to determine the module in the fault state as a fault module and switch the service state of the controlled device to the fourth state if there is a module in the fault state in the module U1 and the module U2; the fault state refers to a state that the network state parameter is not reported within the fifth service detection duration or the heartbeat state is an abnormal state;

[0219] The normal module determination module 34 is configured to determine the module other than the fault module as a normal module in the module U1 and the module U2.

[0220] The normal module type changing module 35 is configured to change the module type corresponding to the normal module from the second type module to the first type module based on the access area associated with the normal module if the module type corresponding to the normal module is the second type module.

[0221] The network aggregation condition obtaining module 36 is configured to obtain a network aggregation condition associated with the controlled device, and perform network aggregation on the first link and the second link through the remote control application run by the controlled device to obtain a second aggregation link when the controlled device meets the network aggregation condition.

[0222] The second data stream sending module 37 is configured to send the first multimedia data stream and the second multimedia data stream to the control device through the second aggregation link.

[0223] The network aggregation condition includes a first aggregation condition or a second aggregation condition; the first aggregation condition refers to that the network state parameter of the first type module or the second type module decreases, and the decreased network state parameter is not less than a second network state threshold; the second aggregation condition refers to that the network load of the geographical position where the controlled device is located is greater than a load threshold.

[0224] The specific implementation manners of the module type determination module 11, the multimedia data stream obtaining module 12, the first data stream sending module 13, the control operation execution module 14, the signal strength obtaining module 15, the heartbeat state detection module 16, the first service condition obtaining module 17, the first state switching module 18, the parameter determination module 19, the second service condition obtaining module 20, the second state switching module 21, the area locking module 22, the parameter configuration notification sending module 23, the module update information receiving module 24, the network aggregation module 25, the target state determination module 26, the third service condition obtaining module 27, the third state switching module 28, the communication module type changing module 29, the area switching notification sending module 30, the area switching information receiving module 31, the fourth service condition obtaining module 32, the fault module determination module 33, the normal module determination module 34, the normal module type changing module 35, the network aggregation condition obtaining module 36, and the second data stream sending module 37 can be referred to the above Figure 9The description of steps S201-S208 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.

[0225] Further, please refer to Figure 9 , Figure 9 is a schematic diagram of a computer device provided by an embodiment of the present application. As Figure 1 shown, the computer device 1000 can be the controlled device 110B in the above-mentioned Figure 9 corresponding embodiment, and the computer device 1000 can include at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display screen (Display) and a keyboard (Keyboard), and the network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 1005 can also be at least one storage device located away from the aforementioned processor 1001. As Figure 9 shown, the memory 1005, as a computer storage medium, can include an operating system, a network communication module, a user interface module, and a device control application.

[0226] In the computer device 1000 shown in Figure 3 , the network interface 1004 is mainly used for network communication with the control device and the remote control management server; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to realize:

[0227] When the service state of the controlled device is in a first state, a first type module and a second type module in the communication module of the controlled device are determined; the first type module and the second type module are determined based on a network state parameter of a geographic location where the controlled device is located; and the priority of a first link corresponding to the first type module is higher than the priority of a second link corresponding to the second type module;

[0228] A first multimedia data stream and a second multimedia data stream for transmission to the control device are obtained; the first multimedia data stream is determined by key data frames in the multimedia data collected by the controlled device; and the second multimedia data stream is determined by auxiliary data frames in the multimedia data;

[0229] The first multimedia data stream is sent to the control device through a first link, and the second multimedia data stream is sent to the control device through a second link, so that the control device determines control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream;

[0230] When the controlled device receives the control signaling, a control operation indicated by the control signaling is performed.

[0231] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the foregoing Figure 4 and Figure 8 corresponding embodiments of the data processing method, and can also execute the foregoing Figure 3 corresponding embodiments of the data processing apparatus 1, which will not be described here. In addition, the beneficial effects of using the same method will not be described here.

[0232] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the data processing method provided by each step in Figure 4 and Figure 3 may be implemented. For details, refer to the implementation manner provided by each step in Figure 4 and Figure 3 , which will not be described here.

[0233] The computer readable storage medium can be the data transmission apparatus or the internal storage unit of the computer device provided in any of the foregoing embodiments, for example, the hard disk or the memory of the computer device. The computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the computer device. The computer readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0234] In one aspect, the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium. The processor executes the computer instructions, so that the computer device can execute the foregoing Figure 4 or ​The data processing method is described in the corresponding embodiments, and will not be described here. In addition, the beneficial effects of using the same method are not described again.

[0235] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The above-mentioned program can be stored in a computer readable storage medium, and when the program is executed, the program can include the processes of the above-mentioned embodiments of each method. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), etc.

[0236] The above only discloses the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

1. A data processing method, characterized in that: include: When the service state of the controlled device is the first state, determining a first type module and a second type module in the communication module of the controlled device; The first type module and the second type module are determined based on the network status parameter of the geographical location of the controlled device; the priority of the first link corresponding to the first type module is higher than the priority of the second link corresponding to the second type module; Acquire a first multimedia data stream and a second multimedia data stream for transmission to a control device; The first multimedia data stream is determined by key data frames in the multimedia data collected by the controlled device; the second multimedia data stream is determined by auxiliary data frames in the multimedia data; sending the first multimedia data stream to the control device through the first link, and sending the second multimedia data stream to the control device through the second link, so that the control device determines control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream; When the controlled device receives the control signaling, it performs the control operation indicated by the control signaling.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining network status parameters of each module in the communication module of the controlled device; respectively detecting the heartbeat status of each module between the remote control application in the controlled device and the control application in the control device; Acquire a service condition associated with the controlled device, and determine a first network status threshold and a first service detection duration in a first service condition from the service condition; When the network status parameters of each module are greater than the first network status threshold within the first service detection time period, and the heartbeat status of each module is in a normal state, it is determined that the controlled device meets the first service condition, and the service status of the controlled device is switched to the first state; the normal state is determined by the remote control application when periodically sending heartbeat packets to the control application.

3. The method according to claim 1, characterized in that The communication module includes N modules; N is a positive integer greater than 1; the first type module among the N modules includes module U1; The second type modules among the N modules include module U2; The method further comprises: The network status parameter of the module U1 is used as the first network status parameter, the heartbeat status of the module U1 is used as the first heartbeat status, the network status parameter of the module U2 is used as the second network status parameter, and the heartbeat status of the module U2 is used as the second heartbeat status; Acquire a second service condition from the service conditions associated with the controlled device; When it is determined that the controlled device meets the second service condition based on the first network status parameter, the second network status parameter, the first heartbeat status, and the second heartbeat status, the service status of the controlled device is switched from the first status to the second status.

4. The method according to claim 3, characterized in that The second service condition includes a first sub-condition; the first sub-condition includes a second network status threshold and a second service detection duration; The method of switching the service state of the controlled device from the first state to the second state when it is determined that the controlled device satisfies the second service condition based on the first network status parameter, the second network status parameter, the first heartbeat status, and the second heartbeat status includes: If the first network status parameter and the second network status parameter are both less than the second network status threshold within the second service detection time, and the first heartbeat status and the second heartbeat status are both in normal state, it is determined that the controlled device meets the first sub-condition, and the service state of the controlled device is switched from the first state to the second state.

5. The method according to claim 3, characterized in that The second service condition includes a second sub-condition; the second sub-condition includes a first network status threshold, a second network status threshold, and a third service detection duration; the second network status threshold is less than the first network status threshold; The method of switching the service state of the controlled device from the first state to the second state when it is determined that the controlled device satisfies the second service condition based on the first network status parameter, the second network status parameter, the first heartbeat status, and the second heartbeat status includes: If the first network status parameter is greater than the first network status threshold within the third service detection time length, the second network status parameter is less than the second network status threshold within the third service detection time length, and the first heartbeat status and the second heartbeat status are both in normal states, it is determined that the controlled device meets the second sub-condition, and the service state of the controlled device is switched from the first state to the second state.

6. The method according to claim 1, characterized in that The first-type module includes module U1 in the communication module; the second-type module includes module U2 in the communication module; the access area associated with module U1 is the first area, and the access area associated with module U2 is the second area; the first area is different from the second area; the first link refers to the communication link corresponding to module U1; the module type of module U1 is the first-type module associated with the first area; The method further comprises: When the service state of the controlled device is the second state, locking the first area associated with the module U1; sending, via the remote control application running on the controlled device, a parameter configuration notification associated with the module U2 to a remote control management server corresponding to the remote control application, so that the remote control management server invokes a communication management system to perform parameter configuration on the second region associated with the module U2; the communication management system is configured to generate module update information; the module update information is generated when the module type corresponding to the module U2 is changed from the second type module to the first type module based on the second region associated with the module U2; receiving the module update information returned by the remote control management server, and determining, based on the module update information and the first type module associated with the second area, that the communication link corresponding to the module U2 is a third link; Through the remote control application, network aggregation is performed on the first link and the third link to obtain a first aggregated link; the first aggregated link is used to transmit the first multimedia data stream and the second multimedia data stream.

7. The method according to claim 1, characterized in that The communication module includes N modules; N is a positive integer greater than 1; the N modules include module U1 and module U2; The method further comprises: determining the first state as a target state, or when changing the first state of the controlled device to a second state, determining the second state as the target state; Acquire a third service condition from the service conditions associated with the controlled device; the third service condition includes a first network status threshold, a second network status threshold, and a fourth service detection duration; the second network status threshold is less than the first network status threshold; If the network status parameter of the module U1 is less than the second network status threshold within the fourth service detection time period, the network status parameter of the module U2 is greater than the first network status threshold within the fourth service detection time period, and the heartbeat status of the module U1 and the heartbeat status of the module U2 are both in normal state, it is determined that the controlled device meets the third service condition, and the service state of the controlled device is switched from the target state to the third state.

8. The method according to claim 7, characterized in that The access area associated with the module U1 is the first area, and the access area associated with the module U2 is the second area; the first region being different from the second region; The method further comprises: When the service state of the controlled device is the third state, determining a module type corresponding to the module U2; if the module type corresponding to the module U2 is the second type module, changing the module type corresponding to the module U2 from the second type module to the first type module based on the second area associated with the module U2; sending, via the remote control application in the controlled device, a region switching notification to the remote control management server corresponding to the remote control application; the region switching notification is used to instruct the remote control management server to configure, through the communication management system, the access region associated with the module U1 from the first region to the second region associated with the module U2; the communication management system is used to generate region switching information associated with the module U1 when the access region associated with the module U1 is successfully switched to the second region; The region switching information returned by the remote control management server is received, and based on the second region associated with the module U1 , the module type corresponding to the module U1 is changed from the first type module to the second type module.

9. The method according to claim 7, characterized in that The method further comprises: Acquire a fourth service condition from the service conditions associated with the controlled device; the fourth service condition includes a fifth service detection duration; If one of the modules U1 and U2 is in a faulty state, the module in the faulty state is determined as a faulty module, and the service state of the controlled device is switched to a fourth state; the faulty state refers to a state in which no network status parameters are reported within the fifth service detection time period or the heartbeat state is abnormal; Among the modules U1 and U2, the modules other than the faulty module are regarded as normal modules; If the module type corresponding to the normal module is the second-type module, the module type corresponding to the normal module is changed from the second-type module to the first-type module based on the access zone associated with the normal module.

10. The method according to claim 1, characterized in that The communication module includes N modules; N is a positive integer greater than 1; The step of determining, when the service state of the controlled device is the first state, the first type module and the second type module in the communication module of the controlled device includes: When the service state of the controlled device is the first state, determining N module auxiliary information corresponding to the N modules; one module corresponds to one module auxiliary information, and a network state parameter in one module auxiliary information is determined by a network state of an access area associated with the corresponding module; the network state of the access area is detected based on the geographical location of the controlled device; The remote control application running on the controlled device transmits the N module auxiliary information to a remote control management server corresponding to the remote control application; the remote control management server is configured to obtain module establishment information from the communication management system; the module establishment information is generated by the communication management system when a first type of module is successfully established based on network status parameters of each module auxiliary information in the N module auxiliary information; the first type of module is the module corresponding to the module auxiliary information having the largest network status parameter; The module establishment information is received through the remote control application, the first type module is determined based on the module establishment information, and the modules other than the first type module among the N modules are determined as second type modules.

11. The method according to claim 1, wherein The acquiring of the first multimedia data stream and the second multimedia data stream for transmission to the control device includes: Collecting multimedia data of the geographical location of the controlled device through a data collection component of the controlled device; Acquiring an encoding configuration policy associated with the multimedia data, and determining key data frames in the multimedia data and auxiliary data frames in the multimedia data based on the encoding configuration policy; encoding the key data frame based on the encoding configuration strategy to obtain a first multimedia data stream for transmission to a control device associated with the controlled device; The auxiliary data frame is encoded based on the encoding configuration strategy to obtain a second multimedia data stream for transmission to the control device.

12. The method according to claim 1, characterized in that The method further comprises: Acquire a network aggregation condition associated with the controlled device, and when the controlled device satisfies the network aggregation condition, perform network aggregation on the first link and the second link through a remote control application run by the controlled device to obtain a second aggregated link; The first multimedia data stream and the second multimedia data stream are both sent to the control device through the second aggregation link.

13. The method according to claim 12, characterized in that The network aggregation condition includes a first aggregation condition or a second aggregation condition; the first aggregation condition refers to that the network status parameter of the first type module or the second type module decreases, and the network status parameter after the decrease is not less than the second network status threshold; the second aggregation condition refers to that the network load at the geographical location of the controlled device is greater than the load threshold.

14. The method according to claim 1, wherein When the controlled device receives the control signaling, executing the control operation indicated by the control signaling includes: receiving, via the first link, a first control signaling from the remote control application of the controlling device to the controlled device; receiving, via the second link, a second control signaling returned by the control device to the remote control application; the second control signaling being obtained by the control application in the control device copying the first control signaling; The first control signaling and the second control signaling are deduplicated by the remote control application to obtain a target control signaling, and a control operation indicated by the target control signaling is executed; the target control signaling is the first control signaling or the second control signaling.

15. A data processing device, characterized in that: include: a module type determination module, configured to determine, when the service state of the controlled device is the first state, a first type module and a second type module in the communication module of the controlled device; The first type module and the second type module are determined based on the network status parameter of the geographical location of the controlled device; the priority of the first link corresponding to the first type module is higher than the priority of the second link corresponding to the second type module; a multimedia data stream acquisition module, configured to acquire a first multimedia data stream and a second multimedia data stream for transmission to a control device; the first multimedia data stream being determined by key data frames in multimedia data collected by the controlled device; and the second multimedia data stream being determined by auxiliary data frames in the multimedia data; a first data stream sending module, configured to send the first multimedia data stream to the control device via the first link, and send the second multimedia data stream to the control device via the second link, so that the control device determines control signaling associated with the multimedia data based on the first multimedia data stream and the second multimedia data stream; The control operation execution module is configured to execute the control operation indicated by the control signaling when the controlled device receives the control signaling.

16. A computer device, characterized in that: include: processor and memory; The processor is connected to a memory, wherein the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 14.

18. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. The computer instructions are suitable for being read and executed by a processor, so as to enable a computer device having the processor to perform the method according to any one of claims 1 to 14.

Images