Multi-device cooperative interaction method and system for digital multimedia

Abstract

The application provides a multi-device cooperative interaction method and system for digital multimedia, relates to the technical field of digital multimedia processing, and first analyzes user interaction requirements of to-be-processed digital multimedia content, and disassembles the requirements into presentation, transmission and operation requirement dimensions; capability information of to-be-cooperated devices is collected to form a device capability list; an adaptation degree is calculated according to the requirement disassembly result and the device capability list, a cooperative interaction link is constructed; operation instructions are issued according to the cooperative interaction link, and interaction result data is acquired; the adaptation degree calculation standard and link node configuration are adjusted based on the interaction result data, and dynamic optimization is realized. The application comprehensively considers user requirements and device capabilities, realizes efficient cooperation of multiple devices, and improves the digital multimedia processing experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital multimedia processing, in particular to a multi-device collaborative interaction method and system for digital multimedia. BACKGROUND

[0002] In the current situation of increasingly wide application of digital multimedia, the processing needs of users for digital multimedia content are becoming more and more diversified and complex. Traditional digital multimedia processing methods are often limited to a single device, for example, when watching a video, a user can only play it through a single device such as a TV or a computer, and cannot fully utilize the advantages of other devices to improve the viewing experience. Even if there is a multi-device collaboration, there is still a lack of scientific and reasonable methods to guide the collaborative work between devices.

[0003] Some existing multi-device collaboration schemes usually simply distribute digital multimedia content to different devices according to preset rules, without fully considering the user's interactive needs for digital multimedia content in different aspects. For example, in terms of presentation needs, different devices may have different display resolutions, color rendering capabilities, etc., but existing schemes cannot reasonably allocate presentation tasks according to these differences; in terms of transmission needs, different devices have different network transmission capabilities and bandwidths, and existing schemes cannot dynamically adjust the transmission strategy according to the actual transmission situation; in terms of operation needs, different devices have different operation methods and convenience, and existing schemes cannot optimize the operation process according to user operation habits and device operation characteristics. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a multi-device collaborative interaction method and system for digital multimedia.

[0005] According to a first aspect of the present application, a multi-device collaborative interaction method for digital multimedia is provided, which comprises:

[0006] analyzing the user interaction needs of the current digital multimedia content to be processed, decomposing the user interaction needs into presentation needs, transmission needs and operation needs, and generating a demand dimension decomposition result;

[0007] collecting the capability information of the devices to be collaborated, forming a device capability list based on the capability information, and the capability information including specific capability items of the devices in terms of presentation, transmission and operation;

[0008] calculating the adaptation degree of each device to different demand dimensions according to the demand dimension decomposition result and the device capability list, and constructing a digital multimedia multi-device collaborative interaction link based on the adaptation degree;

[0009] According to the multi-device cooperative interaction link, interaction operation instructions are issued to each device to be cooperated to guide each device to perform corresponding digital multimedia processing actions, and interaction result data fed back by the device processing actions are acquired.

[0010] Based on the interaction result data, the adaptation degree calculation standard of each device to different demand dimensions and the node configuration of the multi-device cooperative interaction link are adjusted to realize dynamic optimization of multi-device cooperative interaction.

[0011] According to a second aspect of the present application, a multi-device cooperative interaction system for digital multimedia is provided, which comprises a machine readable storage medium and a processor, the machine readable storage medium stores machine executable instructions, and the processor, when executing the machine executable instructions, implements the multi-device cooperative interaction method for digital multimedia as described above.

[0012] According to a third aspect of the present application, a computer readable storage medium is provided, which stores computer executable instructions, and when the computer executable instructions are executed, the multi-device cooperative interaction method for digital multimedia as described above is implemented.

[0013] According to any one of the above aspects, the technical effect of the present application is that:

[0014] By analyzing the user interaction demand of the current digital multimedia content to be processed and decomposing it into presentation demand dimension, transmission demand dimension and operation demand dimension, then collecting the capability information of the devices to be cooperated and forming a device capability list, the specific capability items of each device in the aspects of presentation, transmission and operation are determined in detail, the adaptation degree of each device to different demand dimensions is calculated according to the demand dimension decomposition result and the device capability list, and the digital multimedia multi-device cooperative interaction link is constructed, which can fully exert the advantages of each device, realize the optimal allocation of resources, and make the digital multimedia content be processed and presented most appropriately on different devices. According to the cooperative interaction link, interaction operation instructions are issued to each device to acquire interaction result data, and based on these data, the adaptation degree calculation standard and the node configuration of the interaction link are dynamically adjusted, which can adjust the cooperative strategy in real time according to the actual situation, constantly adapt to the changes of devices and user demands, greatly improve the efficiency, quality and flexibility of multi-device cooperative interaction, and provide users with a more high-quality and efficient digital multimedia processing experience. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A flowchart of the multi-device cooperative interaction method for digital multimedia provided by the embodiment of the present application is shown;

[0016] Figure 2A component structure schematic diagram of a multi-device cooperative interaction system for digital multimedia provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] Figure 1 A flowchart of the multi-device cooperative interaction method and system for digital multimedia provided by an embodiment of the present application is shown, and the detailed steps include:

[0018] Step S110: Analyze the user interaction demand of the current digital multimedia content to be processed, disassemble the user interaction demand into a presentation demand dimension, a transmission demand dimension and an operation demand dimension, and generate a demand dimension disassembly result.

[0019] In this embodiment, the digital intelligence operation scene of an offline large-scale supermarket is taken as an example. The digital multimedia content to be processed in this scene is a supermarket real-time promotion video stream and dynamic commodity price tag data, which needs to be cooperatively interacted through the entry LED large screen in the supermarket, the electronic price tag of each shelf, the hand-held terminal of the guide, the customer self-service inquiry machine and other devices to realize accurate promotion information reaching and shopping guidance. The detailed description of each step will be carried out around this scene.

[0020] Step S111: Obtain the content attribute of the current digital multimedia content to be processed, which includes the content type, the content carrier form and the content data characteristics.

[0021] In the above-mentioned digital intelligence operation scene of the supermarket, the current digital multimedia content to be processed includes two types of core data: one is the promotion video stream, the content type of which is dynamic video, the content carrier form of which is a video format file stored in the local server of the supermarket, and the content data characteristics of which include video encoding mode, key frame interval, average code rate, audio sampling frequency, etc.; the other is the dynamic commodity price tag data, the content type of which is structured text and static image combined data, the content carrier form of which is a structured format real-time update data stream, and the content data characteristics of which include field number, update frequency, data check bit length, image compression algorithm, etc. Through the interface call of the content management system, the specific parameter values of the above-mentioned content attributes can be extracted to form a content attribute description document, which is stored in a markup language format and includes a content unique identifier, an attribute category label, a parameter name and a corresponding parameter value field.

[0022] Step S112: Determine the interaction behavior type that the user is likely to generate based on the content attribute, which includes a watching type behavior, an editing type behavior and a sharing type behavior.

[0023] Step S1121: Based on the content type and the predefined interaction behavior type mapping relationship, determine the basic interaction behavior type set corresponding to the content type, and the predefined interaction behavior type mapping relationship specifies that the video content, audio content, image content and text content are respectively associated with the watching, editing and sharing basic behavior types.

[0024] In the rule configuration part of the store intelligent management platform, the predefined interaction behavior type mapping relationship is stored in a structured table form, and the table contains content type name, basic behavior type list and behavior trigger condition column. For example, in the table row corresponding to video content, the basic behavior type list field value is watching behavior and sharing behavior, and the behavior trigger condition field value is that the video playing time reaches a certain proportion of the total time. After obtaining the content type name of the back kitchen dish making progress video stream, the corresponding basic behavior type set is matched to watching behavior by traversing the structured table in the rule configuration part, and the set is stored in the form of a list, each list item containing a behavior type name, a behavior code and a priority sorting field.

[0025] Step S1122: Based on the content carrier form, activate or deactivate a specific behavior type in the basic interaction behavior type set. If the content carrier form is a mobile terminal storage, activate the offline editing behavior type; if the content carrier form is a cloud storage, activate the online sharing behavior type.

[0026] The content carrier form of the real-time ordering interface data is the store local server storage, and by querying the carrier form and the behavior activation rule library, the local server storage corresponds to the rule "allow to activate the offline editing behavior type", and the rule library contains the carrier form name, behavior type operation, target behavior type name, permission verification level and other fields. First, verify whether the current user role has the permission verification level requirement, and after the verification is passed, call the behavior type set operation interface to change the state field of the offline editing behavior type in the basic behavior type set list from disabled to enabled, and realize the activation of the behavior type. The carrier form of the member marketing activity graphic information is cloud storage, and the corresponding rule in the rule library is "activate the online sharing behavior type", so the online sharing behavior type entry is added to the basic behavior type set thereof.

[0027] Step S1123: Based on the content data characteristics, adjust the technical implementation requirements of a specific behavior type in the basic interaction behavior type set. If the content data volume exceeds the preset upper data volume threshold, associate a high transmission efficiency requirement with the sharing behavior type; if the content data volume is lower than the preset lower data volume threshold, associate a high operation response speed requirement with the editing behavior type.

[0028] The interface component quantity parameter in the content data feature of the real-time ordering interface data is counted, and the corresponding data quantity exceeds the preset upper limit threshold of data quantity (the threshold is stored in the performance parameter node of the system configuration file). According to the mapping rule of content data feature and behavior technical requirement, when the data quantity exceeds the upper limit threshold, a high transmission efficiency requirement needs to be associated for the editing behavior type, which specifically includes setting the data transmission protocol to a high-efficiency transmission protocol, enabling a data compression transmission mechanism, setting the upper limit of the transmission data packet size, and the like. By calling the behavior technical requirement configuration interface, the above-mentioned parameters are written into the technical requirement configuration file of the editing behavior type, which includes a protocol type field, a compression algorithm field, a packet size limit field, and the like. The data quantity of the dynamic commodity price label data is lower than the preset lower limit threshold, so a high operation response speed requirement is associated for the viewing behavior type, and parameters such as touch response delay upper limit and data loading progress feedback frequency are configured.

[0029] Step S1124: integrating the interactive behavior types adjusted by the content type, content carrier form and content data feature to form a final set of interactive behavior types that the user can generate.

[0030] The three types of digital multimedia content after content type matching, carrier form activation / deactivation and data feature adjustment are summarized and merged. For the real-time ordering interface data, the interactive behavior types are viewing behavior, editing behavior and sharing behavior; the kitchen dish making progress video stream is viewing behavior; the member marketing activity text and image information is viewing behavior and sharing behavior. In the merging process, duplicate behavior type entries are removed, and all non-duplicate behavior types are retained, and finally a set of interactive behavior types that the user can generate is formed, including viewing behavior, editing behavior and sharing behavior. The set is stored in a structured format and includes an array of behavior type names, an array of applicable content types for each behavior type, and a technical requirement configuration file path field.

[0031] Step S1125: performing validity verification on the final set of interactive behavior types to ensure that each behavior type in the set has corresponding technical implementation support, and if there is a behavior type that is not supported by technical implementation, the behavior type is removed from the set.

[0032] The technical implementation support verification part is called, which connects the device capability database of the store digital management platform to query whether the technical implementation requirements corresponding to each interactive behavior type in the set are supported by the device. For example, when verifying the editing behavior, it is checked whether there is a device with editing function and the hardware configuration of the device meets the resource requirements of editing operation; when verifying the sharing behavior, it is checked whether there is a device with network connection module and supporting sharing protocol. If a certain behavior type is not supported by the device after verification, it is removed from the interactive behavior type set, and the removal reason is recorded in the verification report.

[0033] Step S113: For each interaction behavior type, collect the user's expected information about the behavior of that interaction behavior type, which includes the user's appeal for the behavior execution effect, execution efficiency, and execution mode.

[0034] In the intelligent operation scene of a catering store, for the viewing behavior, by embedding a user experience feedback part in the table intelligent terminal, when the customer views the dish information, an expected information collection pop-up window is triggered, and the content collected includes the satisfaction of the dish picture display effect (execution effect appeal), the acceptable range of dish information loading speed (execution efficiency appeal), and the preference of information browsing mode (execution mode appeal); for the editing behavior, by analyzing the operation log of the waiter's handheld ordering device, data such as the operation time distribution of the waiter when modifying the order information (execution efficiency appeal), operation step simplification suggestions (execution mode appeal), and order modification accuracy requirements (execution effect appeal) are extracted; for the sharing behavior, by analyzing the operation records of the customer using the table intelligent terminal to share the preferential activities, expected information such as the sharing link generation speed (execution efficiency appeal), the sharing content integrity (execution effect appeal), and the sharing channel diversity (execution mode appeal) is collected. The collected expected information is stored in the user expectation database in the form of a structured document, and the document includes the behavior type identifier, the appeal dimension label, the appeal specific description, and the appeal time field.

[0035] Step S114: Divide the expected information into presentation demand dimension, transmission demand dimension, and operation demand dimension according to the core elements affecting the implementation of the interaction.

[0036] Call the demand dimension division processing part, which is built-in with a core element recognition algorithm. The algorithm input is the expected information text content and the appeal quantification index. By using natural language processing technology, the keyword feature vector is extracted, and the keyword feature vector is compared with the core feature vector library of the three demand dimensions. For example, in the viewing behavior, the keyword "dish picture display effect" has a high similarity with the "visual presentation quality" core feature vector of the presentation demand dimension, so it is divided into the presentation demand dimension; the keyword "dish information loading speed" has a high similarity with the "data transmission efficiency" core feature vector of the transmission demand dimension, so it is divided into the transmission demand dimension; the keyword "information browsing mode preference" has a high similarity with the "interactive operation mode" core feature vector of the operation demand dimension, so it is divided into the operation demand dimension. After the division is completed, a demand dimension attribution report is generated, which includes the original ID of each expected information, the name of the belonging dimension, the matching core feature word, and the similarity calculation result field.

[0037] Step S115: The presentation requirement dimension is refined, and specific requirement items under the presentation requirement dimension are extracted, including content picture clarity requirement, picture color restoration requirement, picture fluency requirement, and picture size adaptation requirement.

[0038] Based on the expectation information in the presentation requirement dimension attribution report, a requirement item refinement processing program is called. For the content picture clarity requirement, the clarity requirements of different dish categories are extracted from the “dish picture display effect satisfaction” expectation information, such as the requirement that fresh dish pictures meet high definition requirements, and the requirement that staple dish pictures meet medium clarity requirements. The picture color restoration requirement is extracted from the customer feedback on the color difference between dish pictures and real objects, including the color saturation requirement for red dish, the color tone accuracy requirement for green dish, and the brightness uniformity requirement for white dish. The picture fluency requirement is refined into interface switching frame rate requirement, animation transition fluency requirement, and scrolling loading fluency requirement based on the fluency feedback data when the dynamic ordering interface switches. The picture size adaptation requirement extracts the picture scaling ratio requirement, layout adaptive adjustment requirement, and font size adaptation requirement for different devices in combination with different device screen parameters. Each specific requirement item includes requirement item name, requirement description text, quantitative index range, and priority identification field.

[0039] Step S116: The transmission requirement dimension is refined, and specific requirement items under the transmission requirement dimension are extracted, including content data transmission speed requirement, data integrity requirement during transmission, transmission delay control requirement, and multi-device transmission synchronization requirement.

[0040] According to the transmission requirement dimension attribution report, a transmission requirement refinement analysis process is started. The content data transmission speed requirement is extracted from the “dish information loading speed acceptable range” expectation information, including first loading speed requirement, secondary loading cache speed requirement, and dynamic data update transmission speed requirement. The data integrity requirement during transmission is refined into data verification method requirement, error retransmission mechanism requirement, and data backup storage requirement based on order data transmission error feedback cases. The transmission delay control requirement extracts order submission delay requirement, dish making progress update delay requirement, and member information synchronization delay requirement in combination with the real-time feedback of kitchen and front desk data interaction. The multi-device transmission synchronization requirement is refined into data update timestamp synchronization requirement, cross-device operation state synchronization requirement, and offline data synchronization strategy requirement according to the multi-terminal display consistency feedback. Each specific requirement item includes requirement item code, requirement detailed description, standard for reaching, and non-standard processing plan field.

[0041] Step S117: The operation requirement dimension is refined, and specific requirement items under the operation requirement dimension are extracted, including user operation trigger response speed requirement, operation instruction synchronization requirement between multiple devices, operation permission division requirement, and operation result feedback method requirement.

[0042] Based on the operation requirement dimension attribution report, the operation requirement item extraction process is performed. The user operation trigger response speed requirement is extracted from the operation fluency feedback in the "information browsing mode preference", including touch operation response delay requirement, key operation response time requirement, voice operation recognition response speed requirement; the operation instruction synchronization requirement between multiple devices is refined to instruction synchronization delay requirement, synchronization failure processing requirement, conflict instruction solution strategy requirement according to the service staff multi-terminal collaborative operation feedback; the operation permission division requirement is combined with the operation range feedback of different user roles to extract the role permission definition requirement, permission change approval requirement, and unauthorized operation interception requirement; the operation result feedback mode requirement is extracted from the user's feedback on the operation result awareness, including visual feedback mode requirement, auditory feedback mode requirement, and tactile feedback mode requirement. Each specific requirement item contains requirement item name, applicable scene description, feedback mode option, and priority sorting field.

[0043] Step S118: Label the demand priority for each specific requirement item under the demand dimension. The priority division basis is the user's attention to the requirement item and the influence of the requirement item on the interactive experience.

[0044] The demand priority labeling part is called, which connects the user feedback database and the interactive experience evaluation model. First, the user mention frequency and satisfaction score data of each specific requirement item are extracted from the user feedback database, and the user attention index is calculated; second, the influence weight of each requirement item on the overall interactive experience is analyzed through the interactive experience evaluation model, and the influence weight is calculated based on the user churn rate change data when the requirement item is not up to standard. The user attention index and the influence weight are combined to calculate the priority score of each requirement item by weighted summation, and the priority score is calculated according to a certain proportion of the user attention index and the influence weight. According to the priority score, the requirement items are divided into high, medium and low priority levels, and the scores in the higher interval are high priority, the scores in the middle interval are medium priority, and the scores in the lower interval are low priority. After labeling, each requirement item adds the priority level field, the priority score field, and the score basis description field.

[0045] Step S119: Integrate the presentation requirement dimension, transmission requirement dimension, operation requirement dimension, and the specific requirement items and demand priorities contained therein to form the requirement dimension disintegration result.

[0046] The integration process of the start-up requirement dimension will present the specific requirement items of the presentation requirement dimension, transmission requirement dimension, and operation requirement dimension according to the dimension classification, and each dimension contains the dimension name, dimension description, and specific requirement item list field. Each element in the specific requirement item list contains the requirement item name, requirement description, quantitative indicator, priority level, and priority score field. During the integration process, repeated or conflicting requirement items are merged and coordinated. For example, the screen display related requirement items involved in the presentation requirement dimension and the operation requirement dimension are unified and merged into the presentation requirement dimension. The final requirement dimension disassembly result is stored in a standardized document format, and the document contains the disassembly result version number, generation time, dimension number, total number of requirement items, and detailed content chapter of each dimension. The document is stored in the requirement management part of the store digitization management platform and serves as the basis for subsequent device capability matching and collaborative link construction.

[0047] Step S120: Collect the capability information of the devices to be coordinated, and form a device capability list based on the capability information. The capability information includes specific capability items of the devices in terms of presentation, transmission, and operation.

[0048] In the scenario of intelligent operation of a catering store, the devices to be coordinated include a table intelligent terminal, a kitchen display large screen, a waiter handheld ordering device, a store entrance welcome advertising screen, and a customer self-service inquiry machine. Through the device management part of the store digitization management platform, a capability information collection request is sent to the above-mentioned devices to collect the specific capability items of each device in terms of presentation, transmission, and operation. The collection process uses a polling method, and the request is sent in order according to the device address. The response timeout time of each device is set to a certain length of time. If the response is not received within the timeout time, a second request is sent. If the response is still not received after the second request, the device is marked as offline. After the collected capability information is sorted, a device capability list is formed, which contains the device basic information, presentation capability item list, transmission capability item list, and operation capability item list fields.

[0049] Step S121: Establish a communication connection with the multiple devices to be coordinated, and send a capability information collection request to each device. The request determines that the capability categories to be collected are presentation capability, transmission capability, and operation capability.

[0050] The device communication part of the store intelligent management platform first initiates a communication connection establishment process with each device to be cooperated, connects by using a network protocol, and the connection port number is a preset device communication port. After the connection is successfully established, the device communication part generates a capability information collection request data packet, which contains a request identifier, a request timestamp, a capability category list (presentation capability, transmission capability, operation capability), a data format requirement, and a response deadline field. The data packet is sent to each device through the established connection, and the sending order is sorted according to the importance of the device. The sending state is monitored in real time during the sending process, and the request sending time, sending result, and failure reason field of each device are recorded.

[0051] Step S122: Receive the basic capability data fed back by each device based on the capability information collection request, wherein the basic capability data includes device hardware configuration information, software function support information, and historical cooperative interaction performance data.

[0052] After receiving the capability information collection request, each device to be cooperated starts a local capability information collection program to collect its own basic capability data. The device hardware configuration information includes processor model, memory capacity, storage capacity, display screen parameters, network interface type, input / output device type, etc.; the software function support information includes operating system version, supported application program interface, pre-installed function module, software update date, etc.; the historical cooperative interaction performance data includes average response time in a certain period of time, data transmission success rate, operation execution accuracy rate, and device online time length proportion, etc. The device encapsulates the collected basic capability data according to the format specified in the request to generate a response data packet, which contains a device identifier, a response timestamp, a basic capability data field, and a data check code field. The device communication part of the store intelligent management platform receives the response data packet of each device, records the receiving time, data size, and check code verification result field, and requires the device to resend the data packet with a failed check.

[0053] Step S123: Extract the specific capability items related to the presentation capability from the basic capability data, including the maximum picture resolution supported by the device, the frame rate range that can be output by the device, the device color processing capability range, the picture scaling mode supported by the device, and the device picture display response speed.

[0054] The rendering capability extraction algorithm is called to analyze the hardware configuration information and software function support information in the basic capability data. The maximum picture resolution supported by the device is extracted from the display screen parameters; the frame rate range that can be output by the device is extracted from the display driver program information; the device color processing capability range is extracted from the color management module information; the picture scaling mode supported by the device is extracted from the graphics processing software function information; and the device picture display response speed is extracted from the display response time test data. Each rendering capability item includes a capability item name, a capability value range, a unit, and a support state field.

[0055] Step S124: Specific capability items related to transmission capability are extracted from the basic capability data, including the maximum data transmission rate of the device, the transmission packet loss rate of the device in different network environments, the recovery speed of the device after transmission interruption, the transmission protocol type supported by the device, and the ability of the device to simultaneously transmit multiple data.

[0056] The transmission capability extraction process is started, and the network interface information and historical collaborative interaction performance data in the basic capability data are analyzed. The maximum data transmission rate of the device is extracted from the network interface parameters; the transmission packet loss rate of the device in different network environments is counted from the historical transmission records; the recovery speed of the device after transmission interruption is extracted from the transmission failure recovery log; the transmission protocol type supported by the device is extracted from the network protocol stack information; and the ability of the device to simultaneously transmit multiple data is extracted from the multi-task processing records. Each transmission capability item includes a capability item name, a test condition, a capability index value, and a stability rating field.

[0057] Step S125: Specific capability items related to operation capability are extracted from the basic capability data, including the response time of the device after receiving an operation instruction, the delay of the device in synchronizing an operation instruction to other devices, the operation instruction type supported by the device, the setting ability of the device for operation permissions, and the way in which the device feeds back operation results.

[0058] The operation capability extraction part is called to process the input and output device information, software function support information, and historical collaborative interaction performance data in the basic capability data. The response time of the device after receiving an operation instruction is extracted from the operation response test data; the delay of the device in synchronizing an operation instruction to other devices is extracted from the multi-device collaboration log; the operation instruction type supported by the device is extracted from the input processing module information; the setting ability of the device for operation permissions is extracted from the permission management module information; and the way in which the device feeds back operation results is extracted from the output feedback module information. Each operation capability item includes a capability item name, a capability description, a supported operation scenario, and a performance index field.

[0059] Step S126: Authenticating the extracted presentation capability item, transmission capability item and operation capability item of each device, the authentication method is to compare the consistency of the historical capability data of the device and the current feedback data, and to verify the matching degree of the actual capability of the device and the feedback capability item through the preset data amount of test data.

[0060] Step S1261: Extracting the past presentation capability data, transmission capability data and operation capability data of the device from the historical cooperative interaction record of the device to form a device historical capability data set.

[0061] The historical data query part of the store digital management platform connects the device historical database, and queries the capability data record of the device in the past period according to the device identifier. The presentation capability historical data includes the maximum picture resolution, average frame rate and color error value recorded daily; the transmission capability historical data includes the average transmission rate, transmission packet loss rate and interruption recovery time recorded daily; the operation capability historical data includes the operation response time, instruction synchronization delay and permission setting success rate recorded daily. The queried data is organized according to the capability category and date to form a device historical capability data set, which includes the capability category, date, capability item name, capability value and data source field.

[0062] Step S1262: Comparing the current extracted presentation capability item, transmission capability item and operation capability item with the corresponding data in the device historical capability data set item by item.

[0063] The capability item comparison part reads the current extracted capability item data and the device historical capability data set, and matches them item by item according to the capability category and capability item name. For numerical capability items, the absolute difference between the current value and the historical average value is calculated; for range type capability items, it is checked whether the current range contains the historical range; for enumeration type capability items, it is checked whether the current supported type set is consistent with the historical set. The comparison result and difference point description field of each capability item are recorded during the comparison process.

[0064] Step S1263: Calculating the deviation value of the current capability item and the historical capability data, if the deviation value is within the preset deviation allowable range, it is determined that the current capability item is verified on the historical data comparison level; if the deviation value exceeds the preset deviation allowable range, the current capability item is marked as a further verification item.

[0065] For a numerical capability item, the deviation value is calculated as the difference between the current value and the historical average value divided by the historical average value, obtaining a percentage deviation value; for a range capability item, the deviation value is calculated as the sum of the difference between the upper and lower limits of the current range and the upper and lower limits of the historical range; for an enumerated capability item, the deviation value is calculated as the sum of the number of missing historical types and the number of current types that are more than the historical types. The preset deviation allowed range is stored in the verification rule configuration file, such as the deviation allowed range of a numerical capability item is a certain percentage interval, the deviation allowed range of a range capability item is that the sum of the upper and lower limits of the difference is not more than a certain percentage of the historical range, and the deviation allowed range of an enumerated capability item is that the number of difference types is zero. The calculated deviation value is compared with the preset allowed range, and if it is within the range, the verification is passed, otherwise it is marked as a further verification item.

[0066] Step S1264: For all capability items, generate a preset amount of test data, and the preset amount of test data needs to cover the core parameter requirements of each capability item. For the maximum picture resolution capability item in the presentation capability item, test images of different resolution specifications are included; for the maximum data transmission rate capability item in the transmission capability item, test files of different sizes are included; for the operation response time capability item in the operation capability item, test operation instructions of different types are included.

[0067] The test data generation part generates test data according to the core parameter requirements of each capability item. For the presentation capability item, test images of different resolution specifications are generated, and the images include standard color test cards, gray gradient patterns, and detailed physical pictures; for the transmission capability item, test files of different sizes are generated, and the file content is random binary data containing data verification information; for the operation capability item, test operation instructions of different types are generated, and the instructions include operation type, coordinate position, trigger time, and expected response content field. The test data amount is generated according to the standard of at least multiple sets of test data for each capability item, ensuring that the minimum requirement, standard requirement, and maximum requirement of the capability item are covered.

[0068] Step S1265: The generated preset amount of test data is sent to the corresponding device to be cooperated, and the corresponding device to be cooperated is required to perform the corresponding action according to the requirements of the preset amount of test data. The corresponding action to be performed by the corresponding device to be cooperated includes displaying test images, transmitting test files, and responding to test operation instructions.

[0069] The test data sending part sends test data through the connection established with the to-be-cooperated device, and the sending sequence is to send the presentation capability test data first, then the transmission capability test data, and finally the operation capability test data. Before sending each kind of test data, a test instruction data packet is sent first, and the instruction data packet contains the test type, test data size, test step, and expected completion time fields. After the to-be-cooperated device receives the test instruction, the corresponding action is prepared to be performed: for the presentation capability test, the display driver is called to display the test image on the screen; for the transmission capability test, the network transmission module is called to receive the test file and return the confirmation information; for the operation capability test, the test operation instruction is simulated to be received and the corresponding operation is performed.

[0070] Step S1266: The actual performance data of the corresponding to-be-cooperated device is recorded, and the actual performance data of the corresponding to-be-cooperated device includes the actual resolution when the test image is displayed, the actual transmission rate when the test file is transmitted, and the actual response time when the test operation instruction is responded.

[0071] The test performance recording part collects the actual performance data of the device in multiple ways: for the presentation capability test, the displayed test image is photographed by the image collection device connected to the device, and the actual resolution, color restoration, and detail clarity of the image are analyzed; for the transmission capability test, the transmission start time, transmission completion time, and received data verification result of the test file are recorded, and the actual transmission rate and transmission success rate are calculated; for the operation capability test, the operation instruction sending time, device response start time, and response completion time are recorded, and the actual response time and operation execution accuracy are calculated. The actual performance data recording is organized according to the test type and test serial number, and contains the test type, test serial number, actual performance value, and whether the standard is met fields.

[0072] Step S1267: The actual performance data of the corresponding to-be-cooperated device is compared with the corresponding capability item fed back by the current device, and if the actual performance data meets the requirements of the capability item description, it is determined that the current capability item is verified to pass at the actual test level; if the actual performance data does not meet the requirements of the capability item description, it is determined that the current capability item fails to be verified.

[0073] The performance data comparison part compares the actual performance data with the requirements of the current feedback capability item item by item. For example, the maximum picture resolution requirement in the presentation capability item is a specific pixel value, and if the resolution of the actual test image reaches the value and is displayed clearly, the verification is passed; the maximum data transmission rate requirement in the transmission capability item is a specific rate value, and if the actual transmission rate reaches the value and lasts for more than a certain time, the verification is passed; the operation response time requirement in the operation capability item is less than a specific millisecond value, and if the actual response time is less than the value, the verification is passed. During the comparison process, the comparison result, actual value, requirement value, and difference percentage field of each capability item are recorded.

[0074] Step S1268: For the further-verified items marked in the historical data comparison, if the actual test level verification is passed, the current capability item is retained; if the actual test level verification is failed, the current capability item is removed.

[0075] The to-be-verified item processing part checks the actual test result of the further-verified items marked in the historical data comparison. If the actual test verification is passed, the current state of the capability item is considered to be valid, retained in the capability item list, and the verification passing reason is recorded; if the actual test verification is failed, the current feedback data of the capability item is considered to be inaccurate, removed from the capability item list, and the removal reason is recorded, and the device maintenance suggestion is generated.

[0076] Step S1269: The verification results of all capability items are summarized to form a device capability item verification report, which includes the capability item list passed by verification and the capability item list failed by verification.

[0077] The verification result summary part collects the verification results of all capability items, organizes the capability item list passed by verification and the capability item list failed by verification according to the capability category. The capability item list passed by verification includes the capability item name, the current capability value, and the verification passing link field; the capability item list failed by verification includes the capability item name, the current feedback value, the actual performance value, the failure link, and the failure reason field. The above lists are integrated to form a device capability item verification report, which includes the device identification, the verification time, the total number of items, the number of passed items, the number of failed items, and the pass rate field.

[0078] Step S127: The capability items failed by verification are removed, and the real and valid presentation capability items, transmission capability items, and operation capability items are retained.

[0079] The capability item screening part reads the device capability item verification report, removes the capability items failed by verification from the current capability item list. For the presentation capability items, if the maximum picture resolution verification is failed, the capability item is removed, and only the other verified presentation capability items are retained; for the transmission capability items, if the maximum data transmission rate verification is failed, the capability item is removed, and the transmission packet loss rate, interruption recovery time, and other verified capability items are retained; for the operation capability items, if the operation response time verification is failed, the capability item is removed, and the operation instruction type, permission setting capability, and other verified capability items are retained. The screened capability item list includes the real and valid presentation capability items, transmission capability items, and operation capability items, which are the core content of the device capability list.

[0080] Step S128: create a dedicated capability record for each device to be coordinated, which contains device identification, list of presentation capability items, list of transmission capability items, and list of operation capability items, integrate the dedicated capability records of all devices to form a device capability list.

[0081] The device capability record generation part creates a dedicated capability record for each device to be coordinated. The dedicated capability record of an online device contains device identification, list of presentation capability items, list of transmission capability items, list of operation capability items, capability item update time, and verification status field; the dedicated capability record of an offline device contains device identification, offline state description, and last known capability item list field. Group the dedicated capability records of all devices by device type to form a device capability list, which is stored in a markup language format and contains list version number, generation time, number of device groups, and total number of devices fields, and supports retrieval by device identification and capability item name.

[0082] Step S130: according to the demand dimension decomposition result and the device capability list, calculate the adaptation degree of each device to different demand dimensions, and build a digital multimedia multi-device collaborative interaction link based on the adaptation degree.

[0083] The collaborative link construction part of the store digital management platform first extracts the specific demand items and priority of the three demand dimensions from the demand dimension decomposition result, and extracts the capability item list of each device from the device capability list. Then, for each demand dimension, calculate the adaptation score of each device, which considers the demand item matching degree, priority weight, and capability item performance level. Integrate the adaptation scores of the three dimensions into a comprehensive adaptation score, and select candidate collaborative devices with a comprehensive adaptation score that meets the standard. Next, select the main responsible device for each demand dimension, determine the interaction relationship between the main responsible devices, set the data transmission path, interaction order, and state feedback mechanism, and configure the backup devices. Finally, integrate the above information to build a digital multimedia multi-device collaborative interaction link, which contains main responsible device nodes, backup device nodes, data transmission path, interaction rules, and state monitoring mechanism fields.

[0084] Step S131: extract the specific demand items and demand priority of the presentation demand dimension from the demand dimension decomposition result, and extract the presentation capability item list of each device from the device capability list.

[0085] The requirement extraction part accesses the requirement dimension disassembly result document, locates the requirement dimension chapter, extracts the specific requirement items under the dimension and the priority of each requirement item. The device capability extraction part accesses the device capability list, extracts the presentation capability item list of each device according to the device identifier. The extracted presentation requirement items and the device presentation capability items are mapped according to the name to form a presentation requirement-capability mapping table, which contains the requirement item name, requirement priority, requirement description and corresponding capability item name field.

[0086] Step S132: For each specific requirement item under the presentation requirement dimension, the matching situation of the presentation capability items of each device and the specific requirement item is compared. If the device presentation capability item meets the requirement of the specific requirement item, it is recorded as a matching success, otherwise it is recorded as a matching failure.

[0087] The presentation requirement matching part traverses each specific requirement item in the presentation requirement-capability mapping table, and compares in turn whether the corresponding presentation capability items of each device meet the requirement. For the content picture clarity requirement, the maximum picture resolution capability item of the device is checked. If the maximum resolution of the device meets the requirement, the matching is successful; otherwise, the matching fails. For the picture color restoration requirement, the color processing capability range of the device is checked. If the color gamut contains the requirement, the matching is successful; otherwise, the matching fails. For the picture smoothness requirement, the frame rate range capability item of the device is checked. If the frame rate range of the device contains the requirement interval, the matching is successful; otherwise, the matching fails. For the picture size adaptation requirement, the picture scaling mode capability item of the device is checked. If it contains the scaling mode of the requirement, the matching is successful; otherwise, the matching fails. The matching result is recorded as the matching state and matching degree fields.

[0088] Step S133: According to the requirement priority of each specific requirement item under the presentation requirement dimension, an adaptation weight is set for each specific requirement item. The adaptation weight of the specific requirement item whose requirement priority meets the preset priority standard is set to meet the preset weight value standard.

[0089] The adaptation weight setting part reads the requirement priority of the presentation requirement item, and queries the preset priority-weight mapping rule. For example, the content picture clarity requirement is high priority, and the adaptation weight is set to a higher value; the picture color restoration requirement is medium priority, and the adaptation weight is set to an intermediate value; the picture smoothness requirement is medium priority, and the adaptation weight is set to an intermediate value; the picture size adaptation requirement is low priority, and the adaptation weight is set to a lower value. After the weight setting is completed, a presentation requirement item weight table is generated, which contains the requirement item name, requirement priority and adaptation weight fields, and the weight sum is a specific value.

[0090] Step S134: Based on the matching results of each device on each specific demand item under the presentation demand dimension and the adaptation weight, the adaptation score of each device to the presentation demand dimension is calculated. The adaptation score is calculated by multiplying the matching result of each specific demand item by the corresponding adaptation weight.

[0091] The presentation adaptation score calculation part initializes the presentation adaptation score value of each device to zero. For each presentation demand item, if the device matches successfully, the adaptation weight value of the demand item is added to the presentation adaptation score; if the matching fails, it is not added. After the calculation is completed, the device presentation adaptation score table is generated, which contains the device identifier, the matching result of each demand item, and the adaptation score field.

[0092] Step S135: In the same way, the adaptation scores of each device to the transmission demand dimension and the operation demand dimension are calculated respectively.

[0093] The transmission demand adaptation score calculation process is similar to the presentation demand: the specific demand items and priorities of the transmission demand dimension are extracted, and the adaptation weight is set; the matching of the transmission capability items of each device and the transmission demand items is compared; the transmission adaptation score is calculated. The operation demand adaptation score calculation process is the same: the specific demand items and priorities of the operation demand dimension are extracted, the adaptation weight is set, the matching of the device operation capability items and the operation demand items is compared, and the operation adaptation score is calculated. Finally, the device transmission adaptation score table and the device operation adaptation score table are formed.

[0094] Step S136: The adaptation scores of each device in the three demand dimensions are normalized respectively, and the normalized three-dimensional adaptation scores are weighted and summed according to the preset proportion to obtain the comprehensive adaptation score of each device.

[0095] The normalization processing part first normalizes the adaptation scores of the three dimensions, mapping the scores to a specific interval. The normalization method uses min-max standardization, and the formula is: the normalized score is calculated according to the relationship of the original score, the minimum value, and the maximum value. After normalization, the comprehensive adaptation score is calculated by weighted sum according to the preset proportion. The device comprehensive adaptation score table is generated, which contains the device identifier, the three-dimensional normalized score, and the comprehensive adaptation score field.

[0096] Step S137: According to the comprehensive adaptation scores of each device, the devices whose comprehensive adaptation scores reach the preset adaptation score threshold are selected as candidate collaborative devices.

[0097] The candidate device screening part compares the comprehensive adaptation score in the device comprehensive adaptation score table with a preset adaptation score threshold, and screens out devices with a score greater than or equal to the threshold as candidate collaborative devices. The preset adaptation score threshold is set according to historical collaboration effect data and current scene requirements, and is stored under the collaboration parameter node of the system configuration file. The screening result generates a candidate collaborative device list, including device identification, comprehensive adaptation score, normalized score of each dimension, and device online state field.

[0098] Step S138: For each requirement dimension in the requirement dimension decomposition result, select the device with the highest dimension adaptation score from the candidate collaborative devices as the primary responsible device for the requirement dimension.

[0099] The primary responsible device selection part processes the presentation, transmission, and operation three requirement dimensions in turn. For the presentation requirement dimension, query the device with the highest presentation adaptation score in the candidate collaborative device list, and select it as the primary responsible device for the presentation requirement dimension; for the transmission requirement dimension, select the device with the highest transmission adaptation score as the primary responsible device for the transmission requirement dimension; for the operation requirement dimension, select the device with the highest operation adaptation score as the primary responsible device for the operation requirement dimension. If there are devices with the same adaptation score, further compare the historical collaboration performance data (such as average response time, operation success rate) of the devices, and select the device with better performance as the primary responsible device. The selected primary responsible device record includes device identification, responsible dimension, adaptation score, and historical performance rating fields.

[0100] Step S139: Determine the interaction relationship between the primary responsible devices of each requirement dimension. The primary responsible device of the presentation requirement dimension needs to provide content presentation parameters to the primary responsible device of the transmission requirement dimension, the primary responsible device of the transmission requirement dimension needs to provide content transmission status to the primary responsible device of the operation requirement dimension, and the primary responsible device of the operation requirement dimension needs to feedback operation instructions to the primary responsible device of the presentation requirement dimension.

[0101] The interaction relationship determination part defines the information interaction direction and content between the primary responsible devices according to the business logic association of the three requirement dimensions. The primary responsible device of the presentation requirement dimension needs to provide content presentation parameters to the primary responsible device of the transmission requirement dimension before starting content presentation, so that the transmission device can adjust the transmission strategy; the primary responsible device of the transmission requirement dimension needs to push the content transmission status to the primary responsible device of the operation requirement dimension in real time during transmission, so that the operation device can master the transmission progress; the primary responsible device of the operation requirement dimension needs to feedback operation instructions to the primary responsible device of the presentation requirement dimension immediately after receiving user operation instructions, so that the presentation device can adjust the presentation content in time. The interaction relationship record includes interaction direction (source device-target device), interaction content type, interaction frequency, and data format fields.

[0102] Step S1310: According to the interaction relationship of each main responsible device, set the data transmission path between devices, the interaction operation instruction interaction sequence and the state feedback mechanism.

[0103] Step S13101: Determine the content presentation parameter type that the presentation demand dimension main responsible device needs to provide to the transmission demand dimension main responsible device, determine the transmission format and transmission frequency of the content presentation parameter, and the content presentation parameter type includes target presentation resolution, target presentation frame rate, color space standard and picture scaling ratio.

[0104] The parameter type determination part analyzes the specific demand items of the presentation demand dimension, extracts the key presentation parameters that affect data transmission, such as target presentation resolution determines the size of the transmission data, target presentation frame rate affects the real-time requirement of transmission, color space standard is related to data encoding method, and picture scaling ratio involves preprocessing operation before transmission. The transmission format is determined as a structured format, including parameter name, parameter value, unit, and effective timestamp field. The transmission frequency is set to two modes: initial parameter one-time transmission and dynamic adjustment parameter real-time transmission.

[0105] Step S13102: Based on the transmission format and transmission frequency of the content presentation parameter, combined with the transmission protocol type in the transmission capability item of the presentation demand dimension main responsible device and the transmission demand dimension main responsible device, select the appropriate transmission channel, and set the one-way data transmission path from the presentation demand dimension main responsible device to the transmission demand dimension main responsible device.

[0106] The transmission channel selection part compares the transmission protocol types supported by the presentation and transmission main responsible devices, and preferentially selects the protocol that supports high bandwidth and low delay. According to the transmission frequency requirement, select reliable transmission protocol for one-time transmission of initial parameters, and select real-time transmission protocol for real-time transmission of dynamic parameters. The transmission path setting part marks the physical connection nodes (such as switch ports, router interfaces) of the source device and the target device in the network topology graph, configures the network parameters (such as IP address, subnet mask, gateway) of the nodes, and ensures that data is transmitted according to the preset path. After the path setting is completed, the connectivity test is carried out to verify whether the data can be successfully transmitted from the source device to the target device.

[0107] Step S13103: Determine the content transmission state type that the transmission demand dimension main responsible device needs to provide to the operation demand dimension main responsible device, and determine the feedback period and feedback format of the content transmission state information, and the content transmission state type includes current transmission rate, transmitted data volume, transmission packet loss and transmission delay.

[0108] The state type determination part extracts key state indicators from specific demand items in the transmission demand dimension. The current transmission rate reflects transmission efficiency, the amount of transmitted data indicates progress, the transmission packet loss situation reflects data integrity, and the transmission delay affects interactive real-time performance. The feedback period is set according to the transmission delay control requirement, and the period length ensures that the operating device can timely master the transmission state change. The feedback format adopts a structured format, including state type, value, unit, collection timestamp, and state level (normal / warning / abnormal) fields.

[0109] Step S13104: According to the feedback period and feedback format of the content transmission state information, combined with the transmission delay control capability in the transmission capability item of the transmission demand dimension master responsible device and the operation demand dimension master responsible device, a one-way data transmission path from the transmission demand dimension master responsible device to the operation demand dimension master responsible device is set, so that the content transmission state information arrives within the feedback period.

[0110] The transmission delay evaluation part calculates the feasibility of the state information arriving within the feedback period according to the transmission delay control capability item (such as historical average transmission delay, maximum allowed delay) of the transmission and operation master responsible device. If the historical average transmission delay is less than half of the feedback period, it is determined that the path is feasible; otherwise, the feedback period is adjusted or the transmission path is optimized (such as selecting a shorter physical link). The transmission path setting process is similar to step S13102, network parameters are configured and connectivity and delay tests are performed to ensure that the state information arrives within the specified time.

[0111] Step S13105: Determine the operation instruction type that the operation demand dimension master responsible device needs to feed back to the presentation demand dimension master responsible device, which includes picture switching instruction, volume adjustment instruction, play progress adjustment instruction, and picture effect adjustment instruction, and determine the encoding format and priority of the operation instruction.

[0112] The instruction type determination part analyzes the specific demand items of the operation demand dimension, combined with common user interaction scenarios, to define the operation instruction type. The encoding format adopts binary encoding, including instruction type identification (occupying a certain number of bits), instruction parameters (occupying a certain number of bits), and check bits (occupying a certain number of bits) fields, to reduce the transmission data volume. The priority is divided into three levels: emergency instruction (such as emergency stop playing), high priority instruction (such as picture switching), and ordinary priority instruction (such as volume fine tuning). The priority field occupies a certain number of bits, which is used for priority scheduling in the transmission process.

[0113] Step S13106: According to the encoding format and priority of the operation instruction, combined with the operation response speed in the operation capability item of the presentation demand dimension master responsible device, set the one-way interactive operation instruction transmission path from the operation demand dimension master responsible device to the presentation demand dimension master responsible device, and the operation instruction with priority meeting the preset priority standard adopts the priority transmission channel.

[0114] The transmission path setting part selects a low-delay transmission protocol according to the operation response speed requirement (such as instruction sending delay of the operation device and instruction processing delay of the presentation device). Independent transmission channels are allocated for operation instructions of different priorities, and the priority transmission channel is configured with higher bandwidth guarantee and shorter routing path. For example, an independent high-speed channel is used for emergency instructions, and a shared channel is used for ordinary instructions. After the configuration is completed, instruction transmission test is performed to verify whether the transmission delay of instructions of different priorities meets the operation response speed requirement.

[0115] Step S13107: Determine the interactive operation instruction interaction sequence. The operation demand dimension master responsible device needs to receive the user operation instruction first, and then feed back the user operation instruction to the presentation demand dimension master responsible device, while the transmission demand dimension master responsible device needs to feed back the content transmission state to the operation demand dimension master responsible device in real time, and the presentation demand dimension master responsible device needs to send the new content presentation parameter to the transmission demand dimension master responsible device after adjusting the content presentation parameter according to the user operation instruction.

[0116] The interaction sequence definition part draws a time sequence flow chart to clearly define the operation steps and time sequence of each device. The interactive process of the user operation instruction is: user operation—operation device receives instruction—operation device permission verification—operation device sends instruction—presentation device receives instruction—presentation device executes instruction—presentation device feeds back result. The interactive process of the content transmission state is: transmission device collects state—transmission device encodes state information—transmission device sends periodically—operation device receives and analyzes. The interactive process of content presentation parameter adjustment is: presentation device adjusts parameter—presentation device encodes new parameter—presentation device sends parameter—transmission device receives and applies. The time sequence flow chart includes nodes (devices), arrows (data flow direction), and time markers (step interval) fields.

[0117] Step S13108: Set the abnormal feedback process in the state feedback mechanism. If the transmission demand dimension master responsible device detects that the transmission packet loss rate exceeds the preset packet loss rate allowed range, it needs to immediately send a transmission exception notification to the operation demand dimension master responsible device. After receiving the transmission exception notification, the operation demand dimension master responsible device suspends the issuance of new interactive operation instructions to the presentation demand dimension master responsible device, and feeds back the transmission exception information to the user.

[0118] The abnormal type definition part classifies the transmission packet loss rate exceeding the preset range as a serious transmission abnormality, which may cause content presentation to be stuck or error. The abnormality detection part is monitored by the network monitoring module of the transmission device in real time. The sampling period is a certain length of time, and if the threshold value is exceeded for a plurality of continuous periods, the abnormality is triggered. The abnormality notification format includes the abnormality type, the current value, the threshold value, the occurrence time, and the recommended handling measure field. After receiving the notification, the operating device calls the instruction queue management module to suspend the issuance of new instructions, and displays the transmission abnormality information (such as text prompts and icon flashing) through the user interface.

[0119] Step S13109: Set the normal feedback process in the state feedback mechanism, and each main responsible device needs to send state normal confirmation information to the associated device according to the preset feedback period.

[0120] The normal state is defined as that the device operating parameters are within the normal range and no abnormal event occurs. The feedback period is set according to the importance of the device, and the main responsible device has a shorter period and the alternative device has a longer period. The confirmation information format includes the device identifier, the responsible dimension, the current state code (normal / busy / idle), the resource occupancy rate, and the next feedback time field. After receiving the confirmation information, the associated device updates the device state table, and if the confirmation information is not received within the preset period, it is determined that the device is lost, and the fault handling process is triggered.

[0121] Step S131010: The set data transmission path, interactive operation instruction interaction sequence and state feedback mechanism are arranged into a link interaction specification, and the link interaction specification is used as a component of the multi-device collaborative interaction link, so that each main responsible device executes the interactive operation according to the link interaction specification.

[0122] The specification arrangement part integrates the network parameter configuration of the data transmission path, the timing flowchart of the interactive operation instruction interaction sequence, and the rule clauses of the state feedback mechanism into a unified link interaction specification document. The document includes version number, effective date, applicable device list, detailed specification chapter (transmission specification, interaction specification, feedback specification), and appendix (term definition, abnormal code table) fields. The document is distributed to the local storage of each main responsible device, and the device loads the specification document and executes the interactive operation according to it when it starts.

[0123] Step S1311: At least one alternative device is configured for each main responsible device, and the alternative device selection standard is the second highest candidate collaborative device adapted to the demand dimension score.

[0124] The alternative device configuration part queries, for each main responsible device, the device with the second highest adaptation score in the candidate collaborative device list under the same demand dimension as the alternative device. If the adaptation score of the second highest score device is within a preset allowed range (e.g., not more than a certain percentage) from that of the main responsible device, the second highest score device is selected as the alternative device; otherwise, the next higher score device is queried. The number of alternative devices configured for each main responsible device is determined according to the demand importance, with two alternative devices configured for the transmission dimension and one alternative device configured for the operation dimension. The alternative device record includes the main responsible device identifier, the alternative device identifier, the adaptation score, and the switching trigger condition field.

[0125] Step S1312: The main responsible device, the alternative device, the interaction relationship between devices, the data transmission path, the interaction operation instruction interaction sequence, and the state feedback mechanism are integrated to construct a digital multimedia multi-device collaborative interaction link.

[0126] The link integration part integrates the main responsible device, the alternative device, the interaction relationship, the transmission path, the interaction sequence, and the feedback mechanism determined in the above steps using a directed graph data structure to form a multi-device collaborative interaction link. The nodes of the directed graph represent devices (main responsible / alternative), the edges represent data transmission paths, and the attributes of the edges include the interaction content type, the transmission protocol, the priority, and the state feedback rule field. After the link is constructed, simulation testing is performed to simulate the collaborative effect under different scenarios (normal operation, device failure, network fluctuation) to verify the stability and reliability of the link. After the testing is passed, the link configuration data is stored in the collaborative link configuration file as the basis for subsequent interaction operation execution.

[0127] Step S140: According to the multi-device collaborative interaction link, interaction operation instructions are issued to each device to be coordinated to guide each device to perform corresponding digital multimedia processing actions and obtain interaction result data fed back by the device processing actions.

[0128] The collaborative instruction issuing part reads the main responsible device, the interaction sequence, and the instruction format from the collaborative link configuration file to generate interaction operation instructions. The instruction content is determined according to the specific demand items in the demand dimension resolution result and the capability items in the device capability list, and includes the instruction identifier, the target device identifier, the action type (presentation / transmission / operation), the action parameter, and the execution time requirement field. Through the data transmission path in the multi-device collaborative interaction link, the instructions are issued to each main responsible device in the interaction sequence. During the execution of the device actions, the execution status is monitored in real time, and the processing progress and result data fed back by the device are received. After all device actions are executed, the processing results are summarized to form interaction result data, which includes the instruction execution log, the device performance data, and the user feedback information field.

[0129] Step S141: Analyze the main responsible device identifier, the alternative device identifier and the responsibility corresponding to each device in the multi-device collaborative interaction link, determine the digital multimedia processing action type that each device needs to perform, which includes content decoding action, content rendering action, content transmission action, operation instruction receiving action and operation result feedback action.

[0130] The link analysis part reads the directed graph data in the collaborative link configuration file, extracts the identifier and responsibility description of each node (device). According to the responsibility description, determine the processing action type of the device: the presentation demand dimension main responsible device performs content decoding and content rendering action, the transmission demand dimension main responsible device performs content transmission action, and the operation demand dimension main responsible device performs operation instruction receiving and operation result feedback action. The mapping relationship between action type and device is recorded in the device action allocation table, including device identifier, responsibility description, action type list, and dependent device identifier field (such as rendering action depends on decoding action completion).

[0131] Step S142: According to the requirements of each specific demand item in the demand dimension decomposition result, determine the specific parameters of each device to perform the corresponding digital multimedia processing action. The specific parameters of content rendering action include rendering resolution, rendering frame rate and rendering color parameters. The specific parameters of content transmission action include transmission rate upper limit, transmission data verification method and transmission delay control target.

[0132] The action parameter determination part converts the specific demand items in the demand dimension decomposition result into action parameters executable by the device. For example, the content picture clarity requirement of the presentation demand dimension is converted into the rendering resolution parameter, the picture fluency requirement is converted into the rendering frame rate parameter, and the picture color restoration requirement is converted into the rendering color parameter (such as color gamut, color temperature). The transmission speed requirement of the transmission demand dimension is converted into the transmission rate upper limit parameter, the data integrity requirement is converted into the transmission data verification method (such as CRC verification, MD5 verification), and the transmission delay control requirement is converted into the transmission delay control target parameter. The determination of parameter value needs to meet the demand requirement and device capability limitation at the same time. If the demand parameter exceeds the device capability, set it according to the maximum device capability value, and record the parameter adjustment description.

[0133] Step S143: According to the interaction operation instruction interaction sequence in the multi-device collaborative interaction link, generate the interaction operation instruction corresponding to each device, which includes device identifier, digital multimedia processing action type, digital multimedia processing action specific parameter and execution time requirement.

[0134] The instruction generation part generates instruction sequences according to the device action allocation table and the action parameter determination result according to the interactive operation instruction interaction sequence. Each instruction includes a fixed field (instruction header) and a variable field (instruction body). The instruction header includes instruction version, instruction type, target device identification, instruction length, and check code field. The instruction body includes digital multimedia processing action type, specific parameter list (parameter name-parameter value pair), execution start time, and execution timeout time field. The execution time requirement is set according to the time node in the interactive sequence flowchart to ensure that the devices perform actions in sequence. After the instructions are generated, syntax checking and semantic checking are performed to ensure that the instruction format is correct and the parameters are valid.

[0135] Step S144: According to the data transmission path in the multi-device cooperative interaction link, a direct communication channel is established between each device. The setting of the direct communication channel needs to meet the transmission protocol type requirement in the device transmission capability item.

[0136] The communication channel establishment part establishes a direct communication connection between each device according to the network parameters (IP address, port number, protocol type) recorded in the data transmission path. For TCP protocol, the three-way handshake process is executed; for UDP protocol, the port listening and data packet format are configured. During the channel establishment process, the protocol support in the device transmission capability item is verified. If the specified protocol device in the path does not support it, the protocol and path are reselected from the link construction step. After the communication channel is successfully established, bandwidth testing and delay testing are performed, and the channel performance parameters (bandwidth, delay, and jitter) are recorded as the basis for subsequent instruction transmission quality evaluation.

[0137] Step S145: According to the interactive operation instruction interaction sequence, the interactive operation instructions corresponding to each device are sequentially sent to the target device through the corresponding direct communication channel.

[0138] The instruction sending part sends the interactive operation instructions to each target device in the order of the interactive operation instruction interaction sequence flowchart. Before sending, the communication channel state of the target device is checked, and the instruction is sent only when the channel is normal. The sending process adopts an acknowledgement mechanism, and the sender waits for the return of the instruction reception acknowledgement information from the receiver. If no acknowledgement is received within the timeout time, the instruction is re-sent, and the number of re-sending times does not exceed the pre-set upper limit. The instruction sending record includes instruction identification, target device identification, sending time, sending result, and acknowledgement receiving time field.

[0139] Step S146: During the interactive operation instruction sending process, the transmission state of the direct communication channel is monitored in real time, and the interactive operation instruction sending time, interactive operation instruction receiving time, and data packet loss during transmission are recorded.

[0140] The transmission state monitoring part collects transmission state parameters in real time through the state monitoring interface of the communication channel: the interactive operation instruction issuing time is the timestamp when the instruction leaves the sending buffer, the interactive operation instruction receiving time is the timestamp when the receiving party returns the confirmation, and the data packet loss is counted by the sequence number checking method (the difference between the sending sequence number and the receiving confirmation sequence number). The monitoring data is stored in the transmission state log in chronological order, including the timestamp, channel identifier, instruction identifier, issuing time, receiving time, packet loss number, and packet loss rate fields. When the packet loss rate exceeds the preset threshold, a channel quality alarm is triggered.

[0141] Step S147: If any device does not receive the interactive operation instruction within the specified time, or the received interactive operation instruction has data missing, the standby device mechanism in the multi-device collaborative interaction link is started, the interactive operation instruction is issued to the corresponding standby device, and the data transmission path is updated.

[0142] For example, step S1471: a preset interactive operation instruction receiving timeout time is set. The receiving timeout time is set based on the transmission delay control requirement in the device transmission capability item and the time node requirement in the interactive operation instruction interaction sequence.

[0143] The timeout time setting part considers two factors: the transmission delay control requirement in the device transmission capability item (such as a specific multiple of the average transmission delay), to ensure that the timeout time is greater than the time required for normal transmission; and the time node requirement in the interactive operation instruction interaction sequence (such as the execution start time of the next instruction), to ensure that the timeout processing does not affect the overall interaction sequence. The timeout time is set according to the instruction type, and the timeout time for instructions with high real-time requirements (such as operation instructions) is shorter, and the timeout time for instructions with large data (such as presentation parameter instructions) is longer. The timeout time is stored in the timeout parameter node of the instruction configuration file.

[0144] Step S1472: After issuing the interactive operation instruction to the main responsible device, start the timer to monitor in real time whether the current main responsible device returns the interactive operation instruction receiving confirmation information within the receiving timeout time.

[0145] The timing monitoring part starts the timer immediately after the instruction is issued, and the timer duration is set to the receiving timeout time. At the same time, the receiving confirmation information on the communication channel is listened to, the instruction identifier in the confirmation information is parsed, and the current issued instruction is matched. If the matching receiving confirmation information is received before the timer expires, the timing is stopped and the confirmation time is recorded; if the timer expires and it is not received, a timeout event is triggered.

[0146] Step S1473: If the interactive operation instruction receiving confirmation information returned by the current main responsible device is not received after the timing ends, it is determined that the current main responsible device does not receive the interactive operation instruction within the specified time.

[0147] The timeout judgment part checks the receiving confirmation information buffer when the timer expires, and if there is no confirmation information matching the current instruction identification, a "not received on time" judgment result is generated. At the same time, the transmission state log is inquired to obtain the issuing time of the instruction, the channel state, the packet loss condition and other information as supplementary description of the judgment basis.

[0148] Step S1474: If the interactive operation instruction receiving confirmation information returned by the current main responsible device is received, but the interactive operation instruction receiving confirmation information contains a prompt of missing interactive operation instruction data, it is judged that the interactive operation instruction received by the current main responsible device has data missing.

[0149] The data missing judgment part analyzes the state field in the receiving confirmation information. If the state field value is "data missing" or contains a specific missing field list, it is judged as "data missing". Data missing may be caused by transmission packet loss, check error, parsing failure and other reasons. The missing reason code is contained in the confirmation information for subsequent analysis.

[0150] Step S1475: When the interactive operation instruction is not received on time or the interactive operation instruction has data missing, the corresponding alternative device identification and the capability item information of the alternative device of the current main responsible device are queried from the multi-device cooperative interaction link.

[0151] The alternative device query part accesses the alternative device configuration table in the cooperative link configuration file, and retrieves the corresponding alternative device identification list according to the current main responsible device identification. Then, the capability item information (presentation, transmission, operation capability item) of the alternative device is extracted from the device capability list, and the capability item related to the current interactive operation instruction (such as the hardware configuration and software module required for instruction processing) is focused on.

[0152] Step S1476: Verify whether the alternative device is currently in an idle state. The verification method is to send a state query request to the alternative device, receive the current task load condition feedback by the alternative device, and if the current task load is lower than the preset task load threshold, it is judged that the alternative device is in an idle state.

[0153] The idle state verification part sends a state query request instruction to the alternative device, which contains the query identification, the query timestamp, and the load index type (CPU utilization, memory occupancy, task queue length) fields. After receiving the instruction, the alternative device collects the current task load data and returns the response in the requested format, which contains the device identification, the response timestamp, and the current value of each load index field. Compare each load index with the preset task load threshold. If all indexes are lower than the threshold, it is judged as an idle state; otherwise, it is a busy state.

[0154] Step S1477: If the alternative device is in an idle state, based on the interactive operation instruction of the original primary responsible device, the digital multimedia processing action specific parameters are adjusted in combination with the capability item information of the alternative device, so that the digital multimedia processing action specific parameters meet the capability range of the alternative device.

[0155] The parameter adjustment part compares the digital multimedia processing action specific parameters in the original instruction with the upper limit of the capability item of the alternative device. If the parameter value exceeds the upper limit of the capability, the upper limit of the capability is adjusted. For example, the original instruction has a rendering resolution of a specific pixel value, and the maximum resolution of the alternative device is a lower pixel value. The rendering resolution is adjusted to the maximum resolution of the alternative device. The upper limit of the transmission rate of the original instruction is higher than the maximum rate supported by the alternative device, and the maximum rate of the alternative device is adjusted. The parameter adjustment record includes the original parameter value, the adjusted parameter value, the adjustment reason (capability limitation), and the adjustment time field.

[0156] Step S1478: An interactive operation instruction for the alternative device is generated, and the interactive operation instruction for the alternative device includes the core information of the original interactive operation instruction and the adjusted digital multimedia processing action specific parameters.

[0157] The alternative instruction generation part copies the core information (instruction identifier, action type, execution time requirement) of the original interactive operation instruction, replaces it with the adjusted digital multimedia processing action specific parameters, adds the alternative device identifier field and the switching reason field (not received on time / data missing). The instruction format remains consistent with the original instruction to ensure that the alternative device can correctly parse it. After generation, parameter verification is performed to ensure that the adjusted parameters are within the capability range of the alternative device.

[0158] Step S1479: All data transmission paths corresponding to the original primary responsible device in the multi-device collaborative interaction link are queried, and the path nodes that need to be updated are determined, and the nodes of the original primary responsible device are replaced with the nodes of the alternative device.

[0159] The path update part traverses the directed graph data structure of the multi-device collaborative interaction link, finds all edges (data transmission paths) with the original primary responsible device as the target node or source node. For the path with the original primary responsible device as the target node (such as the operation device - the original rendering device), the target node is replaced with the alternative device. For the path with the original primary responsible device as the source node (such as the original rendering device - the transmission device), the source node is replaced with the alternative device. The updated path node record includes the original node identifier, the new node identifier, the path type, and the update time field.

[0160] Step S14710: According to the transmission protocol type in the transmission capability item of the alternative device, the transmission protocol configuration of the data transmission path is adjusted, so that the new data transmission path meets the transmission requirements of the alternative device.

[0161] The protocol configuration adjustment part reads the supported transmission protocol type and parameters (such as port number, encryption method, compression algorithm) from the transmission capability item of the alternative device, and compares the transmission protocol configuration of the original path. If the protocol types are different, the protocol type field of the path is modified; if the protocol parameters are different (such as the original path uses a specific port and the alternative device only supports another port), the corresponding parameters are updated. After the configuration is adjusted, the connectivity and transmission performance of the new path are tested to ensure that the transmission requirements of the interaction operation instruction are met.

[0162] Step S14711: The interaction operation instruction for the alternative device is issued to the alternative device through the updated data transmission path.

[0163] The alternative instruction issuing part uses the updated data transmission path to issue the interaction operation instruction to the alternative device according to the same issuing process (establishing a channel, sending an instruction, and waiting for confirmation) as the original instruction. During the issuing process, the transmission delay and packet loss rate are monitored to ensure that the alternative device can receive and process the instruction within the remaining execution time.

[0164] Step S14712: A device replacement notification is sent to other devices associated with the original main responsible device, and the device replacement notification contains the new alternative device identifier and the updated data transmission path information, so that the other devices associated with the original main responsible device can subsequently interact with the alternative device.

[0165] The associated device notification part identifies the associated devices (devices that have data exchange in the interaction relationship) of the original main responsible device, and sends a device replacement notification instruction to them. The notification instruction contains the original device identifier, the new device identifier (the alternative device), the updated data transmission path (IP address, port, protocol), and the effective time field. After receiving the notification instruction, the associated devices update the local device mapping table and path configuration, and subsequently send data to the alternative device during the interaction operation.

[0166] Step S14713: The reception of the interaction operation instruction for the alternative device by the alternative device is monitored. If the alternative device successfully receives the interaction operation instruction for the alternative device and returns the interaction operation instruction reception confirmation information, the starting of the alternative device mechanism and the updating of the data transmission path are completed. If the alternative device still fails to successfully receive the interaction operation instruction for the alternative device, the above steps are repeated, the next alternative device is selected, and the process is repeated until the interaction operation instruction for the alternative device is successfully issued.

[0167] The alternative device monitoring part starts the same receiving monitoring process as the main responsible device after issuing the instruction to the alternative device. If successful reception is achieved, a replacement success log is recorded, and the current main responsible device in the collaborative link configuration file is updated to the alternative device; if unsuccessful reception is achieved, the next alternative device is selected from the alternative device list, and the process of steps S1475 to S14712 is repeated. If all alternative devices cannot receive the instruction, a high-level alarm is triggered to notify the system administrator for manual intervention.

[0168] Step S148: After each device receives the interactive operation instruction, the digital multimedia processing action is executed according to the digital multimedia processing action type and the digital multimedia processing action specific parameters in the interactive operation instruction, and the processing progress, processing resource occupation, and abnormal information in the processing process are recorded in real time during the execution of the digital multimedia processing action.

[0169] The device execution part parses the received interactive operation instruction, and calls the corresponding processing module according to the digital multimedia processing action type: the content decoding action calls the decoder module, the content rendering action calls the rendering engine, the content transmission action calls the transmission module, the operation instruction receiving action calls the input processing module, and the operation result feedback action calls the output feedback module. During the processing, the processing progress (such as decoding completion percentage, rendering frame count, and transmission byte number), processing resource occupation (CPU / memory / network usage), and abnormal information (error code, error description, and occurrence time) are collected in real time and stored in the local processing log.

[0170] Step S149: When the device completes the digital multimedia processing action, a processing result report is generated, which contains the digital multimedia processing action completion time, the digital multimedia processing result description, and the digital multimedia processing process record.

[0171] The result report generation part collects the data in the local processing log after the digital multimedia processing action is executed (the processing progress reaches 100% or a stop instruction is received), and generates a processing result report. The digital multimedia processing action completion time is the timestamp of the end of the last processing step; the digital multimedia processing result description is the summary of the output data after processing (such as the resolution of the rendered image, the total number of transmission data bytes, and the execution status of the operation instruction); and the digital multimedia processing process record is the timestamp and state of the key processing node (such as the decoding start / end time, the rendering frame rate change, and the number of successful / failed transmissions). The report format adopts a structured format, including the report identifier, the device identifier, the instruction identifier, the completion time, the result description, and the process record field.

[0172] Step S1410: report the processing result to the control end that initiates the interactive operation instruction through the direct communication channel, and the control end that initiates the interactive operation instruction collects the processing result reports of all devices, the interactive operation instruction transmission records and the device processing process records, integrates the processing result reports of all devices, the interactive operation instruction transmission records and the device processing process records, and forms the interactive result data.

[0173] The result feedback part sends the processing result report to the control end (store digital management platform) through the direct communication channel (the same as the instruction issuing channel). The result collection part of the control end listens to the feedback channel of each device, receives and stores the processing result report, the interactive operation instruction transmission record (the issuing record of step S145) and the device processing process record (the processing log of step S148). In the integration process, the result data of different devices is associated according to the instruction identifier, and the interactive result data is formed as a unit of instruction. The interactive result data includes instruction identifier, participating device list, each device processing result report, transmission record summary, processing process record summary, and overall execution status (success / partial success / failure) field.

[0174] Step S150: based on the interactive result data, adjusting the adaptation degree calculation standard of each device to different demand dimensions and the node configuration of the multi-device collaborative interaction link, realizing the dynamic optimization of multi-device collaborative interaction.

[0175] The dynamic optimization part analyzes the interactive result data, evaluates the accuracy of the current adaptation degree calculation standard (the consistency of the actual performance of the device and the adaptation score) and the rationality of the collaborative link configuration (device load, transmission efficiency, fault tolerance). For the adaptation degree calculation standard, if the actual performance of the device and the adaptation score deviate greatly, adjust the demand item weight or the matching judgment rule; for the collaborative link configuration, if the main responsible device frequently fails or the resource is overloaded, replace the main responsible device or add the standby device. The optimized standard and configuration are applied to the next collaborative interaction process, and the dynamic optimization is realized through continuous iteration.

[0176] Step S151: analyze the processing result report in the interactive result data, extract the completion quality information of each device processing action, and present whether the picture clarity in the processing result of the demand dimension main responsible device meets the demand requirement, whether the picture fluency meets the standard, whether the transmission speed in the processing result of the transmission demand dimension main responsible device meets the standard, and whether the transmission packet loss rate is within the control range.

[0177] The quality analysis part extracts key quality indicators from the processing result report: the picture clarity of the presentation device (the ratio of the actual resolution to the required resolution), the picture smoothness (the ratio of the actual frame rate to the required frame rate, the number of stalls), the transmission speed of the transmission device (the ratio of the actual average rate to the required rate), the transmission packet loss rate (the ratio of the actual packet loss rate to the control threshold), the response time of the operation device (the ratio of the actual response time to the required response time), and the instruction synchronization success rate. Compare the above indicators with the demand requirements to evaluate whether the completion quality meets the standard (the ratio is greater than or equal to 1, otherwise it is not up to standard). The analysis result is recorded in the completion quality evaluation table, including the device identifier, the responsible dimension, the quality indicator, the actual value, the demand value, and the standard reaching state field.

[0178] Step S152: According to the processing completion quality information, judge whether the actual adaptation performance of each device in the corresponding demand dimension is consistent with the previously calculated adaptation score. If the actual completion quality does not meet the demand requirement, and the adaptation score value exceeds the preset adaptation score threshold, mark that the adaptation calculation standard of the device corresponding to the demand dimension has deviation.

[0179] The adaptation consistency judgment part compares the completion quality reaching standard of the device with the adaptation score: if the device adaptation score is higher than the preset adaptation score threshold (indicating good adaptation), but the completion quality does not meet the standard (actual performance is poor), it is determined that the adaptation calculation standard has deviation; if the adaptation score is lower than the threshold and the completion quality does not meet the standard, or the adaptation score is higher than the threshold and the completion quality meets the standard, it is determined to be consistent. The deviation mark includes the device identifier, the demand dimension, the adaptation score, the completion quality reaching standard, and the deviation degree (slight / severe) field. The deviation degree is calculated according to the difference between the actual value and the demand value (the larger the difference, the more serious the deviation).

[0180] Step S153: For the marked adaptation calculation standard deviation, re-evaluate the adaptation weight of each specific demand item in the demand dimension. If it is determined according to the interaction result data that the actual processing quality of any one specific demand item deviates beyond the preset deviation allowed range, and the adaptation weight value of the specific demand item is lower than the preset adaptation weight low threshold, increase the adaptation weight value of the specific demand item to meet the preset weight value standard.

[0181] The weight adjustment part analyzes the actual processing quality deviation (difference between actual value and demand value) of each specific demand item under the demand dimension of the label bias. If the deviation of a demand item exceeds the preset deviation allowed range (such as a difference greater than a certain percentage), and its current adaptive weight is lower than the preset adaptive weight lower threshold (such as lower than a certain percentage of the average weight of the same dimension), it is determined that the weight setting of the demand item is too low and cannot fully reflect its impact on the interactive experience. The adjustment method is to increase the weight to the preset weight numerical standard (such as the average weight of the high-priority demand item in the same dimension), while proportionally reducing the weight of other demand items with smaller deviations, ensuring that the total weight within the dimension remains unchanged. The adjustment record contains the demand dimension, demand item name, original weight, new weight, adjustment reason, and adjustment time fields.

[0182] Step S154: Redefine the matching judgment standard of specific demand items and device capability items. If the previous judgment is successful but the actual processing quality does not meet the demand requirements, refine the matching judgment conditions and increase the specific parameter requirements of the capability item.

[0183] The matching standard optimization part analyzes the matching details of the device capability item and the demand item for specific demand items that have previously matched successfully but have not met the actual processing quality standards. For example, the content picture clarity requirement matching standard was originally "maximum resolution ≥ demand resolution", but the actual rendering was reduced due to insufficient device memory, so the refined matching judgment condition is "maximum resolution ≥ demand resolution and memory capacity ≥ resolution corresponding memory requirement". The increased specific parameter requirements are selected from the device capability item related to the demand item (such as memory, processor performance, software version). The redefined matching standard is stored in the demand-capability matching rule file and replaces the original rule.

[0184] Step S155: Based on the adjusted adaptive weight and matching judgment standard, recalculate the adaptive score of each device for different demand dimensions and the comprehensive adaptive score.

[0185] The adaptive score recalculation part uses the adjusted adaptive weight (step S153) and matching judgment standard (step S154) to repeat the calculation process of steps S131 to S136 to obtain the presentation, transmission, and operation dimension adaptive scores of each device and the comprehensive adaptive score. Compare the recalculated results with the original scores to evaluate the adjustment effect (whether the scores of the deviation devices are reduced, and whether the scores of the devices that meet the standards are maintained or improved). If the adjustment effect does not meet the expectation (the scores of the deviation devices are still higher than the threshold), return to step S153 to adjust the weight or step S154 to further refine the matching standard.

[0186] Step S156: Analyze the device processing process record in the interaction result data, extract the resource occupation peak value and processing delay in the device processing process, and if the resource occupation peak value of any one of the main responsible devices in the processing process exceeds the preset upper limit of resource occupation, or the processing delay exceeds the preset upper limit of processing delay, it is judged that the device is not suitable to continue as the main responsible device.

[0187] The device performance evaluation part extracts the resource occupation peak value (the maximum value of CPU / memory / network usage) and the processing delay (the total time from the start to the completion of the action) from the device processing process record. The resource occupation peak value is compared with the preset upper limit of resource occupation (set according to the device hardware configuration and stability requirements), and the processing delay is compared with the preset upper limit of processing delay (set according to the execution efficiency requirements of the demand dimension). If any index exceeds the upper limit, it is determined that the device is insufficient in resources or performance, and is not suitable to continue to assume the responsibility of the main responsible device. The evaluation result is recorded in the device performance evaluation table, including the device identifier, the resource occupation peak value, the resource upper limit, the processing delay, the delay upper limit, and the field of whether it is suitable to continue to assume the responsibility of the main responsible device.

[0188] Step S157: From the candidate collaborative devices, select the device with the second highest demand dimension adaptation score and the processing resource occupation peak value lower than the preset upper limit of resource occupation and the processing delay lower than the preset upper limit of processing delay, and replace the original main responsible device.

[0189] The main responsible device replacement part selects the devices in the candidate collaborative device list according to the demand dimension adaptation score from high to low: first, select the device with the second highest score, check whether the processing resource occupation peak value is lower than the preset upper limit of resource occupation (the upper limit value in step S156), and whether the processing delay is lower than the preset upper limit of processing delay (a specific proportion of the demand processing delay upper limit). If the conditions are met, the device is selected as the new main responsible device; if not, the next device with a higher score is checked. The replacement record includes the original device identifier, the new device identifier, the replacement reason (resource overload / delay exceeding standard), and the replacement time field.

[0190] Step S158: Check whether the standby device in the multi-device collaborative interaction link responds effectively when needed, and if the standby device response delay exceeds the preset response delay upper limit, or the processing quality does not meet the preset processing quality standard, reselect the standby device corresponding to the main responsible device and update the standby device configuration.

[0191] The alternative device effectiveness evaluation part analyzes the response performance of the alternative device in the interaction result data: whether the response delay (the time from starting the alternative mechanism to the alternative device returning the reception confirmation) exceeds the preset upper limit of the response delay (set according to the interaction real-time requirement); whether the processing quality (the quality indicator in step S151) meets the preset processing quality standard (a specific proportion of the demand value). If any indicator does not meet the standard, it is determined that the alternative device is invalid, and the device with a higher adaptation score and meeting the response delay and processing quality requirements is re-screened from the candidate collaborative device list, and the alternative device configuration is updated (step S1311).

[0192] Step S159: According to the re-calculated device adaptation score, the replaced main responsible device, and the updated alternative device, the device nodes, data transmission paths, and interaction operation instruction interaction sequences in the multi-device collaborative interaction link are adjusted.

[0193] The link adjustment part uses the re-calculated device adaptation score (step S155), the replaced main responsible device (step S157), and the updated alternative device (step S158) to repeat the link construction process of steps S139 to S1312, adjust the device nodes (replace the original main responsible device node), the data transmission path (update the path pointing to the new main responsible device), and the interaction operation instruction interaction sequence (if the processing speed of the new device is different, adjust the execution time requirement). The adjusted link is simulated and tested to verify the optimization effect.

[0194] Step S1510: The adjusted adaptation degree calculation standard and the optimized multi-device collaborative interaction link are applied to the next digital multimedia multi-device collaborative interaction process.

[0195] The optimization application part updates the adjusted adaptation weight, the matching determination standard (adaptation degree calculation standard), and the optimized collaborative link configuration file to the system runtime environment. When the next collaborative interaction starts, the new standard and configuration are automatically loaded without manual intervention. The application record includes the optimization version number, the adaptation standard update content, the link configuration update content, and the application time field.

[0196] Step S1511: In the next collaborative interaction process, the steps of requirement dimension decomposition, device capability collection, interaction link construction, interaction operation instruction issuance, and interaction result analysis are repeatedly executed to continuously find the optimization space of the adaptation calculation standard and the link configuration, and to continuously adjust and improve, so as to realize the dynamic optimization of multi-device collaborative interaction.

[0197] The continuous optimization part establishes an iterative optimization mechanism. After each collaborative interaction process ends, the analysis and optimization process of steps S151 to S1510 is performed. Through multiple iterations, the deviation between the actual performance of the device and the adaptation score is gradually reduced, and the stability and efficiency of the collaborative link are improved. All optimization records, interaction result data, adaptation standards and link configuration versions in the iteration process are stored in the optimization history database for trend analysis and problem tracing.

[0198] Figure 2 A multi-device collaborative interaction system 100 for digital multimedia provided in an embodiment of the present application is shown, which includes a processor 1001 and a memory 1003 and program code stored in the memory 1003. The processor 1001 executes the above-mentioned program code to implement the steps of the multi-device collaborative interaction method for digital multimedia. The processor 1001 and the memory 1003 are connected, such as through a bus 1002. Optionally, the multi-device collaborative interaction system 100 for digital multimedia can also include a transceiver 1004, which can be used for data interaction between the multi-device collaborative interaction system for digital multimedia and other multi-device collaborative interaction systems for digital multimedia, such as data transmission and / or data reception, etc. It should be noted that the transceiver 1004 is not limited to one in actual scheduling, and the structure of the multi-device collaborative interaction system 100 for digital multimedia does not constitute a limitation on the embodiments of the present application.

[0199] The memory 1003 is used to store the program code for executing the embodiments of the present application and is controlled by the processor 1001 to execute. The processor 1001 is used to execute the program code stored in the memory 1003 to implement the steps shown in the foregoing method embodiments.

[0200] The above is only an optional implementation of some implementation scenarios of the present application. It should be noted that for those skilled in the art, other similar implementation means according to the technical idea of the present application without departing from the technical concept of the present application also belong to the protection scope of the embodiments of the present application.

Claims

1. A method for multi-device collaborative interaction for digital multimedia, characterized in that, The method comprises: analyzing user interaction requirements of current digital multimedia content to be processed, decomposing the user interaction requirements into presentation requirement dimension, transmission requirement dimension and operation requirement dimension, and generating requirement dimension decomposition results; collecting capability information of multiple devices to be cooperated, forming a device capability list based on the capability information, and the capability information including specific capability items of the devices in terms of presentation, transmission and operation; calculating the adaptation degrees of each device to different requirement dimensions according to the requirement dimension decomposition results and the device capability list, and constructing a digital multimedia multi-device cooperative interaction link based on the adaptation degrees; issuing interaction operation instructions to each device to be cooperated according to the multi-device cooperative interaction link, guiding each device to perform corresponding digital multimedia processing actions, and obtaining interaction result data fed back by the processing actions of the devices; adjusting the adaptation degree calculation standard of each device to different requirement dimensions and the node configuration of the multi-device cooperative interaction link based on the interaction result data, and realizing dynamic optimization of multi-device cooperative interaction; The method comprises: extracting specific requirement items and requirement priorities of the presentation requirement dimension from the requirement dimension decomposition results, and extracting a list of presentation capability items of each device from the device capability list; for each specific requirement item under the presentation requirement dimension, comparing the matching of the presentation capability items of each device with the specific requirement item, and if the presentation capability items of the device meet the requirements of the specific requirement item, it is recorded as a matching success, otherwise it is recorded as a matching failure; setting an adaptation weight for each specific requirement item according to the requirement priority of each specific requirement item under the presentation requirement dimension, and the adaptation weight of the specific requirement item meeting the preset priority standard is set as the preset weight value standard; based on the matching results and adaptation weights of each specific requirement item under the presentation requirement dimension of each device, calculating the adaptation score of each device to the presentation requirement dimension, and the adaptation score calculation method being the sum of the product of each specific requirement item matching result and the corresponding adaptation weight; using the same way, respectively calculating the adaptation score of each device to the transmission requirement dimension and the adaptation score of each device to the operation requirement dimension; normalizing the adaptation scores of each device in the three requirement dimensions respectively, and weighting and summing the normalized three-dimensional adaptation scores according to a preset proportion to obtain a comprehensive adaptation score of each device; selecting devices with a comprehensive adaptation score reaching a preset adaptation score threshold as candidate cooperative devices according to the comprehensive adaptation scores of the devices; for each requirement dimension in the requirement dimension decomposition results, selecting a device with the highest dimension adaptation score from the candidate cooperative devices as the main responsible device of the requirement dimension; Determine the interaction relationship between the main responsible devices of each demand dimension, and present that the main responsible device of the demand dimension needs to provide content presentation parameters to the main responsible device of the transmission demand dimension, the main responsible device of the transmission demand dimension needs to provide content transmission state to the main responsible device of the operation demand dimension, and the main responsible device of the operation demand dimension needs to feed back the operation instruction to the main responsible device of the presentation demand dimension; According to the interaction relationship of each main responsible device, set the data transmission path, interactive operation instruction interaction sequence and state feedback mechanism between the devices; Configure at least one alternative device for each main responsible device, and the selection standard of the alternative device is the candidate collaborative device with the second highest demand dimension adaptation score; Integrate the main responsible device, the alternative device, the interaction relationship between the devices, the data transmission path, the interactive operation instruction interaction sequence and the state feedback mechanism to construct a digital multimedia multi-device collaborative interaction link.

2. The multi-device collaborative interaction method for digital multimedia according to claim 1, wherein, The user interaction demand of the current to-be-processed digital multimedia content is analyzed, and the user interaction demand is decomposed into a presentation demand dimension, a transmission demand dimension and an operation demand dimension to generate a demand dimension decomposition result, including: Obtain the content attribute of the current to-be-processed digital multimedia content, and the content attribute includes content type, content carrier form and content data characteristics; Determine the interaction behavior types that the user may generate based on the content attribute, and the interaction behavior types include viewing behavior, editing behavior and sharing behavior; For each interaction behavior type, collect the expected information of the user's behavior of the interaction behavior type, and the expected information includes the user's appeal for behavior execution effect, execution efficiency and execution mode; Divide the expected information into a presentation demand dimension, a transmission demand dimension and an operation demand dimension according to the core elements affecting the implementation of the interaction; Refine the presentation demand dimension, and extract the specific demand items under the presentation demand dimension, including content picture definition requirement, picture color restoration requirement, picture smoothness requirement and picture size adaptation requirement; Refine the transmission demand dimension, and extract the specific demand items under the transmission demand dimension, including content data transmission speed requirement, data integrity requirement in the transmission process, transmission delay control requirement and multi-device transmission synchronization requirement; Refine the operation demand dimension, and extract the specific demand items under the operation demand dimension, including user operation trigger response speed requirement, operation instruction synchronization requirement among multiple devices, operation permission division requirement and operation result feedback mode requirement; Label the demand priority for each specific demand item under each demand dimension, and the priority division basis is the user's attention to the demand item and the influence of the demand item on the interaction experience; Integrate the presentation demand dimension, the transmission demand dimension, the operation demand dimension, the specific demand items and the demand priority contained therein to form a demand dimension decomposition result.

3. The multi-device collaborative interaction method for digital multimedia of claim 1, wherein, The capability information of the to-be-coordinated multiple devices is collected, and a device capability list is formed based on the capability information, including: Establish a communication connection with the to-be-coordinated multiple devices, and send a capability information collection request to each device, and determine the capability categories to be collected in the request as presentation capability, transmission capability and operation capability; Receiving basic capability data collected by each device based on the capability information collection request, the basic capability data including device hardware configuration information, software function support information and historical collaborative interaction performance data; Extracting specific capability items related to presentation capability from the basic capability data, including maximum picture resolution supported by the device, frame rate range that can be output by the device, device color processing capability range, picture scaling mode supported by the device and device picture display response speed; Extracting specific capability items related to transmission capability from the basic capability data, including maximum data transmission rate of the device, transmission packet loss rate of the device under different network environments, recovery speed of the device after transmission interruption, transmission protocol type supported by the device and ability of the device to simultaneously transmit multiple data streams; Extracting specific capability items related to operation capability from the basic capability data, including response time of the device after receiving an operation instruction, delay of the device in synchronizing the operation instruction to other devices, operation instruction type supported by the device, operation permission setting capability of the device and operation result feedback mode of the device; Verifying the authenticity of the extracted presentation capability items, transmission capability items and operation capability items of each device, the verification method being to compare the consistency of the historical capability data and the current feedback data of the device, and to verify the matching degree of the actual capability and the feedback capability items through test data of a preset data volume; Removing the capability items that fail the verification, and retaining the real and valid presentation capability items, transmission capability items and operation capability items; Creating a dedicated capability record for each device to be cooperated, the dedicated capability record including device identification, a list of presentation capability items, a list of transmission capability items and a list of operation capability items, integrating the dedicated capability records of all devices to form a device capability list.

4. The method for multi-device collaborative interaction for digital multimedia according to claim 1, wherein, According to the multi-device collaborative interaction link, the interactive operation instruction is issued to each device to be cooperated, guiding each device to perform corresponding digital multimedia processing actions, and obtaining interactive result data fed back by the device processing actions, including: Analyzing the main responsible device identification, alternative device identification and corresponding responsibilities of each device in the multi-device collaborative interaction link, determining the type of digital multimedia processing action to be performed by each device, the type of digital multimedia processing action including content decoding action, content rendering action, content transmission action, operation instruction receiving action and operation result feedback action; According to the requirements of each specific requirement item in the demand dimension decomposition result, determining the specific parameters of each device performing corresponding digital multimedia processing action, the specific parameters of the content rendering action including rendering resolution, rendering frame rate and rendering color parameters, the specific parameters of the content transmission action including transmission rate upper limit, transmission data verification method and transmission delay control target; According to the interactive operation instruction interaction sequence in the multi-device collaborative interaction link, generating the interactive operation instruction corresponding to each device, the interactive operation instruction corresponding to each device including device identification, type of digital multimedia processing action, specific parameters of digital multimedia processing action and execution time requirement; According to the data transmission path in the multi-device collaborative interaction link, a direct communication channel between devices is established, and the setting of the direct communication channel needs to meet the transmission protocol type requirement in the device transmission capability item; According to the interaction operation instruction interaction sequence, the interaction operation instructions corresponding to each device are sequentially issued to the target device through the corresponding direct communication channel; During the issuance of the interaction operation instruction, the transmission state of the direct communication channel is monitored in real time, and the interaction operation instruction issuance time, interaction operation instruction reception time, and data packet loss in the transmission process are recorded; If any device does not receive the interaction operation instruction within a specified time, or the received interaction operation instruction has data loss, the standby device mechanism in the multi-device collaborative interaction link is started, the interaction operation instruction is issued to the corresponding standby device, and the data transmission path is updated; After each device receives the interaction operation instruction, the digital multimedia processing action type and digital multimedia processing action specific parameters in the interaction operation instruction are executed, and the processing progress, processing resource occupation, and abnormal information in the processing process are recorded in real time during the execution of the digital multimedia processing action; When the device completes the digital multimedia processing action, a processing result report is generated, which includes the digital multimedia processing action completion time, digital multimedia processing result description, and digital multimedia processing process record; The processing result report is fed back to the control end that initiates the interaction operation instruction through the direct communication channel, the control end that initiates the interaction operation instruction collects the processing result report, interaction operation instruction transmission record, and device processing process record of all devices, integrates the processing result report, interaction operation instruction transmission record, and device processing process record of all devices, and forms an interaction result data.

5. The method for multi-device collaborative interaction for digital multimedia according to claim 1, wherein, Based on the interaction result data, the adjustment of the adaptation degree calculation standard of each device to different demand dimensions and the node configuration of the multi-device collaborative interaction link are realized to dynamically optimize the multi-device collaborative interaction, including: The processing result report in the interaction result data is analyzed to extract the completion quality information of each device processing action, and whether the picture clarity in the processing result of the demand dimension main responsible device meets the demand requirement and whether the picture smoothness meets the standard is presented, whether the transmission speed in the processing result of the transmission demand dimension main responsible device meets the standard and whether the transmission packet loss rate is within the control range; According to the processing completion quality information, it is judged whether the actual adaptation performance of each device to the corresponding demand dimension is consistent with the previously calculated adaptation score, if the actual completion quality does not meet the demand requirement, and the adaptation score value exceeds the preset adaptation score threshold, the adaptation calculation standard of the corresponding demand dimension of the device is marked as having deviation; For the marked adaptation calculation standard deviation, the adaptation weight of each specific demand item under the demand dimension is reevaluated, if it is determined according to the interaction result data that the actual processing quality deviation of any one specific demand item exceeds the preset deviation allowed range, and the adaptation weight value of the specific demand item before is lower than the preset adaptation weight low threshold, the adaptation weight value of the specific demand item is increased to meet the preset weight value standard. Redefine the matching judgment standard of specific demand items and equipment capability items. If the previous judgment is successful but the actual processing quality does not meet the demand requirements, refine the matching judgment condition and increase the specific parameter requirements of the capability item; Based on the adjusted adaptation weight and matching judgment standard, recalculate the adaptation score of each device to different demand dimensions and the comprehensive adaptation score; Analyze the device processing process record in the interaction result data, extract the resource occupation peak and processing delay in the device processing process, if the resource occupation peak of any one main responsible device exceeds the preset resource occupation upper limit or the processing delay exceeds the preset processing delay upper limit, it is judged that the device is not suitable to continue as the main responsible device; Select the device with the second highest demand dimension adaptation score from the candidate collaborative device, and the processing resource occupation peak is lower than the preset resource occupation upper limit, and the processing delay is lower than the preset processing delay lower limit, to replace the original main responsible device; Check whether the standby device in the multi-device collaborative interaction link responds effectively when needed, if the standby device response delay exceeds the preset response delay upper limit, or the processing quality does not meet the preset processing quality standard, reselect the standby device corresponding to the main responsible device and update the standby device configuration; According to the recalculated device adaptation score, the replaced main responsible device and the updated standby device, adjust the device node, data transmission path and interaction operation instruction interaction sequence in the multi-device collaborative interaction link; Apply the adjusted adaptation degree calculation standard and the optimized multi-device collaborative interaction link to the next digital multimedia multi-device collaborative interaction process; In the next collaborative interaction process, repeat the demand dimension decomposition, device capability collection, interaction link construction, interaction operation instruction issuing and interaction result analysis steps, continuously find the optimization space of adaptation calculation standard and link configuration, and constantly adjust and improve, realize the dynamic optimization of multi-device collaborative interaction.

6. The multi-device collaborative interaction method for digital multimedia of claim 2, wherein, The content attribute is determined based on the content attribute, and the interaction behavior type that the user may generate is determined based on the content attribute. Based on the content type and the pre-defined interaction behavior type mapping relationship, the basic interaction behavior type set corresponding to the content type is determined, and the pre-defined interaction behavior type mapping relationship specifies that the video content, audio content, image content and text content are respectively associated with the watching, editing and sharing basic behavior types. Based on the content carrier form, activate or deactivate specific behavior types in the basic interaction behavior type set. If the content carrier form is a mobile terminal storage, activate the offline editing behavior type; if the content carrier form is a cloud storage, activate the online sharing behavior type. Based on the content data characteristics, adjust the technical implementation requirements of specific behavior types in the basic interaction behavior type set. If the content data volume exceeds the preset data volume upper limit threshold, set the transmission efficiency requirement for the sharing behavior type; if the content data volume is lower than the preset data volume lower limit threshold, set the operation response speed requirement for the editing behavior type. The integrated set of basic interaction behavior types activated or deactivated by the content carrier and adjusted by the content data features forms a set of interaction behavior types corresponding to the current digital multimedia content to be processed.

7. The multi-device collaborative interaction method for digital multimedia of claim 3, wherein, The authenticity of the extracted presentation capability item, transmission capability item and operation capability item of each device is verified by comparing the historical capability data of the device with the current feedback data, and by verifying the matching degree of the actual capability of the device with the feedback capability item through the preset amount of test data, including: Extracting the past presentation capability data, transmission capability data and operation capability data of the device from the historical collaborative interaction record of the device to form a historical capability data set of the device; Comparing the presentation capability item, transmission capability item and operation capability item currently extracted from the basic capability data with the corresponding data in the historical capability data set of the device item by item; Calculate the deviation value of the current capability item and the historical capability data. If the deviation value is within the preset deviation allowable range, it is determined that the current capability item is verified on the historical data comparison level. If the deviation value exceeds the preset deviation allowable range, the current capability item is marked as a further verification item; For all capability items, generate a preset amount of test data. The preset amount of test data needs to cover the core parameter requirements of each capability item. For the maximum picture resolution capability item in the presentation capability item, it contains test images of different resolution specifications. For the maximum data transmission rate capability item in the transmission capability item, it contains test files of different sizes. For the operation response time capability item in the operation capability item, it contains different types of test operation instructions; Send the generated preset amount of test data to the corresponding collaborative device, and require the corresponding collaborative device to perform the corresponding action according to the requirements of the preset amount of test data. The corresponding collaborative device needs to perform the corresponding action, including displaying test images, transmitting test files, and responding to test operation instructions. Record the actual performance data of the corresponding collaborative device performing the corresponding action. The actual performance data of the corresponding collaborative device includes the actual resolution when displaying test images, the actual transmission rate when transmitting test files, and the actual response time when responding to test operation instructions. Compare the actual performance data of the corresponding collaborative device with the corresponding capability item feedback by the current device. If the actual performance data meets the requirements of the capability item description, it is determined that the current capability item is verified on the actual test level. If the actual performance data does not meet the requirements of the capability item description, it is determined that the current capability item fails to verify. For the further verification items marked in the historical data comparison, if they are verified on the actual test level, the current capability item is retained. If they also fail to verify on the actual test level, the current capability item is excluded. Summarize the verification results of all capability items to form a device capability item verification report. The device capability item verification report includes a list of verified capability items and a list of failed capability items.

8. The multi-device collaborative interaction method for digital multimedia of claim 1, wherein, The data transmission path, interaction operation instruction interaction sequence and state feedback mechanism between devices are set according to the interaction relationship of each main responsible device, including: The content presentation parameter type provided by the presentation demand dimension master device to the transmission demand dimension master device is determined, the transmission format and transmission frequency of the content presentation parameter are determined, and the content presentation parameter type includes target presentation resolution, target presentation frame rate, color space standard, and picture scaling ratio; Based on the transmission format and transmission frequency of the content presentation parameter, the transmission protocol type in the transmission capability item of the presentation demand dimension master device and the transmission demand dimension master device is combined, a suitable transmission channel is selected, and a one-way data transmission path from the presentation demand dimension master device to the transmission demand dimension master device is set; The content transmission state type provided by the transmission demand dimension master device to the operation demand dimension master device is determined, and the feedback period and feedback format of the content transmission state information are determined, and the content transmission state type includes current transmission rate, transmitted data volume, transmission packet loss, and transmission delay; According to the feedback period and feedback format of the content transmission state information, the transmission delay control capability in the transmission capability item of the transmission demand dimension master device and the operation demand dimension master device is combined, a one-way data transmission path from the transmission demand dimension master device to the operation demand dimension master device is set, so that the content transmission state information reaches within the feedback period; The operation instruction type fed back by the operation demand dimension master device to the presentation demand dimension master device is determined, the operation instruction type includes picture switching instruction, volume adjustment instruction, playing progress adjustment instruction, and picture effect adjustment instruction, and the encoding format and priority of the operation instruction are determined; According to the encoding format and priority of the operation instruction, the operation response speed in the operation capability item of the operation demand dimension master device and the presentation demand dimension master device is combined, a one-way interactive operation instruction transmission path from the operation demand dimension master device to the presentation demand dimension master device is set, and the operation instruction with priority meeting the preset priority standard adopts a priority transmission channel; The interactive operation instruction interaction sequence is determined, the operation demand dimension master device needs to receive a user operation instruction first, and then feed back the user operation instruction to the presentation demand dimension master device, at the same time, the transmission demand dimension master device needs to feed back the content transmission state to the operation demand dimension master device in real time, and the presentation demand dimension master device needs to send new content presentation parameters to the transmission demand dimension master device according to the user operation instruction to adjust the content presentation parameters; An abnormal feedback process in the state feedback mechanism is set, if the transmission demand dimension master device detects that the transmission packet loss rate exceeds the preset packet loss rate allowed range, the transmission exception notification needs to be sent to the operation demand dimension master device immediately, after the operation demand dimension master device receives the transmission exception notification, the new interactive operation instruction is paused to be issued to the presentation demand dimension master device, and the transmission exception information is fed back to the user; A normal feedback process in the state feedback mechanism is set, and each master device needs to send state normal confirmation information to the associated device according to the preset feedback period. The set data transmission path, interactive operation instruction interactive sequence and state feedback mechanism are arranged into a link interactive specification, and the link interactive specification is a component of the multi-device cooperative interactive link, so that each main responsible device performs interactive operation according to the link interactive specification.

9. A multi-device collaborative interaction system for digital multimedia, characterized by, The method comprises a processor and a computer readable storage medium, the computer readable storage medium stores machine executable instructions, and the machine executable instructions are executed by the processor to realize the multi-device cooperative interactive method for digital multimedia in any one of claims 1-8.

Images