Multimedia fusion system and method based on task cooperation technology

Abstract

The invention relates to the technical field of multimedia fusion, in particular to a multimedia fusion system and a multimedia fusion method based on a task cooperation technology. A fusion adjustment module in the system simulates a fusion picture of a current cluster task summary set based on the position of main body reference object binding equipment; and performing adaptive adjustment on elements in the current cluster task summary set based on the main body reference object, and generating and outputting a fusion picture set. According to the method, the fluctuation state information of the receiving response duration of the multi-machine cluster to the display task, the main body reference object of each display task in the historical cluster task summary set and the picture corresponding to the rendered display task are selected; according to the method, the stability of each device in the multi-machine cluster and the collaborative adaptation difference condition among the display tasks are quantified, so that the main body reference object of the current cluster task summary set is dynamically screened, and a reference basis is provided for adaptive adjustment of each display task in the subsequent cluster task summary set.

Description

Technical Field

[0001] The present invention relates to the technical field of multimedia fusion, and specifically provides a multimedia fusion system and method based on task collaboration technology. Background Art

[0002] In the field of multimedia display, with the progress of display technology, a single display device often cannot meet the requirements of large-size and high-resolution displays. Therefore, the multi-screen splicing and fusion technology has emerged. By combining multiple display devices, a larger display screen is formed. However, traditional splicing technologies have problems such as discontinuous pictures and inconsistent colors, which affect the user experience; and the existing splicing of multi-screen display pictures often targets the splicing of multi-screen pictures on the same plane. For the splicing of multi-screen pictures on different planes, when realizing picture splicing, usually only a perfect spliced picture can be seen at a specific position. When the position deviates, the visual perception of the spliced picture will be significantly reduced, and the visual angle of the picture content at the splicing joint of the spliced picture will have a large deviation; furthermore, when performing picture splicing, it is often necessary to adaptively adjust the remaining pictures to be spliced on the basis of keeping part of the pictures unchanged, and the data processing volume is large. However, the existing technology lacks a means to dynamically select a reference picture (the unchanged display picture) when processing the picture adjustment. Summary of the Invention

[0003] The purpose of the present invention is to provide a multimedia fusion system and method based on task collaboration technology to solve the problems raised in the above background art.

[0004] To solve the above technical problems, the present invention provides the following technical solution: A multimedia fusion method based on task collaboration technology, the method includes the following steps: S1. Obtain a multi-machine cluster corresponding to the display end, and receive in real time the display tasks matched by each device in the multi-machine cluster, and summarize the display tasks respectively corresponding to each device in the multi-machine cluster at the same time to generate a cluster task summary set corresponding to the corresponding task time; S2. Based on the fluctuation state information of the receiving response duration of the multi-machine cluster for the display tasks in history, evaluate the stability of each device to generate a stability reference object deviation value of the multi-machine cluster; S3. Obtain the main reference object of each display task in the cluster task summary set to be sent most recently based on the current time, and the corresponding picture of the rendered display task, and analyze the collaborative adaptation difference value of each device's display task respectively based on the previous time; combined with the stability reference object deviation value of the multi-machine cluster, select the main reference object of the current cluster task summary set; S4. Based on the location to which the device belonging to the subject reference object is bound, simulate the fusion screen of the current cluster task summary set, and perform adaptive adjustment on the elements in the current cluster task summary set based on the subject reference object, and generate and output a set of fusion screens.

[0005] In the process of generating the set of fusion screens in the present invention, considerations are taken from the stability of the receiving device, the subject reference objects of each display task in the historical cluster task summary set, and the corresponding screens of the rendered display tasks, so as to realize the dynamic screening of the subject reference objects in the cluster task summary set corresponding to the multi-machine cluster at the current task time. And according to the dynamic screening result of the subject reference object, and based on the simulation result of the panoramic screen stitching software, perform adaptive adjustment on the elements in the current cluster task summary set, and generate the corresponding set of fusion screens (the rendering result of the display task screen corresponding to the corresponding task time).

[0006] Further, in S1, the display end is composed of a multi-machine cluster, and each device in the multi-machine cluster is a display screen; each display screen executes one display task each time; the display tasks executed by the same display screen at different times are different; the screen stitching result of the display task corresponding to each device in the multi-machine cluster at each task time is the display screen to be displayed at the display end corresponding to the corresponding task time.

[0007] Further, S2 includes: The fluctuation state information of the receiving response duration of the multi-machine cluster for the display task in the history includes the summary set of the fluctuation state information of the receiving response duration of each device in the multi-machine cluster for the corresponding display task; the fluctuation state information of the receiving response duration of each device in the multi-machine cluster for the corresponding display task includes the maximum value, minimum value, and average value of the interval duration from the start of receiving to the start of displaying the corresponding task screen for each display task by the corresponding device in the historical data. The specific calculation formula involved in evaluating the stability of each device is as follows: , Among them, W n represents the stability evaluation value of the n th device in the multi-machine cluster; TG （n，max） represents the maximum value of the corresponding interval duration in the fluctuation state information of the receiving response duration of the n th device in the multi-machine cluster for the corresponding display task; TG （n，min） represents the minimum value of the corresponding interval duration in the fluctuation state information of the receiving response duration of the n th device in the multi-machine cluster for the corresponding display task; TG （n，avg） represents the n th device in the multi-machine clustern The average value of the corresponding interval duration in the fluctuation state information of the reception response duration of a device for the corresponding display task; ℇ Represents a preset conversion factor; dec{} Represents a screening judgment function. When TG （n，max） -TG （n，min） =0 then it is determined that dec{TG （n，max） -TG （n，min） }=1 ; otherwise, it is determined that dec{TG （n，max） - TG （n，min） }=TG （n，max） -TG （n，min） ; The calculation formula for generating the stability reference object deviation value of the multi - machine cluster is as follows: , Wherein, RB n Represents the stability reference object deviation value of the multi - machine cluster based on the n th device; N Represents the total number of devices in the multi - machine cluster.

[0008] In the process of generating the stability reference object deviation value of the multi - machine cluster in the present invention, by combining the fluctuation state information of the reception response duration of each device in the multi - machine cluster for the corresponding display task in the historical data, a quantitative evaluation of the stability of the reception state of the display task screen by each device in the multi - machine cluster is realized, and data support is provided for analyzing the main reference object of the cluster task summary set in the subsequent steps.

[0009] Furthermore, in the process of analyzing the display tasks of current individual devices respectively based on the previous collaborative adaptation difference value in S3, the calculation formula is as follows: , Wherein, F n Represents the collaborative adaptation difference value of the display task of the current n th device based on the corresponding screen of the display task after the previous rendering; BC n Represents the background color deviation coefficient between the display task of the current n th device and the corresponding screen of the same device's display task after the previous rendering; The BC n value is equal to the currentn The quotient obtained by dividing the absolute value of the difference between the average gray value of each pixel point in the screen corresponding to the display task of a device and the average gray value of each pixel point in the screen corresponding to the display task of the same device after the previous rendering by 256; M n Indicates the number of display task screens connected to the screen corresponding to the display task of the n device after the previous rendering; Texe （n，a） Indicates that in the display task screen after the previous rendering, the n th display task screen connected to the screen corresponding to the display task of the a device and the n image texture matching coefficient at the connection position with the screen corresponding to the display task of the current μ Indicates the preset collaborative adaptation conversion coefficient; , M （n，a） Indicates the number of pixel points within the screen corresponding to the display task of the n th display task screen connected to the screen corresponding to the display task of the a device and the n screen corresponding to the display task of the current n device at the connection position; GV （n，a，b） Indicates the gray value of the n th pixel point within the screen corresponding to the display task of the a th display task screen connected to the screen corresponding to the display task of the n device and the n screen corresponding to the display task of the current b device at the connection position; M （n，a，b） Indicates the number of pixel points adjacent to the n th pixel point within the screen corresponding to the display task of the a th display task screen connected to the screen corresponding to the display task of the n device and the n screen corresponding to the display task of the current b device at the connection position and belonging to the n th display task screen connected to the screen corresponding to the display task of the a device in the display task screen after the previous rendering; GV （n，a，b，d） IndicatesM （n，a，b） For the gray value of the d th pixel among each pixel point; H{} Indicates a preset function. If d∈[1, M （n，a，b） ] When, the corresponding each GV （n，a，b，d） There is a situation where the absolute value of the difference from GV （n，a，b） is less than or equal to the preset value, then it is determined that ; otherwise, it is determined that .

[0010] Further, in the process of the S3 selecting the subject reference object of the current cluster task summary set, obtain the subject reference objects of each display task in the cluster task summary set to be sent most recently at the current time, the stability reference object deviation value of the multi-machine cluster based on the n th device RB n ; and the collaborative adaptation difference value of the display task of the current n th device based on the corresponding screen after the previous rendering F n ; Calculate the comprehensive reference object deviation value of the n th device in the multi-machine cluster according to the evaluation formula. The evaluation formula is as follows: , Among them, P n Represents the comprehensive reference object deviation value of the n th device in the multi-machine cluster; L n Represents the distance between the center point of the device position where the subject reference object of each display task in the cluster task summary set to be sent most recently at the current time belongs and the center point of the n th device position; L Y Represents the average value of the lengths corresponding to each device in the multi-machine cluster; β1 And β2 Are both preset constants; Select the element with the largest comprehensive reference object deviation value of the corresponding device in the current cluster task summary set as the subject reference object of the current cluster task summary set.

[0011] Further, the S4 includes: S41. Obtain the positions among devices in the multi-machine cluster, the display tasks corresponding to each device in the cluster task summary set at the current task time, and the main reference object of the current cluster task summary set; S42. Based on the positions among devices in the multi-machine cluster, construct a spatial model of the device positions within the multi-machine cluster in the panoramic image stitching software, denoted as the spatial display model; S43. Map the display task images corresponding to each element in the current cluster task summary set in the constructed spatial display model, and mark the main reference object of the current cluster task summary set; S44. Based on the preset viewing perspectives in the spatial display model, starting from the main reference object of the current cluster task summary set, adjust each display task image around the marked display task image one by one to obtain the adjusted display task images; S45. Taking the adjusted display task images as references, adjust each image around the adjusted display task images one by one; and so on, until all elements in the current cluster task summary set except the main reference object are adjusted, and transfer the current cluster task summary set after the adjustment process to step S46; S46. Receive the transmission result of step S45, and use the summary set of the display task images in the received transmission result as the corresponding fusion image set; During the process of adjusting each display task image around the marked display task image in step S44, respectively match each preset image adjustment scheme in the database for the display task images to be adjusted, and use the adjustment result of the display task image with the smallest image texture matching coefficient at the connection position with the marked display task image after respectively executing the preset image adjustment schemes as the adjusted display task image.

[0012] A multimedia fusion system based on task collaboration technology, the system includes the following modules: Cluster task summary module, the cluster task summary module obtains the multi-machine cluster corresponding to the display end, and receives in real time the display tasks matched by each device in the multi-machine cluster, and summarizes the display tasks respectively corresponding to each device in the multi-machine cluster at the same time to generate a cluster task summary set for the corresponding task time; Device stability analysis module, the device stability analysis module evaluates the stability of each device based on the fluctuation state information of the reception response duration of the multi-machine cluster for display tasks in history, and generates a stability reference object deviation value for the multi-machine cluster; A reference analysis acquisition module, wherein the reference analysis acquisition module acquires the main reference object of each display task in the cluster task summary set to be sent most recently based on the current time, as well as the corresponding screen of the rendered display task, and analyzes the display tasks of each current device based on the previous collaborative adaptation difference value; combines the stability reference object bias value of the multi-machine cluster, and selects the main reference object of the current cluster task summary set; A fusion adjustment module simulates the fusion picture of the current cluster task summary set based on the location of the subject reference object binding device, and adaptively adjusts the elements in the current cluster task summary set based on the subject reference object to generate a fusion picture set output.

[0013] Furthermore, the reference analysis acquisition module includes a task adaptation difference analysis unit and a subject reference object screening unit. The task adaptation difference analysis unit obtains the main reference object of each display task in the cluster task summary set to be sent most recently based on the current time, and the corresponding screen of the rendered display task, and analyzes the collaborative adaptation difference value of the display tasks of each current device based on the previous one; The subject reference object screening unit selects the subject reference object of the current cluster task summary set in combination with the stability reference object bias value of the multi-machine cluster.

[0014] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: in the process of multimedia display, the present invention takes into account the incoordination of the screen caused by the influence of the viewing angle and the placement of the device when each display device in the multi-machine cluster executes the corresponding display task screen, and quantifies the stability of each device in the multi-machine cluster and the collaborative adaptation differences between the display tasks from the fluctuation status information of the reception response time of the multi-machine cluster to the display task, the main reference objects of each display task in the historical cluster task summary set, and the corresponding screen of the rendered display task, thereby realizing dynamic screening of the main reference objects of the current cluster task summary set, and providing a reference basis for the adaptive adjustment of each display task in the subsequent cluster task summary set; on the basis of realizing the coordination of multimedia display tasks, it not only ensures the fusion screen effect between the corresponding display tasks in the multi-machine cluster, but also effectively reduces the work of adaptive adjustment of the elements in the cluster task summary set. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 It is a structural schematic diagram of a multimedia fusion system based on task collaboration technology of the present invention; Figure 2 It is a schematic flowchart of a multimedia fusion method based on task collaboration technology of the present invention. Detailed implementation manners

[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0017] Please refer to Figure 1 , the present invention provides a technical solution: a multimedia fusion system based on task collaboration technology, and the system includes the following modules: A cluster task summary module, which obtains a multi-machine cluster corresponding to a display end, and receives in real time the display tasks matched by each device in the multi-machine cluster, and summarizes the display tasks respectively corresponding to each device in the multi-machine cluster at the same time to generate a cluster task summary set corresponding to the corresponding task time; A device stability analysis module, which evaluates the stability of each device based on the fluctuation state information of the receiving response duration of the multi-machine cluster for display tasks in history, and generates a stability reference object deviation value of the multi-machine cluster; A reference analysis and acquisition module, which includes a task adaptation difference analysis unit and a main reference object screening unit, The task adaptation difference analysis unit obtains the main reference object of each display task in the cluster task summary set to be sent most recently based on the current time, as well as the corresponding picture of the rendered display task, and analyzes the collaborative adaptation difference value of each device's display task based on the previous time; The main reference object screening unit selects the main reference object of the current cluster task summary set in combination with the stability reference object deviation value of the multi-machine cluster; A fusion adjustment module, which simulates the fusion picture of the current cluster task summary set based on the position where the device to which the main reference object is bound, and adaptively adjusts the elements in the current cluster task summary set based on the main reference object to generate a fusion picture set for output.

[0018] As Figure 2 shown, a multimedia fusion method based on task collaboration technology, and the method includes the following steps: S1. Obtain the multi-machine cluster corresponding to the display end, and receive in real time the display tasks matched by each device in the multi-machine cluster, and summarize the display tasks corresponding to each device in the multi-machine cluster at the same time to generate a cluster task summary set for the corresponding task time; In S1, the display end is composed of a multi-machine cluster, and each device in the multi-machine cluster is a display screen; each display screen executes one display task each time; the display tasks executed by the same display screen at different times are different; the picture splicing result of the display task corresponding to each device in the multi-machine cluster at each task time is the picture to be displayed at the display end for the corresponding task time.

[0019] S2. Based on the fluctuation state information of the reception response duration of the multi-machine cluster for the display task in history, evaluate the stability of each device, and generate a stability reference object deviation value for the multi-machine cluster; S2 includes: The fluctuation state information of the reception response duration of the multi-machine cluster for the display task in history includes a summary set of the fluctuation state information of the reception response duration of each device in the multi-machine cluster for the corresponding display task; the fluctuation state information of the reception response duration of each device in the multi-machine cluster for the corresponding display task includes the maximum value, minimum value, and average value of the interval duration from the start of receiving to the start of displaying the corresponding task picture for the corresponding device in the historical data for each display task. The specific calculation formula involved in evaluating the stability of each device is as follows: , Among them, W n represents the stability evaluation value of the n th device in the multi-machine cluster; TG （n，max） represents the maximum value of the corresponding interval duration in the fluctuation state information of the reception response duration of the n th device in the multi-machine cluster for the corresponding display task; TG （n，min） represents the minimum value of the corresponding interval duration in the fluctuation state information of the reception response duration of the n th device in the multi-machine cluster for the corresponding display task; TG （n，avg） represents the average value of the corresponding interval duration in the fluctuation state information of the reception response duration of the n th device in the multi-machine cluster for the corresponding display task; ℇ represents a preset conversion factor; dec{} represents a screening judgment function. When TG （n，max） -TG （n，min） =0 then it is determined thatdec{TG （n，max） -TG （n，min） }=1 ; Conversely, it is determined that dec{TG （n，max） - TG （n，min） }=TG （n，max） -TG （n，min） ; In this embodiment, when evaluating the stability of the device, on the one hand, it is considered from the fluctuation situation of the interval duration corresponding to each display task from the start of reception to the start of displaying the corresponding task screen (that is, the length of the interval duration fluctuation range between the maximum interval duration and the minimum interval duration, and the time difference between the average interval duration and the minimum interval duration); on the other hand, the overall picture response time situation is reflected by the average interval duration. The larger the average interval duration, the faster the picture response speed of the corresponding device can be reflected. The average interval duration reflects the overall picture response level of the device.

[0020] In this embodiment, if in the fluctuation state information of the reception response duration of device A in the multi-machine cluster for the corresponding display task, the maximum and minimum values of the interval duration from the start of reception to the start of displaying the corresponding task screen for each display task by the corresponding device in the historical data are equal, the corresponding maximum and minimum values of the interval duration are denoted as TG 甲 , then the average value of the interval duration from the start of reception to the start of displaying the corresponding task screen for each display task by the corresponding device in the historical data is also TG 甲 , Furthermore, the stability evaluation value of device A in the multi-machine cluster is ; Because when TG （n，max） -TG （n，min） =0, dec{TG （n，max） -TG （n，min）}=1, Therefore, the stability evaluation value of device A in the multi-machine cluster is ; The calculation formula for generating the stability reference object deviation value of the multi-machine cluster is as follows: , wherein, RB n represents the stability reference object deviation value of the multi-machine cluster based on the n th device; N represents the total number of devices in the multi-machine cluster.

[0021] S3. Obtain the main reference objects of each display task in the most recent cluster task summary set to be sent based on the current time, as well as the corresponding rendered display task images, and analyze the collaborative adaptation difference values of the current display tasks of each device respectively based on the previous ones; in combination with the stability reference object deviation value of the multi-machine cluster, select the main reference object of the current cluster task summary set. In the process of analyzing the collaborative adaptation difference values of the current display tasks of each device respectively based on the previous ones in S3, the calculation formula is as follows: , where, F n represents the collaborative adaptation difference value of the display task of the current n th device based on the corresponding image of the previous rendered display task; BC n represents the background color deviation coefficient between the display task of the current n th device and the corresponding image of the display task of the same device after the previous rendering; the BC n value is equal to the quotient obtained by dividing the absolute value of the difference between the average gray value of each pixel point in the corresponding image of the display task of the current n th device and the average gray value of each pixel point in the corresponding image of the display task of the same device after the previous rendering by 256; M n represents the number of display task images in the display task image after the previous rendering that are connected to the corresponding image of the display task of the n th device; Texe （n，a） represents the image texture matching coefficient at the connection position between the n th display task image in the display task image after the previous rendering that is connected to the corresponding image of the display task of the a th device and the corresponding image of the display task of the current n th device; μ represents the preset collaborative adaptation conversion coefficient; , M （n，a） represents the number of pixel points in the corresponding image of the display task of the current n th device at the connection position between the a th display task image in the display task image after the previous rendering that is connected to the corresponding image of the display task of the n th device and the corresponding image of the display task of the current n th device; GV （n，a，b） represents in the display task image after the previous rendering, then The gray value of the a th display task screen corresponding to the display task of a device that is connected to the n th display task screen of the device and the n th pixel point within the display task corresponding screen of the device; b The gray value of the M （n，a，b） Indicates that within the display task screen after the previous rendering, the n th display task screen corresponding to the display task of a device that is connected to the a th display task screen and the n th display task screen of the device, at the connection position between the two; among the n th pixel points within the display task corresponding screen of the device, the b th pixel point that is adjacent to the n th pixel point within the display task corresponding screen of the device after the previous rendering and belongs to the a th display task screen that is connected to the GV （n，a，b，d） Indicates M （n，a，b） The gray value of the d th pixel point corresponding to each pixel point; H{} Indicates a preset function. If d∈[1, M （n，a，b） ] at this time, for each GV （n，a，b，d） there is a situation where the absolute value of the difference between GV （n，a，b） is less than or equal to a preset value, then it is determined that ; otherwise, it is determined that .

[0022] During the process of the S3 step of selecting the theme reference object of the current cluster task summary set, obtain the main reference objects of each display task within the cluster task summary set that is to be sent most recently at the current time, the stability reference object deviation value of the multi-machine cluster based on the n th device RB n ; and the collaborative adaptation difference value of the display task of the n th device based on the display task corresponding screen after the previous rendering F n ; Calculate the comprehensive deviation value of the reference object corresponding to the n th device in the multi-machine cluster according to the evaluation formula. The evaluation formula is as follows: , where,P n represents the comprehensive deviation value of the reference object corresponding to the n th device in the multi-machine cluster; L n represents the distance between the center point of the device location to which the main reference object of each display task in the summary set of cluster tasks to be sent most recently at the current time belongs and the center point of the n th device location; L Y represents the average value of the lengths respectively corresponding to each device in the multi-machine cluster; β1 and β2 are both preset constants; Select the element with the largest comprehensive deviation value of the reference object corresponding to the device in the current cluster task summary set as the main reference object of the current cluster task summary set.

[0023] S4. Based on the position to which the main reference object is bound to the device, simulate the fusion screen of the current cluster task summary set, and perform adaptive adjustment on the elements in the current cluster task summary set based on the main reference object, and generate and output the fusion screen set; The said S4 includes: S41. Obtain the positions between each device in the multi-machine cluster, the display tasks corresponding to each device in the summary set of cluster tasks at the current task time, and the main reference object of the current cluster task summary set; S42. Based on the positions between each device in the multi-machine cluster, construct a spatial model of the device positions in the multi-machine cluster in the panoramic screen stitching software, denoted as the spatial display model; in this embodiment, the panoramic screen stitching software is PTGui; S43. Map the display task screen corresponding to each element in the current cluster task summary set in the constructed spatial display model, and mark the main reference object of the current cluster task summary set; S44. Based on the preset viewing angle in the spatial display model, starting from the main reference object of the current cluster task summary set, adjust each display task screen around the marked display task screen one by one to obtain the adjusted display task screen; S45. Taking the adjusted display task screen as a reference, adjust each screen around the adjusted display task screen one by one; and so on, until all elements in the current cluster task summary set except the main reference object are adjusted, and transmit the current cluster task summary set after the adjustment process to step S46; S46. Receive the transmission result of step S45, and use the summary set of each display task screen in the received transmission result as the corresponding fusion screen set; In the process of adjusting each display task screen around the marked display task screen in step S44, each preset screen adjustment scheme in the database is respectively matched to the display task screen to be adjusted, and the display task screen adjustment result with the smallest image texture matching coefficient at the connection position with the marked display task screen after respectively executing the preset screen adjustment scheme is used as the adjusted display task screen.

[0024] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0025] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multimedia fusion method based on task collaboration technology, characterized in that: The method comprises the following steps: S1. Obtain the multi-machine cluster corresponding to the display terminal, and receive the display tasks matched by each device in the multi-machine cluster in real time, and summarize the display tasks corresponding to each device in the multi-machine cluster at the same time to generate a cluster task summary set of the corresponding task time; S2. Based on the historical fluctuation status information of the reception response time of the multi-machine cluster for the display task, the stability of each device is evaluated to generate a stability reference object bias value of the multi-machine cluster; S3, obtain the main reference object of each display task in the cluster task summary set to be sent most recently based on the current time, as well as the corresponding screen of the rendered display task, analyze the display tasks of each current device based on the previous collaborative adaptation difference value; combine the stability reference object bias value of the multi-machine cluster, and select the main reference object of the current cluster task summary set; S4. Based on the location of the subject reference object binding device, simulate the fusion picture of the current cluster task summary set, and adaptively adjust the elements in the current cluster task summary set based on the subject reference object to generate a fusion picture set output.

2. The multimedia fusion method based on task collaboration technology according to claim 1, characterized in that: The display end in S1 is composed of a multi-machine cluster, each device in the multi-machine cluster is a display screen; each display screen executes one display task at a time; the same display screen executes different display tasks at different times; at each task time, the image splicing result of the corresponding display task of each device in the multi-machine cluster is the image to be displayed on the display end at the corresponding task time.

3. The multimedia fusion method based on task collaboration technology according to claim 1 is characterized in that: The S2 includes: The fluctuation status information of the reception response time of the multi-machine cluster to the display task in the history includes a summary set of the fluctuation status information of the reception response time of each device in the multi-machine cluster to the corresponding display task; the fluctuation status information of the reception response time of each device in the multi-machine cluster to the corresponding display task includes the maximum value, minimum value and average value of the interval time between the start of reception and the start of display of the corresponding task screen for each display task of the corresponding device in the historical data; The calculation formulas involved in evaluating the stability of each device are as follows: ， in, W n Indicates the number of n Stability assessment value of each device; TG （n，max） Indicates the number of n The maximum value of the corresponding interval duration in the fluctuation status information of the reception response duration of each device to the corresponding display task; TG （n，min） Indicates the number of n The minimum value of the corresponding interval duration in the fluctuation status information of the reception response duration of the corresponding display task by each device; TG （n，avg） Indicates the number of n The average value of the corresponding interval duration in the fluctuation status information of the reception response duration of each device to the corresponding display task; ℇ Indicates the preset conversion factor; dec{} Represents the screening judgment function, when TG （n，max） -TG （n，min） =0 When dec{TG （n，max） -TG （n，min） }=1 Otherwise, it is judged dec{TG （n，max） - TG （n，min） }=TG （n，max） -TG （n，min） ; The calculation formula for generating the stability reference object bias value of a multi-machine cluster is as follows: ， in, RB n Indicates that the multi-machine cluster is based on n The stability of each device is referenced by the bias value of the object; N Indicates the total number of devices in a multi-machine cluster.

4. The multimedia fusion method based on task collaboration technology according to claim 1 is characterized in that: In the process of analyzing the display tasks of each device in the current state in S3 based on the previous collaborative adaptation difference value, the calculation formula is as follows: ， in, F n Indicates the current n The display task of each device is based on the collaborative adaptation difference value of the corresponding screen of the display task after the previous rendering; BC n Indicates the current n The background color deviation coefficient between the display task of a device and the corresponding picture of the display task of the same device after the previous rendering; BC n The value is equal to the current n The quotient of the absolute value of the difference between the average grayscale value of each pixel in the picture corresponding to the display task of the device and the average grayscale value of each pixel in the picture corresponding to the display task of the same device after the previous rendering divided by 256; M n Indicates that the previous rendering shows the same n The number of display task screens corresponding to the display task of each device connected; Texe （n，a） Indicates that the previous rendering task screen is displayed, and the n The display task of each device corresponds to the first a The task screen is displayed with the current n The image texture matching coefficient of the connection position of the screen corresponding to the display task of each device; μ represents the preset cooperative adaptation conversion coefficient; ， M （n，a） Indicates that the previous rendering shows the same content as the first n The display task of each device corresponds to the first a The displayed task screen is the same as the current n In the connection position of the corresponding screen of the display task of each device, the current n The display task of each device corresponds to the number of pixels in the screen; GV （n，a，b） Indicates that the previous rendering shows the same content as the first n The display task of each device corresponds to the first a The displayed task screen is the same as the current n In the connection position of the corresponding screen of the display task of each device, the current n The display task of each device corresponds to the b The gray value of each pixel; M （n，a，b） Indicates that the previous rendering shows the same content as the first n The display task of each device corresponds to the first a The displayed task screen is the same as the current n The display task of each device corresponds to the connection position of the screen, which is consistent with the current n The display task of each device corresponds to the b The pixels are adjacent and belong to the same task screen after the previous rendering. n The display task of each device corresponds to the first a The number of pixels that display the task screen; GV （n，a，b，d） express M （n，a，b） Corresponding to each pixel d The gray value of each pixel; H{} Represents a preset function. If d∈[1,M （n，a，b） ] The corresponding GV （n，a，b，d） Existence and GV （n，a，b） If the absolute value of the difference between the two is less than or equal to the preset value, it is determined Otherwise, it is judged .

5. The multimedia fusion method based on task collaboration technology according to claim 1 is characterized in that: In the process of selecting the subject reference object of the current cluster task summary set in S3, the subject reference object of each display task in the cluster task summary set to be sent most recently at the current time is obtained, and the multi-machine cluster is based on the first n The stability of each device is referenced by the bias value of the object RB n ; and the current n The display task of each device is based on the collaborative adaptation difference value of the corresponding screen of the display task after the previous rendering F n ; According to the evaluation formula, the n The comprehensive bias value of the reference object corresponding to each device, the evaluation formula is as follows: ， in, P n Indicates the number of n The comprehensive bias value of the reference object corresponding to each device; L n Indicates the center point of the device location to which the main reference object of each display task in the most recent cluster task summary set to be sent at the current time belongs and the n The distance between the center points of the device locations; L Y Indicates the average length of each device in a multi-machine cluster; β1 and β2 All are preset constants; The element with the largest comprehensive bias value of the reference object corresponding to the device in the current cluster task summary set is selected as the main reference object of the current cluster task summary set.

6. The multimedia fusion method based on task collaboration technology according to claim 5 is characterized by: The S4 includes: S41, obtaining the positions between the devices in the multi-machine cluster, the display tasks corresponding to each device in the cluster task summary set at the current task time, and the subject reference object of the current cluster task summary set; S42, based on the positions of the devices in the multi-machine cluster, construct a spatial model of the device positions in the multi-machine cluster in the panoramic image stitching software, recorded as a spatial display model; S43, mapping the display task screen corresponding to each element in the current cluster task summary set in the constructed spatial display model, and marking the main reference object of the current cluster task summary set; S44, based on the preset observation perspective in the spatial display model, taking the main reference object of the current cluster task summary set as the starting point, adjusting each display task screen around the marked display task screen one by one to obtain an adjusted display task screen; S45, taking the adjusted display task screen as a reference, adjusting each screen around the adjusted display task screen one by one; and so on, until all elements in the current cluster task summary set except the main reference object are adjusted, and the adjusted current cluster task summary set is transmitted to step S46; S46, receiving the transmission result of step S45, and taking the summary set of each display task screen in the received transmission result as the corresponding fusion screen set; In the process of adjusting the display task screens around the marked display task screen in step S44, the display task screens to be adjusted are matched with the preset screen adjustment schemes in the database respectively, and the display task screen adjustment result with the smallest image texture matching coefficient at the connection position with the marked display task screen after executing the preset screen adjustment schemes is used as the adjusted display task screen.

7. A multimedia fusion system based on task collaboration technology, characterized in that: The system includes the following modules: A cluster task summary module, which obtains the multi-machine cluster corresponding to the display terminal, receives the display tasks matched by each device in the multi-machine cluster in real time, and summarizes the display tasks corresponding to each device in the multi-machine cluster at the same time to generate a cluster task summary set of the corresponding task time; An equipment stability analysis module, which evaluates the stability of each device based on the historical fluctuation status information of the reception response time of the multi-machine cluster to the display task, and generates a stability reference object bias value of the multi-machine cluster; A reference analysis acquisition module, wherein the reference analysis acquisition module acquires the main reference object of each display task in the cluster task summary set to be sent most recently based on the current time, as well as the corresponding screen of the rendered display task, and analyzes the display tasks of each current device based on the previous collaborative adaptation difference value; combines the stability reference object bias value of the multi-machine cluster, and selects the main reference object of the current cluster task summary set; A fusion adjustment module simulates the fusion picture of the current cluster task summary set based on the location of the subject reference object binding device, and adaptively adjusts the elements in the current cluster task summary set based on the subject reference object to generate a fusion picture set output.

8. The multimedia fusion system based on task collaboration technology according to claim 7 is characterized in that: The reference analysis acquisition module includes a task adaptation difference analysis unit and a subject reference object screening unit. The task adaptation difference analysis unit obtains the main reference object of each display task in the cluster task summary set to be sent most recently based on the current time, and the corresponding screen of the rendered display task, and analyzes the collaborative adaptation difference value of the display tasks of each current device based on the previous one; The subject reference object screening unit selects the subject reference object of the current cluster task summary set in combination with the stability reference object bias value of the multi-machine cluster.

Images