Information interaction method based on video channel link establishment

Abstract

The invention provides a method for carrying out information interaction based on video channel link establishment, which belongs to the technical field of communication, and comprises the following steps: an image sender generates an image frame to be transmitted, a colored tape is added to the edge of the image frame, the colored tape is divided into a plurality of areas, each area is filled with a color block, and the color block is filled with a color block; coding the control information through permutation and combination of color blocks with different colors; the image sender sends the image frame with the color band to an image receiver through a video channel; an image receiver receives an image frame, extracts the colored tape from the edge of the image frame, and analyzes the permutation and combination of color blocks in the colored tape to decode the control information; and the image receiver executes corresponding operation according to the decoded control information. According to the invention, the colored tape is embedded in the video stream for information coding, and the control signaling and the video content are synchronously transmitted without occupying an additional channel, so that the resource value of the existing video channel is utilized to the maximum extent.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method for information exchange based on video channel link establishment. Background Technology

[0002] With the continuous advancement of technology, satellite communication applications are becoming increasingly widespread. Currently, most satellite communication resources are used for data communication. Given the limited satellite channel resources, there is a growing demand for applications utilizing audio and video transmission channels for high-definition image transmission and control information exchange. In particular, to facilitate remote guidance and assistance to shipboard personnel from land, the audio and video service channels of the ship's satellite communication system can be used as communication channels to achieve information coordination between the ship and land, enabling remote guidance and assistance to shipboard personnel in handling various faults. However, with limited satellite audio and video transmission channel resources, it is impossible to solve the problem of automated interaction in transmitting high-definition images and control information between devices on both sides, nor is it possible to complete tasks such as automated high-definition image stitching using low-resolution video transmission channels. Summary of the Invention

[0003] This invention provides a method, electronic device, storage medium, and computer program product for information interaction based on video channel link establishment, in order to overcome the deficiencies in the prior art.

[0004] In a first aspect, the present invention provides a method for information interaction based on video channel link establishment, comprising: During video transmission between the two devices, each frame of the image will provide a portion of the image area as an information interaction area. This information area will be presented in the form of a strip of color. The inside of the strip will be divided into multiple square areas according to the area of ​​the information interaction area. These areas will be converted into different color codes according to the digital information of the information interaction content. All the areas combined will form the information code string of the entire information interaction data.

[0005] The rules for digital information and color encoding are as follows: Black is encoded as 0b000, white as 0b001, red as 0b010, yellow as 0b011, blue as 0b100, green as 0b101, orange as 0b110, and purple as 0x111. Each color block can be encoded as a 3-bit information area. Assuming 6 bytes of data need to be transmitted, such as a hexadecimal data segment of 0x230135071279, the data can be converted into a binary string of 0b001000110000000100110101000001110001001001111001. Dividing this into 16 information segments of 3 bits each, designated as 0b001, 0b000, 0b110, etc. 0b000, 0b000, 0b100, 0b110, 0b101, 0b000, 0b001, 0b110, 0b001, 0b001, 0b001, 0b111, 0b001 are then converted into 16 different color blocks in the information interaction area: white, black, orange, white, white, blue, orange, green, white, black, orange, black, black, black, purple, black. The corresponding information is conveyed using these color blocks.

[0006] Furthermore, according to the method of information interaction based on video channel linking according to the present invention, the color bands added to each frame of the image can be distributed on the left edge of the original image, or on the right, upper or lower edge of the original image, mainly using the color blocks in the color bands to identify each frame of the image.

[0007] Furthermore, according to the method for information interaction based on video channel linking according to the present invention, the color band is generated by the image sender and acquired and identified by the image receiver. The image sender and receiver interact with each other based on the color band. The image sender arranges and combines the color blocks within the color band to complete the information encoding based on each frame of the image, and appends the color band information to each frame of the image to be sent before transmission.

[0008] Furthermore, according to the method of information interaction based on video channel link establishment according to the present invention, the sending and receiving parties of the image rely on color blocks in the color band to interact with each other. The interaction information includes, but is not limited to, heartbeat, operation mode (link establishment, image stitching, annotation, QR code, etc.), and operation content (image position, etc.).

[0009] The operation content in the color ribbon interaction information depends on the operation mode; different operation modes represent different meanings of the operation content. In the link building mode, the operation content is the link building status information; in the splicing mode, the operation content is information such as image sequence number and splicing status; in the annotation mode, the operation content is the annotation status information; and in the QR code mode, the operation content is information such as transmission status.

[0010] Before video transmission between the two devices, both parties need to synchronize their current high-definition image library information.

[0011] Only after the connection between the two devices is established can subsequent image transmission tasks such as image stitching, annotation, and QR code transmission begin.

[0012] In video transmission between the two devices, the heartbeat in the color band interaction information serves as a keep-alive mechanism between the image receiver and sender. It uses the alternating change of two colors in a single color block as a heartbeat message during transmission. The image sender and receiver agree on a fixed color block in the color band as the heartbeat interaction information. Each time the image sender transmits a frame, it alternates the color of this color block. The image receiver determines whether image transmission and reception are normal by extracting information from the color block at a fixed position in the color band and checking for alternating changes. Both parties use this mechanism to achieve heartbeat perception for each transmitted image frame.

[0013] Furthermore, according to the method for information interaction based on video channel linking of the present invention, high-definition image transmission is achieved based on the resolution of the image supported by the low-resolution video transmission channel. The image sender, according to the resolution of the image to be transmitted and combined with the resolution supported by the low-resolution video transmission channel, segments the high-definition image into several images suitable for transmission in the low-resolution video transmission channel. The color blocks in the color band are encoded, and the operation mode of the color blocks in the color band is set to splicing. The operation content includes information such as the sequence number, splicing status, and relative position of the currently transmitted image segment. The image receiver parses the image according to the color block operation mode and operation content in the color band, processes each received frame, and thus realizes the restoration of the high-definition image at the receiving end.

[0014] Furthermore, according to the method for information interaction based on video channel link establishment of the present invention, information interaction of images supported by a low-resolution video transmission channel is achieved to realize reverse annotation of high-definition images. After the link is successfully established between the image sender and receiver, the user makes annotations on the high-definition image of the image receiver. The annotation outline is sent to the image sender through interactive information and superimposed on the high-definition image of the image sender, thereby completing the reverse annotation of the high-definition image.

[0015] Furthermore, according to the method for information interaction based on video channel linking of the present invention, information interaction based on images supported by a low-resolution video transmission channel is realized through QR code transmission. The core advantages of using QR codes for data transmission are high efficiency and convenience; information can be quickly read by scanning the code without manual input, and it boasts high storage density, capable of accommodating various types of data such as text, URLs, and images, carrying a larger amount of information. The transmission of QR codes by both ends of the device based on a low-resolution video transmission channel allows a single image to convey more information, improving transmission efficiency.

[0016] In a second aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for information interaction based on video channel link establishment as described above.

[0017] Thirdly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for information interaction based on video channel link establishment as described above.

[0018] Fourthly, the present invention also provides a computer program product, wherein when the computer program is executed by a processor, it implements the steps of the method for information interaction based on video channel link establishment as described in any of the preceding claims.

[0019] In summary, the information interaction scheme based on video channel link establishment provided by this invention has the following advantages compared with the prior art: (1) This invention uses color bands embedded in the video stream for information encoding, and transmits control signaling synchronously with the video content without occupying additional channels. This is particularly suitable for bandwidth-constrained scenarios such as satellites, maximizing the resource value of existing video channels.

[0020] (2) The image segmentation and stitching are guided by color band signaling. This invention adapts the high-bandwidth high-definition image transmission task to the low-resolution video channel, solving the key technical problem of transmitting broadband content in narrowband environment.

[0021] (3) By utilizing the heartbeat mechanism (color alternation) in the color band, the link connection failure and frame loss can be monitored frame by frame. This low-overhead real-time monitoring provides a basic guarantee for the stability of remote interaction and can quickly detect and locate transmission faults.

[0022] (4) By defining different operation modes (such as link building, splicing, and annotation), this invention enables the basic video channel to have rich interactive capabilities. It can flexibly switch between different working modes, perfectly supporting complex applications such as two-way interaction in remote assistance (such as reverse annotation), and simplifying the system architecture. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1This is a schematic diagram of a color band information interaction method in one embodiment; Figure 2 This is a flowchart illustrating an interactive chain-building method in one embodiment; Figure 3 This is a schematic diagram of image segmentation in one embodiment; Figure 4 This is a schematic diagram of image stitching in one embodiment; Figure 5 This is a flowchart of the image fragment transmission process in one embodiment; Figure 6 This is a schematic diagram of image labeling in one embodiment; Figure 7 This is a schematic diagram of transmitting a QR code in one embodiment; Figure 8 It is an electronic device in one embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0027] This invention provides a method for information exchange based on video channel link establishment, comprising: The image sender generates an image frame to be transmitted, and adds a color band to the edge of the image frame. The color band is divided into multiple regions, and each region is filled with a color block. Control information is encoded by the arrangement and combination of different color blocks. The control information includes operation mode information and operation content information. The operation mode information is used to indicate the purpose of the current interaction. The image sender transmits the image frame with the color band attached to the image receiver through the video channel; The image receiver receives the image frame, extracts the color band from the edge of the image frame, and analyzes the arrangement and combination of color blocks in the color band to decode the control information; The image receiver performs corresponding operations based on the decoded control information.

[0028] The following is combined with Figures 1-8 The above method is illustrated by several embodiments.

[0029] Figure 1 This is a schematic diagram of color band and color block information interaction in one embodiment. Taking the information interaction area expected to occupy the left side of the image as an example (occupying the vertical area is because the image is expected to be displayed horizontally, and the horizontal length is relatively long; even if a portion of the area is cropped, the occupied image area is small), assuming a horizontal occupation of 50 pixels, even in low-resolution video mode, it only occupies 7% of the image area. Using this image area as the information interaction area, there are at least 576 pixels vertically. Assuming each 50*50 pixel is a single information interaction point, also called a color block, then the color band has at least 11 information interaction points. Each information interaction point has a 4-bit state, which meets the requirements for basic interactive information transmission. After the image is sent to the other end device, the other end device takes pixels in a fixed area and completes one-way information transmission based on the changes in the pixels. The other end then completes the two-way information interaction process by sending back the state information of the information interaction points. In order to enable the image receiver to accurately extract the information of the color block identifiers in the color band, the image receiver will use a multi-point detection method to identify the color blocks in the color band, improving the accuracy and reliability of color block identification in the color band.

[0030] Reference Figure 1 In another embodiment, the heartbeat interaction between the image sender and receiver is achieved by changing the color information of color blocks at fixed positions in the color band carried by the image frame in the low-resolution video transmission channel. For example, 1 bit is used to correspond to one color block, with 0 indicating a black block and 1 indicating a white block. The change in the color of this block serves as the heartbeat interaction between the image sender and receiver. The first color block in the image color band is filled with black, and the first color block in the next frame's image color band is filled with white. This black-and-white alternation of the first color block in the color band achieves the heartbeat interaction between the image sender and receiver. The image receiver identifies the color blocks marked with heartbeats in the color band, thereby monitoring whether frame loss has occurred during image transmission, providing an efficient means of monitoring the integrity of image transmission.

[0031] In another embodiment, color bands are used to implement interactive link establishment based on a low-resolution video transmission channel. For example, 2 bits are used to correspond to 2 color blocks, and a combination of red and blue is used to encode the 2 color blocks, supporting four encoding methods corresponding to four operation modes. The color blocks in the color band representing the operation mode are filled with red + red to indicate the link establishment operation mode. In the link establishment operation mode, the operation content is represented by 1 bit to correspond to 2 color blocks, and yellow and green are used to indicate successful link establishment and link establishment failure status.

[0032] Reference Figure 2 The interactive connection establishment implementation is carried out in devices 101 and 102. The interactive connection establishment process specifically includes the following steps S201 to S207.

[0033] S201, the device 101 initiates a link establishment request. The link establishment request color band consists of a color block that marks the operation mode (link establishment), a color block that marks the link establishment status of device 101 (link establishment in progress), and a color block that marks the synchronization status (default). S202, after receiving the link establishment request, device 102 collects and identifies the color blocks in the color band. First, it sets the color block that marks the operation mode as link establishment, sets the link establishment status color block as link establishment in progress, sets the synchronization status color block as synchronization request, and sends a synchronization request to device 101. S203, after receiving the synchronization request, device 101 collects and identifies the color blocks in the color band. First, it sets the color block marked with the operation mode to establish the link, sets the color block in the link establishment state to establish the link, sets the color block in the synchronization state to the synchronization response, and sends the synchronization response to device 102. S204, after receiving the synchronization response, device 102 confirms the synchronization information. It collects and identifies the color blocks in the color ribbon, first setting the color block marked with the operation mode to establish the link, setting the color block with the link establishment status to establish the link successfully, and sending the link establishment success status to device 101; S205, after receiving the link establishment success information, device 101 collects and identifies the color blocks in the color band and updates the link establishment status of this device to "link establishment successful".

[0034] In another embodiment, refer to Figure 3This paper provides a method for high-definition image transmission based on image segmentation using color ribbons in a low-resolution video transmission channel. For example, in a 1080P (1920*1080) video transmission channel, if an 8K (7680*4320) high-definition image needs to be transmitted, the image needs to be segmented first, then transmitted, and the receiving end needs to stitch together all image segments to reconstruct the high-definition 8K image. (7680*4320) / (1920*1080)=16. According to the above calculation, in order to achieve high-definition transmission of an 8K image in a 4K channel, the 8K image needs to be segmented into 16 image segments. For the image segmentation sending end, in order to use the color band for information exchange, the first color block of the color band uses 1 bit to correspond to 1 color block, which is still used for heartbeat interaction. The color band uses 2 bits to correspond to 2 color blocks, which is used to identify image stitching. The first operation content uses 4 bits to correspond to 4 color blocks, which is used to mark the image segment sequence number. The next operation content uses 2 bits to correspond to 2 color blocks, which is used to mark the image segment position. The other operation content uses 3 bits to correspond to 3 color blocks, which is used to mark the stitching status.

[0035] Reference Figure 4 The image receiver uses the color block information in the color band to stitch image segments together. First, it identifies the color blocks indicating the operation mode, confirming an image stitching operation. Next, it identifies the color blocks indicating the image segment number, thus identifying the number of the received headshot segment. Then, it identifies the color blocks indicating the image position, thus identifying the position information of the received headshot segment. Combining this information with the image stitching status information in the color band, the high-definition image segments are stitched together to reconstruct a high-definition image.

[0036] In one embodiment, see Figure 5 The image sender and the image receiver transmit image blocks, and execute steps S501 to S507.

[0037] S501, after device 101 and device 102 complete the link establishment, device 101 initiates an image sending request, sets the color block in the color band corresponding to the operation mode as image stitching, sets the color block corresponding to the stitching state as stitching request, and at this time the content of other color blocks is the default; S502, after receiving the splicing request, device 102 sets the color block in the corresponding operation mode within the color band to image splicing, and sets the color block corresponding to the splicing status to receive response. At this time, the content of other color blocks is the default. S503, after receiving the response, device 101 sets the color block corresponding to the operation mode in the color band to image stitching, sets the color block corresponding to the stitching state to image segment sending, sets the image segment sequence number color block to the sequence number of the image segment, sets the image position color block to the relative position information of the image segment, and then sends the image block and color band to device 102. In step S504, after device 102 identifies the image stitching mode using color blocks and determines that the image is in the image segment sending state, it identifies the image sequence number using color block information corresponding to the image segment sequence number, and identifies the relative position of the image using color block information corresponding to the image segment position. The received image is then processed. After processing, the color block corresponding to the stitching state is set to indicate that the image segment reception processing is complete. The image segment sequence number color block is set to the sequence number of the image segment, and the image position color block is set to the relative position information of the image segment. Finally, the color band is sent to device 101. If the image segment is not completely transmitted, steps S503 and S504 will be repeated multiple times until the entire image is transmitted. S505, after confirming that all image segments have been transmitted, device 101 will send an image segment transmission completion request to device 102 and set the color block corresponding to the stitching status to the image segment transmission completion request. S506, after receiving the image segment transmission completion request, device 102 will send an image reception completion confirmation to device 101, set the color block corresponding to the stitching status to the image reception completion confirmation, and complete the stitching of all image segments; S507, after receiving confirmation that image reception is complete, device 101 ends the transmission of the entire high-definition image.

[0038] In one embodiment, a method for image labeling using color blocks is also involved. See schematic diagram. Figure 6 As shown, it supports marking images using a stylus and mouse, and the markings support irregular shapes, numbers, letters, etc. The marker sender sets the operation mode of the color block in the color band to annotation and sends the outline information of the marker. The marker receiver collects the color band information and overlays the marker information onto the original image to recover the high-definition image with the markings.

[0039] In one embodiment, a method for transmitting QR codes using color ribbons is also involved. See schematic diagram. Figure 7 As shown, after the image sender and receiver successfully establish a link, they can send and receive QR codes based on the color band. The initiator of the QR code uses the operation mode color block in the color band to mark the current QR code transmission mode, and uses the operation content color block in the color band to represent the QR code transmission status. The receiver of the QR code first identifies the operation mode color block of the color band. When it is identified that it is a QR code operation, it identifies the current QR code transmission status through the QR code transmission status, thereby completing the entire QR code transmission.

[0040] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a method for information interaction based on video channel connection establishment. This method includes: an image sender generating an image frame to be transmitted, attaching a color band to the edge of the image frame, the color band being divided into multiple regions, each region being filled with a color block, and encoding control information through the arrangement and combination of different color blocks; wherein the control information includes operation mode information and operation content information, the operation mode information indicating the purpose of the current interaction; the image sender sending the image frame with the attached color band to an image receiver via the video channel; the image receiver receiving the image frame, extracting the color band from the edge of the image frame, and parsing the arrangement and combination of color blocks in the color band to decode the control information; and the image receiver performing corresponding operations based on the decoded control information.

[0041] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to execute the method for information interaction based on video channel link establishment provided in the above embodiments. In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for information interaction based on video channel link establishment provided in the above embodiments.

[0042] In summary, the information interaction scheme based on video channel link establishment provided by this invention has the following advantages compared with the prior art: (1) This invention uses color bands embedded in the video stream for information encoding, and transmits control signaling synchronously with the video content without occupying additional channels. This is particularly suitable for bandwidth-constrained scenarios such as satellites, maximizing the resource value of existing video channels.

[0043] (2) The image segmentation and stitching are guided by color band signaling. This invention adapts the high-bandwidth high-definition image transmission task to the low-resolution video channel, solving the key technical problem of transmitting broadband content in narrowband environment.

[0044] (3) By utilizing the heartbeat mechanism (color alternation) in the color band, the link connection failure and frame loss can be monitored frame by frame. This low-overhead real-time monitoring provides a basic guarantee for the stability of remote interaction and can quickly detect and locate transmission faults.

[0045] (4) By defining different operation modes (such as link building, splicing, and annotation), this invention enables the basic video channel to have rich interactive capabilities. It can flexibly switch between different working modes, perfectly supporting complex applications such as two-way interaction in remote assistance (such as reverse annotation), and simplifying the system architecture.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for information interaction based on video channel link establishment, characterized in that, The method comprises: An image sender generates an image frame to be transmitted, and appends a color band to an edge of the image frame, the color band being divided into multiple regions, each region being filled with a color block, and control information being encoded by arrangement and combination of different color blocks; wherein the control information comprises operation mode information and operation content information, and the operation mode information is used to indicate a use of current interaction; The image sender transmits the image frame with the appended color band to an image receiver through a video channel; The image receiver receives the image frame, extracts the color band from the edge of the image frame, and parses arrangement and combination of the color blocks in the color band to decode the control information; The image receiver performs corresponding operations according to the decoded control information. 2.The method of claim 1, wherein, The control information further comprises heartbeat information; The heartbeat information is represented by color of a color block at a predetermined position in the color band, and is used to monitor integrity of image frame transmission. 3.The method of claim 1, wherein, The operation mode information comprises a link establishment mode; Before image transmission starts, the image sender and the image receiver perform link establishment negotiation by setting the operation mode information in the color band to the link establishment mode, and the operation content information comprises a link establishment state; After link establishment succeeds, subsequent image data transmission starts.

4. The method for information interaction based on video channel link establishment according to claim 1, characterized in that, When resolution of an original image to be transmitted is higher than image resolution that can be directly transmitted by the video channel, the method further comprises: The image sender divides the original image into multiple image segments; The operation mode information in the color band is set to an image splicing mode, and the operation content information is set to sequence information and relative position information of the image segments; The image sender transmits the image segments and corresponding color bands in sequence; The image receiver splices and restores the received multiple image segments to the original image according to the decoded operation content information.

5. The method for information interaction based on video channel link establishment according to claim 1, characterized in that, The operation mode information comprises a labeling mode; In the labeling mode, the image receiver generates labeling information of a received image, and carries the labeling information through the color band of an image frame returned to the image sender; After decoding the labeling information, the image sender superimposes and displays the labeling information on a locally displayed image.

6. The method for information interaction based on video channel link establishment according to claim 1, characterized in that, The operation mode information comprises a two-dimensional code transmission mode, and the operation content information comprises a two-dimensional code transmission state.

7. The method for information interaction based on video channel link establishment according to claim 1, characterized in that, The color band is appended to a left edge, a right edge, an upper edge or a lower edge of the original image.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement steps of the method for information interaction based on link establishment through a video channel according to any one of claims 1 to 7. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement steps of the method for information interaction based on link establishment through a video channel according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement steps of the method for information interaction based on link establishment through a video channel according to any one of claims 1 to 7.

Images