A method and device for data processing

Abstract

The embodiment of the present invention provides a method and device for data processing, the method comprising: receiving a browsing request for non-audio and video files sent by a visual network terminal; generating one or more sliced ​​image data corresponding to the non-audio and video files; determining target sliced ​​image data from the one or more sliced ​​image data; generating video frame data according to the target sliced ​​image data, and sending the video frame data to the visual network terminal. Through the embodiment of the present invention, the visual network terminal can directly access the non-audio and video files on the visual network storage gateway without making special modifications to the visual network terminal, which greatly improves the speed at which users access non-audio and video files, meets the user's usage needs, and improves the user's experience of using the visual network device.

Description

Technical Field

[0001] The present invention relates to the field of visual networking technology, and in particular to a method and device for data processing. Background Art

[0002] The visual Internet is an important milestone in the development of the Internet. It is a real-time network that can realize real-time transmission of high-definition video and push many Internet applications towards high-definition video and high-definition face-to-face communication.

[0003] At present, in order to facilitate users to use the visual network, a visual network storage gateway can be deployed in the visual network, which can be a visual network dedicated storage interface host, providing a unified interface for the user's terminal and providing storage services for other visual network services. When using the storage service of the visual network storage gateway, it can mainly store office documents, audio, video and other formats.

[0004] However, the visual networking terminals usually use embedded video devices. If no special modification is made to the visual networking terminals (mainly the modification of terminal software function support), it is impossible to directly display the non-audio and video document files in the visual networking storage gateway. It can be seen that the existing visual networking storage gateway is difficult to meet the functional requirements of enterprise customers. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide a method and device for overcoming the above problems or at least partially solving the above problems, including:

[0006] In a first aspect, a method for data processing is provided, the method comprising:

[0007] Receiving a browsing request for a non-video file sent by a video network terminal;

[0008] Generate one or more slice image data corresponding to the non-video file;

[0009] Determine target slice picture data from the one or more slice picture data;

[0010] Video frame data is generated according to the target slice image data, and the video frame data is sent to the visual network terminal.

[0011] Optionally, the generating one or more slice image data corresponding to the non-video file includes:

[0012] Opening the non-video file in a preset virtual window;

[0013] Capturing the image displayed in the virtual window to obtain screenshot image data;

[0014] The screenshot image data is sliced ​​to obtain one or more slice image data.

[0015] Optionally, determining target slice image data from the one or more slice image data includes:

[0016] Get the screen resolution and determine the number of sliced ​​images based on the screen resolution;

[0017] From the one or more slice picture data, target slice picture data satisfying the number of slice pictures is determined.

[0018] Optionally, determining, from the one or more slice picture data, target slice picture data that satisfies the number of slice pictures includes:

[0019] Determine, from the one or more slice picture data, target slice picture data that meets the number of slice pictures and includes initial slice picture data;

[0020] Alternatively, in response to the playback control operation, target slice picture data satisfying the number of slice pictures is determined from the one or more slice picture data.

[0021] Optionally, there are multiple target slice picture data, and generating video frame data according to the target slice picture data includes:

[0022] Performing image stitching on the plurality of target slice image data to obtain stitched image data;

[0023] The format of the spliced ​​picture data is converted to obtain video frame data.

[0024] Optionally, sending the video frame data to the visual networking terminal includes:

[0025] The video frame data is sent to the visual networking terminal in a loop until the next playback control operation is detected or the browsing of the non-video file is stopped.

[0026] Optionally, the one or more slice image data have numbering information.

[0027] In a second aspect, a data processing device is provided, the device comprising:

[0028] A browsing request receiving module, used for receiving a browsing request for a non-video file sent by a video network terminal;

[0029] A slice image data generation module, used to generate one or more slice image data corresponding to the non-video file;

[0030] A target slice picture data determination module, used to determine target slice picture data from the one or more slice picture data;

[0031] The video frame data generation module is used to generate video frame data according to the target slice image data, and send the video frame data to the visual network terminal.

[0032] Optionally, the slice image data generating module further includes:

[0033] A non-audio and video file opening submodule, used to open the non-audio and video file in a preset virtual window;

[0034] A virtual window capture submodule, used to capture the picture displayed in the virtual window to obtain screenshot picture data;

[0035] The screenshot image data slicing submodule is used to slice the screenshot image data to obtain one or more slice image data.

[0036] Optionally, the target slice image data determination module further includes:

[0037] The submodule for determining the number of sliced ​​images is used to obtain the screen resolution and determine the number of sliced ​​images according to the screen resolution;

[0038] The target slice picture data determination submodule is used to determine target slice picture data that meets the number of slice pictures from the one or more slice picture data.

[0039] Optionally, the target slice image data determination submodule further includes:

[0040] a target slice picture data determining unit, configured to determine, from the one or more slice picture data, target slice picture data that meets the number of slice pictures and includes the initial slice picture data;

[0041] Alternatively, in response to the playback control operation, target slice picture data satisfying the number of slice pictures is determined from the one or more slice picture data.

[0042] Optionally, the video frame data generation module includes:

[0043] A target slice image data splicing submodule is used to splice the plurality of target slice image data to obtain spliced ​​image data;

[0044] The spliced ​​picture data conversion submodule is used to convert the format of the spliced ​​picture data to obtain video frame data.

[0045] Optionally, the video frame data generating module further includes:

[0046] The video frame data cyclic sending submodule is used to cyclically send the video frame data to the visual network terminal until the next playback control operation is detected or the browsing of the non-audio and video files is stopped.

[0047] Optionally, the one or more slice image data have numbering information.

[0048] According to a third aspect, a server is provided, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the data processing method as described above when executed by the processor.

[0049] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for data processing as described above is implemented.

[0050] The embodiments of the present invention have the following advantages:

[0051] In an embodiment of the present invention, by receiving a browsing request for a non-video file sent by a visual network terminal, one or more slice image data corresponding to the non-video file is generated, target slice image data is determined from the one or more slice image data, video frame data is generated according to the target slice image data, and the video frame data is sent to the visual network terminal, so that the visual network terminal can directly access the non-audio and video files on the visual network storage gateway without making special modifications to the visual network terminal, which greatly improves the speed at which users access non-audio and video files, meets user usage needs, and improves the user experience of visual network devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0053] Figure 1 This is a schematic diagram of a visual network provided by an embodiment of the present invention;

[0054] Figure 2 It is a schematic diagram of the hardware structure of a node server provided by an embodiment of the present invention;

[0055] Figure 3 It is a schematic diagram of the hardware structure of an access switch provided by an embodiment of the present invention;

[0056] Figure 4 It is a hardware structure diagram of an Ethernet protocol conversion gateway provided by one embodiment of the present invention;

[0057] Figure 5 is a flowchart of a data processing method provided by an embodiment of the present invention;

[0058] Figure 6 is a flowchart of another data processing method provided by an embodiment of the present invention;

[0059] Figure 7 is a specific example diagram of a data processing method provided by an embodiment of the present invention;

[0060] Figure 8 It is a specific flow chart of a data processing method provided by one embodiment of the present invention;

[0061] Fig. 9 It is a structural block diagram of a data processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] The visual Internet is an important milestone in the development of the Internet. It is a real-time network that can realize real-time transmission of high-definition video and push many Internet applications towards high-definition video and high-definition face-to-face communication.

[0064] The visual network adopts real-time high-definition video exchange technology. It can integrate the required services such as high-definition video conferencing, video monitoring, intelligent monitoring analysis, emergency command, digital radio and television, delayed TV, online teaching, live broadcast, VOD on demand, TV mail, personalized recording (PVR), intranet (self-operated) channels, intelligent video broadcast control, information release and dozens of other video, voice, picture, text, communication, data and other services on a single system platform, and realize high-definition quality video playback through TV or computer.

[0065] In order to enable those skilled in the art to better understand the embodiments of the present invention, the visual network is introduced as follows:

[0066] Some of the technologies used in the visual Internet of Things are as follows:

[0067] Network Technology

[0068] The network technology innovation of visual networking improves the traditional Ethernet to cope with the huge potential video traffic on the network. Different from simple network packet switching or network circuit switching, visual networking technology uses packet switching to meet streaming needs. Visual networking technology has the flexibility, simplicity and low price of packet switching, and the quality and security guarantee of circuit switching, realizing the seamless connection of the whole network switching virtual circuit and data format.

[0069] Switching Technology

[0070] Visual networking uses the two advantages of Ethernet: asynchronism and packet switching. It eliminates Ethernet defects under the premise of full compatibility, and has seamless end-to-end connection across the entire network, directly connects to user terminals, and directly carries IP data packets. User data does not require any format conversion within the entire network. Visual networking is a more advanced form of Ethernet and a real-time switching platform that can achieve large-scale real-time transmission of high-definition video across the entire network, which is currently unattainable on the Internet, and pushes many network video applications towards high definition and unification.

[0071] Server Technology

[0072] The server technology on the visual network and unified video platform is different from the traditional server. Its streaming media transmission is based on connection-oriented. Its data processing capability is independent of traffic and communication time. A single network layer can include signaling and data transmission. For voice and video services, the complexity of streaming media processing on the visual network and unified video platform is much simpler than data processing, and its efficiency is more than 100 times higher than that of traditional servers.

[0073] Storage Technology

[0074] The ultra-high-speed storage technology of the unified video platform adopts the most advanced real-time operating system to adapt to the media content with ultra-large capacity and ultra-large flow, mapping the program information in the server command to the specific hard disk space. The media content no longer passes through the server, but is directly delivered to the user terminal in an instant. The user generally waits for less than 0.2 seconds. The optimized sector distribution greatly reduces the mechanical movement of the hard disk head seeking, and the resource consumption accounts for only 20% of the same level of IP Internet, but it generates 3 times more concurrent traffic than the traditional hard disk array, and the overall efficiency is improved by more than 10 times.

[0075] Network Security Technology

[0076] The structural design of the visual network completely eradicates the network security issues that plague the Internet through separate licensing for each service and complete isolation of equipment and user data. Generally, anti-virus programs and firewalls are not required, and attacks by hackers and viruses are eliminated, providing users with a structurally worry-free and secure network.

[0077] Service Innovation Technology

[0078] The unified video platform integrates services and transmission. Whether it is a single user, a private network user or a network, it is just an automatic connection. User terminals, set-top boxes or PCs are directly connected to the unified video platform to obtain rich and colorful multimedia video services in various forms. The unified video platform uses a "recipe-style" table configuration mode to replace traditional complex application programming. Complex applications can be implemented with very little code, realizing "unlimited" new business innovation.

[0079] The networking of the visual network is as follows:

[0080] Visual networking is a centrally controlled network structure. The network can be a tree network, star network, ring network, etc., but on this basis, there needs to be a centralized control node in the network to control the entire network.

[0081] like Figure 1 As shown in the figure, the visual network is divided into two parts: access network and metropolitan area network.

[0082] The equipment in the access network can be divided into three categories: node server, access switch, and terminal (including various set-top boxes, encoding boards, storage, etc.). The node server is connected to the access switch, and the access switch can be connected to multiple terminals and can be connected to Ethernet.

[0083] The node server is a node in the access network that performs centralized control functions and can control access switches and terminals. The node server can be directly connected to the access switch or directly connected to the terminal.

[0084] Similarly, the devices in the metropolitan area network can also be divided into three categories: metropolitan area servers, node switches, and node servers. The metropolitan area servers are connected to the node switches, and the node switches can be connected to multiple node servers.

[0085] The node server is a node server of the access network part, that is, the node server belongs to both the access network part and the metropolitan area network part.

[0086] The metropolitan area server is a node with centralized control function in the metropolitan area network, which can control the node switch and node server. The metropolitan area server can be directly connected to the node switch or the node server.

[0087] It can be seen from this that the entire visual network is a hierarchical and centrally controlled network structure, and the network controlled by the node server and the metropolitan area server can be various structures such as tree, star, and ring.

[0088] Figuratively speaking, the access network part can form a unified video platform (the part in the dotted circle), and multiple unified video platforms can form a visual network; each unified video platform can be interconnected through the metropolitan area and wide area visual networks.

[0089] 1. Classification of visual networking equipment

[0090] 1.1 The devices in the visual network of the embodiment of the present invention can be mainly divided into three categories: servers, switches (including Ethernet protocol conversion gateways), terminals (including various set-top boxes, encoding boards, storage devices, etc.). The visual network as a whole can be divided into a metropolitan area network (or a national network, a global network, etc.) and an access network.

[0091] 1.2 The equipment in the access network can be mainly divided into three categories: node server, access switch (including Ethernet protocol conversion gateway), terminal (including various set-top boxes, encoding boards, storage, etc.).

[0092] The specific hardware structure of each access network device is as follows:

[0093] Node server:

[0094] like Figure 2 As shown, it mainly includes a network interface module 201, a switching engine module 202, a CPU module 203, and a disk array module 204;

[0095] Among them, the packets coming from the network interface module 201, the CPU module 203, and the disk array module 204 all enter the switching engine module 202; the switching engine module 202 performs an operation of checking the address table 205 on the incoming packets, thereby obtaining the packet's guidance information; and according to the packet's guidance information, the packet is stored in the queue of the corresponding packet buffer 206; if the queue of the packet buffer 206 is close to full, it is discarded; the switching engine module 202 polls all packet buffer queues, and forwards if the following conditions are met: 1) the port sending buffer is not full; 2) the queue packet counter is greater than zero. The disk array module 204 mainly implements the control of the hard disk, including the initialization, reading and writing of the hard disk; the CPU module 203 is mainly responsible for the protocol processing between the access switch and the terminal (not shown in the figure), the configuration of the address table 205 (including the downstream protocol packet address table, the upstream protocol packet address table, and the data packet address table), and the configuration of the disk array module 204.

[0096] Access switch:

[0097] like Figure 3 As shown, it mainly includes a network interface module (a downlink network interface module 301, an uplink network interface module 302), a switching engine module 303 and a CPU module 304;

[0098] Among them, the packet (uplink data) coming from the downlink network interface module 301 enters the packet detection module 305; the packet detection module 305 detects whether the destination address (DA), source address (SA), data packet type and packet length of the packet meet the requirements. If they meet the requirements, the corresponding stream identifier (stream-id) is allocated and enters the switching engine module 303, otherwise it is discarded; the packet (downlink data) coming from the uplink network interface module 302 enters the switching engine module 303; the data packet coming from the CPU module 304 enters the switching engine module 303; the switching engine module 303 processes the incoming packet The address table 306 is looked up to obtain the packet's guidance information; if the packet entering the switching engine module 303 is from the downstream network interface to the upstream network interface, the packet is stored in the queue of the corresponding packet buffer 307 in combination with the stream identifier (stream-id); if the queue of the packet buffer 307 is nearly full, it is discarded; if the packet entering the switching engine module 303 is not from the downstream network interface to the upstream network interface, the data packet is stored in the queue of the corresponding packet buffer 307 according to the packet's guidance information; if the queue of the packet buffer 307 is nearly full, it is discarded.

[0099] The switching engine module 303 polls all packet buffer queues. In the embodiment of the present invention, there are two situations:

[0100] If the queue is from the downlink network interface to the uplink network interface, it is forwarded if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero; 3) a token generated by the bit rate control module is obtained;

[0101] If the queue is not going from the downstream network interface to the upstream network interface, it is forwarded if the following conditions are met: 1) the port send buffer is not full; 2) the queue packet counter is greater than zero.

[0102] The code rate control module 308 is configured by the CPU module 304 to generate tokens for all packet buffer queues from the downstream network interface to the upstream network interface within a programmable interval to control the code rate of the upstream forwarding.

[0103] The CPU module 304 is mainly responsible for protocol processing with the node server, configuration of the address table 306 , and configuration of the bit rate control module 308 .

[0104] Ethernet protocol conversion gateway:

[0105] like Figure 4 As shown, it mainly includes a network interface module (a downlink network interface module 401, an uplink network interface module 402), a switching engine module 403, a CPU module 404, a packet detection module 405, a bit rate control module 408, an address table 406, a packet buffer 407 and a MAC adding module 409, and a MAC deleting module 410.

[0106] The data packet from the downlink network interface module 401 enters the packet detection module 405; the packet detection module 405 detects whether the Ethernet MAC DA, Ethernet MAC SA, Ethernet length or frame type, visual network destination address DA, visual network source address SA, visual network data packet type and packet length of the data packet meet the requirements, and if they meet the requirements, a corresponding stream identifier (stream-id) is assigned; then, the MAC deletion module 410 subtracts the MAC DA, MAC SA, length or frame type (2 bytes) and enters the corresponding receiving buffer, otherwise it is discarded;

[0107] The downlink network interface module 401 detects the sending buffer of the port. If there is a packet, it obtains the Ethernet MAC DA of the corresponding terminal based on the visual network destination address DA of the packet, adds the Ethernet MAC DA of the terminal, the MACSA of the Ethernet protocol conversion gateway, and the Ethernet length or frame type, and sends it.

[0108] The functions of other modules in the Ethernet protocol conversion gateway are similar to those of the access switch.

[0109] terminal:

[0110] It mainly includes network interface module, business processing module and CPU module; for example, the set-top box mainly includes network interface module, audio and video encoding and decoding engine module, CPU module; the encoding board mainly includes network interface module, audio and video encoding engine module, CPU module; the memory mainly includes network interface module, CPU module and disk array module.

[0111] 1.3 The equipment in the metropolitan area network can be divided into two categories: node server, node switch, and metropolitan area server. Among them, the node switch mainly includes network interface module, switching engine module and CPU module; the metropolitan area server mainly includes network interface module, switching engine module and CPU module.

[0112] 2. Definition of Visual Network Data Packet

[0113] 2.1 Access Network Data Packet Definition

[0114] The data packet of the access network mainly includes the following parts: destination address (DA), source address (SA), reserved bytes, payload (PDU), and CRC.

[0115] As shown in the following table, the data packet of the access network mainly includes the following parts:

[0116] DA SA Reserved Payload CRC

[0117] in:

[0118] The destination address (DA) consists of 8 bytes. The first byte indicates the type of data packet (such as various protocol packets, multicast data packets, unicast data packets, etc.), with a maximum of 256 possibilities. The second to sixth bytes are the metropolitan area network address, and the seventh and eighth bytes are the access network address.

[0119] The source address (SA) is also composed of 8 bytes and has the same definition as the destination address (DA).

[0120] The reserved bytes consist of 2 bytes;

[0121] The payload part has different lengths depending on the type of datagram. If it is a protocol packet, it is 64 bytes. If it is a single multicast data packet, it is 32+1024=1056 bytes. Of course, it is not limited to the above two types.

[0122] CRC consists of 4 bytes and its calculation method follows the standard Ethernet CRC algorithm.

[0123] 2.2 Definition of Metropolitan Area Network Data Packet

[0124] The topology of the metropolitan area network is a graph. There may be two or more connections between two devices, that is, there may be more than two connections between a node switch and a node server, a node switch and a node switch, and a node switch and a node server. However, the metropolitan area network address of a metropolitan area network device is unique. In order to accurately describe the connection relationship between metropolitan area network devices, a parameter: a label is introduced in the embodiment of the present invention to uniquely describe a metropolitan area network device.

[0125] The definition of labels in this manual is similar to the definition of labels in MPLS (Multi-Protocol Label Switch). Assuming that there are two connections between device A and device B, there are two labels for the data packet from device A to device B, and there are two labels for the data packet from device B to device A. Labels are divided into input labels and output labels. Assuming that the label of the data packet entering device A (input label) is 0x0000, the label of the data packet when leaving device A (output label) may become 0x0001. The access process of the metropolitan area network is a centralized access process, which means that the address allocation and label allocation of the metropolitan area network are dominated by the metropolitan area server, and the node switch and node server are only passively executed. This is different from the label allocation of MPLS, which is the result of mutual negotiation between the switch and the server.

[0126] As shown in the following table, the data packets of the metropolitan area network mainly include the following parts:

[0127] DA SAReserved TagPayload CRC

[0128] That is, destination address (DA), source address (SA), reserved byte (Reserved), label, payload (PDU), CRC. The label format can refer to the following definition: the label is 32 bits, of which the upper 16 bits are reserved and only the lower 16 bits are used. It is located between the reserved bytes and payload of the data packet.

[0129] Reference Figure 5 , shows a flowchart of a data processing method provided by an embodiment of the present invention, which may specifically include the following steps:

[0130] Step 501: receiving a browsing request for non-audio and video files sent by a video network terminal;

[0131] Among them, the visual networking terminal may include physical terminals such as a set-top box (SetTopBox, STB). The set-top box can be called a set-top box or a set-top box, which is a device that connects a television set with an external signal source. It can convert compressed digital signals into television content and display it on the television set. Generally speaking, the set-top box can be connected to a camera and a microphone to collect multimedia data such as video data and audio data, and can also be connected to a television set to play multimedia data such as video data and audio data.

[0132] Non-audio and video files can refer to office format files that are not audio or video, such as text files, word files, excel files, pdf files, etc.

[0133] In actual applications, the visual network terminal can provide a video-on-demand channel for non-audio and video files. When a user needs to view non-audio and video files directly in the visual network, the video-on-demand function of the visual network terminal can be used to watch non-audio and video files in the visual network. Then, the visual network storage gateway can receive the browsing request for non-audio and video files generated and sent by the visual network terminal used by the user, and perform the extraction of non-audio and video files. It should be noted that the browsing request can be transmitted in an encrypted or unencrypted manner, and the visual network storage gateway can receive the browsing request in a wireless or wired manner such as Bluetooth, wireless cellular network, wireless local area network, optical fiber, coaxial cable, etc., and the present invention does not specifically limit this.

[0134] In a specific implementation, a browsing request for non-audio and video files may include the case name, classification label, page number information, visual network terminal address, and other information used to identify the browsing object and the requesting terminal, so that the visual network storage gateway can flexibly select non-audio and video files according to the browsing request, and flexibly select the specific content to be viewed in the non-audio and video files.

[0135] Step 502: Generate one or more slice image data corresponding to the non-audio and video files;

[0136] Among them, one or more sliced ​​image data can be data of image type intercepted by the visual network storage gateway for non-audio and video files, and the intercepted one or more sliced ​​image data can record the substantial content of the non-audio and video files.

[0137] In actual applications, after receiving a browsing request sent by a visual network terminal, the visual network storage gateway can analyze the browsing request to determine the non-audio and video files that the visual network terminal actually needs to request to view. The visual network storage gateway can also open the non-audio and video files through the software corresponding to the non-audio and video files, and then call the screenshot tool to intercept the non-audio and video files that the visual network terminal needs to request to view and generate one or more sliced ​​image data. The screenshot specifications used by the screenshot tool can be the same, that is, the one or more sliced ​​image data intercepted by the screenshot tool can have the same length pixels and the same width pixels, so that the splicing operation of the sliced ​​image data can be better performed according to the same screenshot specifications.

[0138] It should be noted that one or more slice image data may adopt a conventional image data format, for example, the image data format may be a bitmap format, a jpeg format, etc. The present invention does not make any specific limitation to this.

[0139] Step 503: Determine target slice picture data from the one or more slice picture data;

[0140] In actual applications, when playing non-audio and video files on demand through a visual network terminal, users often need to carefully review and analyze a specific part of the content in the non-audio and video file. When the visual network storage gateway receives a browsing request for a non-audio and video file from the visual network terminal, it can select the target slice image data containing the specific part of the content that the user needs to review from one or more slice image data generated for the non-audio and video file.

[0141] In a specific implementation, for one or more sliced ​​image data generated for non-audio and video files, after selecting the target sliced ​​image data, the remaining one or more sliced ​​image data do not need to be deleted, but can continue to be stored in the visual network storage gateway. When the user needs to review other parts of the non-audio and video files, the target sliced ​​image data can be continued to be determined from the remaining one or more sliced ​​image data, so that the visual network storage gateway does not need to repeatedly cut the same non-audio and video files. When the user chooses to review the same non-audio and video file, the visual network storage gateway can quickly meet the review needs.

[0142] Step 504: Generate video frame data based on the target slice image data, and send the video frame data to the visual network terminal.

[0143] The video frame data may be data in a video format that is formed by combining frame-by-frame picture data and can be played at a certain frame rate.

[0144] In practical applications, the visual network storage gateway can synthesize the determined target slice image data to obtain video frame data that can be played by the visual network terminal, and then the visual network storage gateway can transmit the video frame data obtained by synthesizing the target slice image data to the visual network terminal that has sent a browsing request for the non-audio and video file through the visual network. Therefore, the visual network terminal, as an embedded video device, can directly play the video frame data, meeting the user's need to view the non-audio and video file.

[0145] As an example, the target slice image data can be a bitmap. In the process of generating video frame data, the visual network storage gateway can synthesize and convert the target slice image into YUV420 frame format, and then encode and compress the YUV420 format in H264 mode, and then push the H264 video frame at a preset frame rate to form a video stream output, and send it to the visual network terminal that has on-demand the non-audio and video file.

[0146] In an embodiment of the present invention, by receiving a browsing request for a non-audio and video file sent by a visual network terminal; generating one or more slice image data corresponding to the non-audio and video file; determining target slice image data from the one or more slice image data; generating video frame data based on the target slice image data, and sending the video frame data to the visual network terminal, the visual network terminal can directly access the non-audio and video files on the visual network storage gateway without making special modifications to the visual network terminal, which greatly improves the speed at which users access non-audio and video files, meets user usage needs, and improves the user experience of visual network devices.

[0147] Reference Figure 6 , shows a flowchart of another data processing method provided by an embodiment of the present invention, which may specifically include the following steps:

[0148] Step 601: receiving a browsing request for non-audio and video files sent by a video network terminal;

[0149] In actual applications, the visual network storage gateway can receive browsing requests sent by the visual network terminal for various non-audio and video files. The visual network storage gateway can analyze the non-audio and video file objects requested by the visual network terminal through the browsing requests, and can determine the specific file format used by the non-audio and video files.

[0150] Step 602: Open the non-audio and video file in a preset virtual window;

[0151] In actual applications, after receiving a browsing request sent by a visual network terminal, the visual network storage gateway can call the application software corresponding to the specific file format used by the non-audio and video files. Then, the visual network storage gateway can open the non-audio and video files requested by the visual network terminal in a virtual window through the application software.

[0152] The content in the virtual window is not on the screen, but is generated by the API interface provided by the operating system on the visual network storage gateway. A buffer is generated in the memory for storing application software that can open various non-audio and video files. From the perspective of the program, the virtual window is like a desktop, but its content is not displayed on the screen, but "displayed" in a specific memory area.

[0153] Step 603: capturing the image displayed in the virtual window to obtain screenshot image data;

[0154] In actual applications, the visual network storage gateway can call a preset screenshot tool to take screenshots of non-audio and video files opened by the application software in the virtual window until all the substantial content of the non-audio and video files is captured. Moreover, the visual network storage gateway can obtain the location of the non-audio and video files displayed by the application software through the graphical user interface (GUI) provided by the operating system.

[0155] In the specific implementation, when taking screenshots, non-audio and video files displayed by the application software can be screenshoted according to the preset width and preset height, and the virtual keyboard command page down can be called to turn the page, and the captured pictures can be buffered and saved, so that when the video is played, the pictures can be spliced ​​according to the height pixels of the screen and converted into video format, and output to the visual network terminal.

[0156] Step 604: Slice the screenshot image data to obtain one or more slice image data;

[0157] In practical applications, after capturing the image displayed in the virtual window and obtaining the screenshot image data, the screenshot content can be further sliced ​​according to a specific pixel height, wherein the specific pixel height can be an integer fraction of the on-demand video resolution, for example: when the standard definition resolution is 1280*720, the pixel width of the video or image is 1280 pixels, the pixel height is fixed to 720 pixels, and the pixel height of the slice can be set to 240 pixels, which is one-third of 720. Therefore, after the sliced ​​images are spliced ​​and converted, they can just cover the screen of the visual networking terminal.

[0158] In an embodiment of the present application, the one or more slice image data have numbering information.

[0159] Among them, obtaining one or more slice image data can be to generate one or more slice image format files in sequence, and the one or more slice image format files can be numbered, so that the networked storage gateway can select slice images in order according to the numbers, for example, a series of jpeg format files can be numbered 0001-0051, and temporarily saved to the file system of the storage gateway. It should be noted that the above specific numbers are only examples for explanation, and the numbering number and numbering method can be adjusted according to actual conditions, and the present invention does not specifically limit this.

[0160] Step 605: Obtain the screen resolution, and determine the number of sliced ​​images according to the screen resolution;

[0161] In actual applications, after obtaining one or more sliced ​​image data, the visual network storage gateway can obtain the screen resolution of the visual network terminal through the browsing request sent by the visual network terminal, and then calculate the maximum number of slices that the visual network terminal can display while keeping the on-demand video resolution unchanged based on the screen resolution of the visual network terminal and the pixel height of the slice.

[0162] It should be noted that the on-demand video resolution can adopt two conventional resolution modes of visual networking, namely, high-definition mode: 1920*1080p, and standard-definition mode: 1280*720p, and most of the visual networking terminals in the visual networking are compatible with displaying these two conventional resolution modes. That is, when the on-demand video resolution is the conventional resolution mode of the visual networking, there is no need to detect the screen resolution of various types of visual networking terminals, and 1920 pixels can be directly used as the pixel height of the visual networking terminal screen.

[0163] Step 606: Determine target slice picture data satisfying the number of slice pictures from the one or more slice picture data;

[0164] In actual applications, after determining the number of sliced ​​images according to the screen resolution, target sliced ​​image data that the visual network terminal actually needs to request to view and that meets the number of sliced ​​images can be selected from one or more sliced ​​image data.

[0165] In an embodiment of the present application, step 606 may include the following sub-steps:

[0166] Determine, from the one or more slice picture data, target slice picture data that meets the number of slice pictures and includes initial slice picture data;

[0167] Alternatively, in response to the playback control operation, target slice picture data satisfying the number of slice pictures is determined from the one or more slice picture data.

[0168] In actual applications, when a user sends a browsing request through a visual network terminal to open the non-audio and video file, and the user does not specify the number of pages to be viewed, the visual network storage gateway can select target slice image data that meets the number of slice images from the beginning of the non-audio and video file, that is, target slice image data including initial slice image data; when the user specifies the number of pages to be viewed for the first time in the non-audio and video file through the remote control device of the visual network terminal, or performs playback control operations such as fast forward, fast rewind, and page turning operations on the opened non-audio and video file, the visual network storage gateway can respond to the playback control operation and select the target slice image data corresponding to the playback control operations such as fast forward, fast rewind, and page turning operations.

[0169] The following are some examples of the playback control operations:

[0170] When the number of pages to be viewed is specified through the remote control device, page number detection can be performed on the non-audio and video files opened by the application software, the content that meets the specified number of pages to be viewed can be intercepted and sliced, and target slice image data that meets the number of slice images can be selected.

[0171] When a playback control operation of turning the page down is sent through the remote control device, according to the determined number of slice pictures, slice pictures are selected backward according to the number of slice pictures each time the page is turned down. For example, when the determined number of slice pictures is three slice pictures displayed on the current screen, three slice pictures are selected backward according to the number information of the slice pictures when the page is turned down.

[0172] When a fast-forward playback control operation is sent through the remote control device, several slice images currently being displayed and several slice images to be displayed can be spliced ​​into a large image, and the image can be captured in a downward sliding manner according to the pixel height of a screen in the large image, where the number of pixels to slide downward can be set according to the degree of fast forward, that is, the higher the degree of fast forward, the more pixels to slide downward, and then a picture with a screen pixel height is captured.

[0173] like Figure 7 A specific example diagram of a data processing method is shown. In the virtual desktop (virtual window) of the visual network storage gateway, the corresponding word file (non-audio and video file) can be opened through word (application software). The visual network storage gateway takes a screenshot of the displayed part of the document content and can automatically turn the page until all the content in the document is captured. The screenshot is then sliced, which can be specifically divided into 0001 slice picture, 0002 slice picture, 0003 slice picture...nmn slice picture; wherein, the visual network terminal can display up to three slice pictures. Then you can select 0003 slice picture-0005 slice picture for slice splicing, and push it out as a video stream after format conversion. When the visual network storage gateway receives the next page control of the visual network terminal, you can select 0006 slice picture-0008 slice picture for slice splicing and conversion output.

[0174] Step 607: Generate video frame data based on the target slice image data, and send the video frame data to the visual network terminal.

[0175] In actual applications, the visual network storage gateway can synthesize the determined target slice image data to obtain video frame data that can be played by the visual network terminal. Then the visual network storage gateway can transmit the video frame data obtained by synthesizing the target slice image data through the visual network to the visual network terminal that has sent a browsing request for the non-audio and video file.

[0176] In an embodiment of the present application, step 607 may include the following sub-steps:

[0177] Performing image stitching on the plurality of target slice image data to obtain stitched image data;

[0178] The format of the spliced ​​picture data is converted to obtain video frame data.

[0179] Among them, after determining the target slice picture data that meets the number of slice pictures, the multiple target slice picture data can be synthesized in a splicing manner to obtain spliced ​​picture data. Then, the format of the spliced ​​picture data is converted to convert the spliced ​​picture data of the picture type into video frame data in a video format that is composed of frame-by-frame picture data and can be played at a certain frame rate. For example, the target slice picture format in bitmap or jpeg format can be converted into a YUV420 frame format, and then encoded and compressed in H264, and the H264 video frame is pushed at a preset frame rate to form a video stream output.

[0180] In another embodiment of the present application, step 607 may further include the following sub-steps:

[0181] The video frame data is sent to the visual networking terminal in a loop until the next playback control operation is detected or the browsing of the non-audio and video files is stopped.

[0182] In actual applications, after generating video frame data, H264 video frames converted from image format can be sent cyclically according to a preset frame rate to form a continuous video stream and output to the visual networking terminal to ensure that the display screen on the visual networking terminal can remain unchanged. When the visual networking terminal sends to the next playback control operation, or stops browsing the non-audio and video files, it means that the user no longer needs to check the non-audio and video files on the current page, that is, the visual networking storage gateway stops sending H264 video frames to the visual networking terminal in a loop at this time.

[0183] It should be noted that although the H264 video frames converted from the image format are sent in a loop according to the preset frame rate, because the video frame data is a splicing of static images, the picture only needs to change when the page is turned or stopped. Therefore, a very low frame rate can be selected for loop playback, such as 2-3 frames per second, which can effectively reduce the bandwidth required to load the video stream.

[0184] In an embodiment of the present invention, by receiving a browsing request for a non-audio and video file sent by a visual network terminal, the non-audio and video file is opened in a preset virtual window, a picture displayed in the virtual window is captured to obtain screenshot picture data, the screenshot picture data is sliced ​​to obtain one or more slice picture data, the screen resolution is obtained, and the number of slice pictures is determined according to the screen resolution, and target slice picture data that meets the number of slice pictures is determined from the one or more slice picture data, video frame data is generated according to the target slice picture data, and the video frame data is sent to the visual network terminal, so that the visual network terminal can directly access the non-audio and video files on the visual network storage gateway, and the visual network terminal can implement a variety of access operations for non-audio and video files by sending playback control operations, thereby increasing the flexibility of accessing non-audio and video files through the visual network storage gateway and improving the user experience.

[0185] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0186] Reference Figure 8 , shows a specific flow chart of a data processing method provided by an embodiment of the present invention, which may specifically include the following steps:

[0187] The visual network V2V interface can be used to connect with the visual network terminal.

[0188] When the visual network terminal sends a browsing request for non-audio and video files, the visual network storage gateway can receive the document browsing request for the non-audio and video files by calling the API interface and process it.

[0189] The visual network storage gateway calls the virtual desktop (virtual window) to take screenshots and slices, and stores them in the file system of the visual network storage gateway.

[0190] Then the visual network storage gateway can perform slice playback selection and image synthesis conversion, and convert it into a video stream and output it to the visual network terminal through the visual network V2V interface.

[0191] When the visual network storage gateway receives the user's playback control operation through the API interface, it performs playback processing, that is, re-executes the slice playback selection according to the type of user playback control operation.

[0192] After re-executing the slice playback selection, the visual network storage gateway converts the image synthesis format and outputs it to the visual network terminal in the form of a video stream through the visual network V2V interface.

[0193] Reference Fig. 9 : shows a structural block diagram of a data processing device provided by an embodiment of the present invention, which may specifically include the following modules:

[0194] A browsing request receiving module 901 is used to receive a browsing request for non-audio and video files sent by a video network terminal;

[0195] A slice image data generating module 902, configured to generate one or more slice image data corresponding to the non-audio and video files;

[0196] A target slice picture data determination module 903 is used to determine target slice picture data from the one or more slice picture data;

[0197] The video frame data generation module 904 is used to generate video frame data according to the target slice image data, and send the video frame data to the visual network terminal.

[0198] In one embodiment of the present invention, the slice image data generation module further includes:

[0199] A non-audio and video file opening submodule, used to open the non-audio and video file in a preset virtual window;

[0200] A virtual window capture submodule, used to capture the picture displayed in the virtual window to obtain screenshot picture data;

[0201] The screenshot image data slicing submodule is used to slice the screenshot image data to obtain one or more slice image data.

[0202] In one embodiment of the present invention, the target slice image data determination module further includes:

[0203] The submodule for determining the number of sliced ​​images is used to obtain the screen resolution and determine the number of sliced ​​images according to the screen resolution;

[0204] The target slice picture data determination submodule is used to determine target slice picture data that meets the number of slice pictures from the one or more slice picture data.

[0205] In one embodiment of the present invention, the target slice image data determination submodule further includes:

[0206] a target slice picture data determining unit, configured to determine, from the one or more slice picture data, target slice picture data that meets the number of slice pictures and includes the initial slice picture data;

[0207] Alternatively, in response to the playback control operation, target slice picture data satisfying the number of slice pictures is determined from the one or more slice picture data.

[0208] In one embodiment of the present invention, the video frame data generation module includes:

[0209] A target slice image data splicing submodule is used to splice the plurality of target slice image data to obtain spliced ​​image data;

[0210] The spliced ​​picture data conversion submodule is used to convert the format of the spliced ​​picture data to obtain video frame data.

[0211] In one embodiment of the present invention, the video frame data generating module further includes:

[0212] The video frame data cyclic sending submodule is used to cyclically send the video frame data to the visual network terminal until the next playback control operation is detected or the browsing of the non-audio and video files is stopped.

[0213] In an embodiment of the present invention, the one or more slice image data have number information.

[0214] In an embodiment of the present invention, by receiving a browsing request for a non-audio and video file sent by a visual network terminal; generating one or more slice image data corresponding to the non-audio and video file; determining target slice image data from the one or more slice image data; generating video frame data based on the target slice image data, and sending the video frame data to the visual network terminal, the visual network terminal can directly access the non-audio and video files on the visual network storage gateway without making special modifications to the visual network terminal, which greatly improves the speed at which users access non-audio and video files, meets user usage needs, and improves the user experience of visual network devices.

[0215] An embodiment of the present invention further provides a server, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and the computer program implements the above data processing method when executed by the processor.

[0216] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above data processing method is implemented.

[0217] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0218] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0219] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0220] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0221] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0222] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0223] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0224] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0225] The above is a detailed introduction to a data processing method and device provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for data processing, characterized in that: The method comprises: Receiving a browsing request for non-audio and video files sent by a video network terminal; Generate one or more slice image data corresponding to the non-audio and video files; Determine target slice picture data from the one or more slice picture data; Generate video frame data according to the target slice image data, and send the video frame data to the visual network terminal; The step of determining target slice image data from the one or more slice image data comprises: Get the screen resolution and determine the number of sliced ​​images based on the screen resolution; Determine, from the one or more slice picture data, target slice picture data that satisfies the number of slice pictures; The generating one or more slice image data corresponding to the non-audio and video file includes: Open the non-audio and video file in a preset virtual window; Capturing the image displayed in the virtual window to obtain screenshot image data; The screenshot image data is sliced ​​to obtain one or more slice image data.

2. The method according to claim 1, characterized in that The determining, from the one or more slice picture data, target slice picture data that satisfies the number of slice pictures comprises: Determine, from the one or more slice picture data, target slice picture data that meets the number of slice pictures and includes initial slice picture data; Alternatively, in response to the playback control operation, target slice picture data satisfying the number of slice pictures is determined from the one or more slice picture data.

3. The method according to claim 1, characterized in that There are a plurality of target slice picture data, and generating video frame data according to the target slice picture data includes: Performing image stitching on multiple target slice image data to obtain stitching image data; The format of the spliced ​​picture data is converted to obtain video frame data.

4. The method according to claim 1, characterized in that: The sending of the video frame data to the visual networking terminal comprises: The video frame data is sent to the visual networking terminal in a loop until the next playback control operation is detected or the browsing of the non-audio and video files is stopped.

5. The method according to claim 1, characterized in that The one or more slice picture data have number information.

6. A data processing device, characterized in that: The device comprises: A browsing request receiving module, used to receive a browsing request for non-audio and video files sent by a video networking terminal; A slice image data generation module, used to generate one or more slice image data corresponding to the non-audio and video files; A target slice picture data determination module, used to determine target slice picture data from the one or more slice picture data; A video frame data generation module, used to generate video frame data according to the target slice image data, and send the video frame data to the visual network terminal; The target slice image data determination module also includes: The submodule for determining the number of sliced ​​images is used to obtain the screen resolution and determine the number of sliced ​​images according to the screen resolution; A target slice picture data determination submodule, used to determine target slice picture data that meets the number of slice pictures from the one or more slice picture data; The slice image data generation module also includes: A non-audio and video file opening submodule, used to open the non-audio and video file in a preset virtual window; A virtual window capture submodule, used to capture the picture displayed in the virtual window to obtain screenshot picture data; The screenshot image data slicing submodule is used to slice the screenshot image data to obtain one or more slice image data.

7. A server, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the data processing method according to any one of claims 1 to 5 when executed by the processor.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for data processing according to any one of claims 1 to 5 is implemented.

Images