Multi-picture splicing method and system

Abstract

The invention discloses a multi-picture splicing method and system. The method comprises the following steps: acquiring multiple frames of video images in real time; decoding the multiple frames of video images to obtain video frame contents; storing the video frame content into a memory; splicing the corresponding video frame content in the memory based on an output video demand input by a user to obtain an output video stream; pushing the output video stream to an application system; according to the method, the video frames are stored in the same memory, the needed video frames are extracted from the memory when splicing is needed, the mode is high in flexibility, in addition, multiple user-defined splicing modes can be supported, and the video frames stored in the memory can be used for various human body, human face, object and behavior recognition application systems and the like through a public protocol.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-image processing, and more specifically, to a multi-image splicing method and system. Background Art

[0002] The existing multi-screen splicing solution is mainly used on a single NVR (Network Video Recorder) or a dedicated multi-screen splicer. The NVR or multi-screen splicer reads the video stream from the network, decodes it internally, calculates the scaled size according to the splicing method selected by the user, and finally splices multiple scaled images into a single image, which is then output through the streaming media system. However, when faced with the splicing of different numbers of images, this multi-screen splicing solution has the problem of system inscalability and poor flexibility.

[0003] In summary, at this stage, there is an urgent need to develop a multi-image splicing method and system to solve one or more of the above-mentioned problems. Summary of the invention

[0004] An object of the present invention is to provide a new technical solution for a multi-image splicing method and system.

[0005] According to a first aspect of the present invention, a multi-image stitching method is provided, the method comprising:

[0006] Acquire multiple frames of video images in real time;

[0007] Decoding multiple video frames to obtain video frame content;

[0008] Storing the video frame content in a memory;

[0009] Based on the output video requirements input by the user, corresponding video frame contents in the memory are spliced ​​to obtain an output video stream;

[0010] The output video stream is pushed to the application system.

[0011] Optionally, the method further includes: analyzing the video frame content and storing the analysis result in the memory.

[0012] Optionally, the user inputs the output video requirement through output video content options, multi-screen format selection and whether to display analysis results options.

[0013] Optionally, in the case where the display analysis result is determined based on the output video requirement input by the user, the corresponding analysis result in the memory is processed and pushed to the application system together with the output video stream.

[0014] Optionally, the splicing of corresponding video frame contents in the memory based on the output video requirement input by the user to obtain the output video stream specifically includes:

[0015] Extracting corresponding video frame content from the memory based on output video requirements input by a user;

[0016] In the case where it is determined that scaling is required based on the output video requirement input by the user, scaling the video frame content;

[0017] In the case where a corresponding splicing method is determined based on an output video requirement input by a user, splicing the scaled video frame content;

[0018] The spliced ​​video frame content is encoded to obtain the output video stream.

[0019] Optionally, the memory is a distributed shared memory.

[0020] According to a second aspect of the present invention, there is provided a multi-image splicing system, the system comprising: multiple cameras, at least one decoder, a memory, a splicing device, and a streaming media device;

[0021] The multiple cameras are used to acquire multiple frames of video images in real time;

[0022] The decoder is used to decode multiple frames of video images to obtain video frame content;

[0023] The memory is used to store the video frame content;

[0024] The splicing device is used to splice the corresponding video frame contents in the memory based on the output video requirements input by the user to obtain an output video stream;

[0025] The streaming media device is used to push the output video stream to the application system.

[0026] Optionally, the system further includes: at least one analyzer, which is used to analyze the video frame content and store the analysis result in the memory.

[0027] Optionally, the user inputs the output video requirement through output video content options, multi-screen format selection and whether to display analysis results options.

[0028] Optionally, the streaming media device is further configured to process the corresponding analysis results in the memory and push them together with the output video stream to the application system when displaying the analysis results based on the output video requirements input by the user.

[0029] Optionally, the splicing device specifically includes:

[0030] an extraction unit, configured to extract corresponding video frame content from the memory based on an output video requirement input by a user;

[0031] A scaling unit, configured to perform scaling processing on the video frame content when scaling is required based on the output video demand input by the user;

[0032] A splicing unit configured to perform splicing processing on the scaled video frame content when a corresponding splicing method is determined based on an output video requirement input by a user;

[0033] The spliced ​​video frame content is encoded to obtain the output video stream.

[0034] Optionally, the memory is a distributed shared memory.

[0035] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in the multi-screen splicing method described in the first aspect of the present invention are implemented.

[0036] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the multi-screen splicing method described in the first aspect of the present invention are implemented.

[0037] According to an embodiment disclosed in the present invention, a multi-image splicing method and system of the present invention have the following beneficial effects:

[0038] In the multi-screen splicing method of the present invention, all video frames are stored in the same memory, and the required video frames are extracted from the memory when splicing is required. This method has high flexibility and can support a variety of user-defined splicing methods. The video frames stored in the memory can be used in various human body, face, object and behavior recognition application systems through public protocols.

[0039] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0041] Figure 1A schematic diagram of a multi-image splicing method according to an embodiment Figure 1 ;

[0042] Figure 2 A schematic diagram of a multi-image splicing method according to an embodiment Figure 2 ;

[0043] Figure 3 A schematic diagram of the structure of a multi-image splicing system provided according to an embodiment;

[0044] Figure 4 A schematic diagram of an electronic device. DETAILED DESCRIPTION

[0045] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0046] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0047] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0048] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0049] Embodiment 1:

[0050] See also Figure 1 As shown, an embodiment of the present invention provides a multi-image splicing method, the method comprising:

[0051] Step S1: acquiring multiple frames of video images (streams) in real time;

[0052] In this embodiment, video images (streams) can be acquired in real time through multiple cameras.

[0053] Step S2: Decoding the multiple frames of video images (streams), wherein the decoding process is to decode the video images (streams) into frame-by-frame images to obtain video frame content;

[0054] Optionally, the multi-image stitching method of this embodiment further includes: analyzing the video frame content and storing the analysis result in a memory; the analysis includes target detection, behavior analysis, target tracking, attribute recognition, OCR recognition, etc.

[0055] Step S3: storing the video frame content in a memory;

[0056] Preferably, the memory in the multi-image splicing method of this embodiment is a distributed shared memory. Distributed shared memory (DSM) is an important technology that has emerged in the development of parallel processing. It provides programmers with a logically unified address space, and any processor can directly read and write to this address space. It has the advantages of the scalability of the distributed memory structure, and also has the advantages of the shared memory structure, which is universal, portable, and easy to program.

[0057] Step S4: splicing the corresponding video frame contents in the memory based on the output video requirement input by the user to obtain an output video stream;

[0058] Optionally, in the multi-image stitching method of this embodiment, the user inputs output video requirements through output video content options, multi-image format selection, and whether to display analysis results options.

[0059] It should be noted that the output video content option, the multi-image format selection and the option of whether to display the analysis result can be set on the user input interface for the user to choose.

[0060] Optionally, in the multi-screen splicing method of this embodiment, when the display analysis results are determined based on the output video requirements input by the user, the corresponding analysis results in the memory are processed. The processing mainly refers to the size and combination of the output screens, such as single-screen output, 1×2 screens, 2×1 screens, 2×2 screens, 3×3 screens, 4×3 screens, and 4×4 screen display lights; and they are pushed to the application system together with the output video stream.

[0061] Optionally, step S4 in the multi-image stitching method of this embodiment specifically includes:

[0062] Extract corresponding video frame content from the memory based on the output video requirements input by the user;

[0063] When it is determined that scaling is required based on the output video requirement input by the user, scaling the video frame content;

[0064] In the case of determining a corresponding splicing method based on an output video requirement input by a user, performing splicing processing on the scaled video frame content;

[0065] The spliced ​​video frame content is encoded to obtain an output video stream.

[0066] Step S5: Push the output video stream to the application system.

[0067] In this embodiment, the application system can be a display interface, i.e., a display terminal, for displaying spliced ​​video images and / or analysis results; it can also be an analysis system for various human body, face, object and behavior recognition.

[0068] It should be noted that in this embodiment, the output video stream can be transmitted through video stream protocols such as rtsp / rtmp / hls.

[0069] See also Figure 2 As shown, the multi-image splicing method of the present invention is described in detail with a specific embodiment as follows:

[0070] 1) The camera acquires video images in real time;

[0071] 2) The decoder decodes the video image and stores the video frame in the distributed shared memory;

[0072] 3) Analyzing the video image and storing the analysis results in the distributed shared memory;

[0073] 4) Obtaining video frames and analysis results from the distributed shared memory, and determining whether scaling is required and the splicing method, so as to encode and splice the video frames to form an output video stream;

[0074] 5) Push the output video stream to the application system.

[0075] In the multi-screen stitching method of the embodiment of the present invention, all video frames are stored in the same memory, and the required video frames are extracted from the memory when stitching is required. This method has high flexibility and can support a variety of user-defined stitching methods. The video frames stored in the memory can be used in various human body, face, object and behavior recognition application systems through public protocols.

[0076] Embodiment 2:

[0077] The embodiment of the present invention provides a multi-image splicing system, based on the multi-image splicing method described in the first embodiment, see Figure 3 As shown, the system includes: multiple cameras 1, at least one decoder 2, a memory 3, a splicing device 4 and a streaming media device 5;

[0078] Among them, the multi-channel camera 1 is used to obtain multiple frames of video images in real time;

[0079] Decoder 2 is used to decode multiple video frames to obtain video frame content;

[0080] Memory 3 is used to store video frame content;

[0081] The splicing device 4 is used to splice the corresponding video frame contents in the memory based on the output video requirements input by the user to obtain an output video stream;

[0082] The streaming media device 5 is used to push the output video stream to the application system 6 .

[0083] Optionally, the multi-image splicing system of this embodiment further includes: at least one analyzer 7, which is used to analyze the video frame content and store the analysis result in the memory 3.

[0084] Optionally, in the multi-image splicing system of this embodiment, the user inputs output video requirements through output video content options, multi-image format selection, and whether to display analysis results options.

[0085] Optionally, the streaming media device 5 in the multi-screen splicing system of this embodiment is also configured to process the corresponding analysis results in the memory and push them to the application system 6 together with the output video stream when determining the display analysis results based on the output video requirements input by the user.

[0086] Optionally, the splicing device 4 in the multi-image splicing system of this embodiment specifically includes:

[0087] An extraction unit, configured to extract corresponding video frame content from the memory based on an output video requirement input by a user;

[0088] A scaling unit, configured to scale the video frame content when scaling is required based on the output video requirement input by the user;

[0089] A splicing unit is configured to splice the scaled video frame content in the case of determining a corresponding splicing method based on an output video requirement input by a user;

[0090] The spliced ​​video frame content is encoded to obtain an output video stream.

[0091] Optionally, the memory of the multi-image splicing system of this embodiment is a distributed shared memory.

[0092] It should be noted that in the multi-screen splicing system of this embodiment, the decoder is an edge device (decoding box) responsible for video stream access, decoding and video frame writing, and the analyzer is an edge device (analysis box) responsible for video analysis. All edge devices use distributed shared memory (DSM) for data exchange. When the number of screens to be spliced ​​increases, the system performance can be improved simply by increasing the number of edge devices (decoding boxes); when the number of screens decreases, the decoding box can be removed or changed to other uses, and the saved computing resources can be used for tasks such as video intelligent analysis.

[0093] To sum up, the multi-screen splicing system of this embodiment has high flexibility, and the splicing device can support a variety of user-defined splicing methods, video stream protocol formats and other options; the video frames stored in the memory can be used in various human body, face, object and behavior recognition application systems through public protocols.

[0094] Embodiment three:

[0095] The present invention discloses an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a multi-image splicing method in any one of the first embodiments of the present invention are implemented.

[0096] Figure 4 is a structural diagram of an electronic device according to an embodiment of the present invention, such as Figure 4 As shown, the electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, near field communication (NFC) or other technologies. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the housing of the electronic device, or an external keyboard, touch pad or mouse, etc.

[0097] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a structural diagram of the part related to the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the technical solution of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0098] Embodiment 4:

[0099] The embodiment of the present invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the multi-image splicing methods in the first embodiment of the present invention are implemented.

[0100] Please note that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. The above embodiments only express several implementation methods of the present application, and their descriptions are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, without departing from the concept of the present application, several variations and improvements can be made, which all belong to the scope of protection of the present application. Therefore, the scope of protection of the patent in this application shall be based on the attached claims.

[0101] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0102] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A multi-image splicing method, characterized in that: The method comprises: Acquire multiple frames of video images in real time; Decoding multiple video frames to obtain video frame content; Storing the video frame content in a memory; Based on the output video requirements input by the user, corresponding video frame contents in the memory are spliced ​​to obtain an output video stream; The output video stream is pushed to the application system.

2. The multi-image splicing method according to claim 1, characterized in that: The method further includes: analyzing the content of the video frame and storing the analysis result in the memory.

3. The multi-image splicing method according to claim 2, characterized in that: The user inputs the output video requirement through output video content options, multi-screen format selection and whether to display analysis results options.

4. The multi-image splicing method according to claim 3, characterized in that: In the case where the display analysis result is determined based on the output video requirement input by the user, the corresponding analysis result in the memory is processed and pushed to the application system together with the output video stream.

5. The multi-image splicing method according to claim 3, characterized in that: The splicing of the corresponding video frame contents in the memory based on the output video requirement input by the user to obtain the output video stream specifically includes: Extracting corresponding video frame content from the memory based on output video requirements input by a user; In the case where it is determined that scaling is required based on the output video requirement input by the user, scaling the video frame content; In the case where a corresponding splicing method is determined based on an output video requirement input by a user, splicing the scaled video frame content; The spliced ​​video frame content is encoded to obtain the output video stream.

6. The multi-image splicing method according to any one of claims 1 to 5, characterized in that: The memory is a distributed shared memory.

7. A multi-image splicing system, characterized in that: The system includes: multiple cameras, at least one decoder, a memory, a splicing device, and a streaming media device; The multiple cameras are used to acquire multiple frames of video images in real time; The decoder is used to decode multiple frames of video images to obtain video frame content; The memory is used to store the video frame content; The splicing device is used to splice the corresponding video frame contents in the memory based on the output video requirements input by the user to obtain an output video stream; The streaming media device is used to push the output video stream to the application system.

8. The multi-image splicing system according to claim 7, characterized in that: The system further comprises at least one analyzer, which is used to analyze the content of the video frame and store the analysis result in the memory.

9. The multi-image splicing method according to claim 8, characterized in that: The system also includes an input terminal, and a user inputs the output video requirement through the output video content option, multi-screen format selection and whether to display the analysis result option set on the input terminal.

10. The multi-image splicing method according to claim 9, characterized in that: The application system is a video analysis system for human body, face, object or behavior recognition.

Images