Multimedia processing method and device, storage medium and electronic product

[0042] The implementation of any technical solution of the embodiments of the present application does not necessarily need to achieve all the above advantages at the same time.

[0043] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the description The embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art shall fall within the protection scope of the embodiments of the present application.

[0044] The specific implementation of the embodiments of the present application will be further described below in conjunction with the drawings of the embodiments of the present application.

[0045] figure 1 This is a schematic flowchart of the multimedia processing method in the first embodiment of this application; figure 1 As shown, it includes:

[0046] S101. Determine multimedia to be processed, where the multimedia includes pictures or video frames;

[0047] In this embodiment, the multimedia to be processed is, for example, a video frame or picture stored locally in a smart phone, or it may be a video frame or picture taken in real time by a user through a camera on the smart phone. It should be noted that the video frames or pictures taken in real-time include the video frames or pictures taken by the camera started by a third-party application on the smartphone.

[0048] In this embodiment, the video frame may be derived from a video segment, or a video frame obtained by performing screen division on a video segment.

[0049] S102. Incorporate a copyright identifier into the multimedia. After the copyright identifier is identified, it can be used to provide an access to the resource address bound to the multimedia.

[0050] In this embodiment, the copyright mark may be a visible copyright mark or a hidden copyright mark. The visible copyright mark is, for example, a two-dimensional code that is visually visible to the user, and the hidden copyright mark is, for example, a mark that is invisible to the user's vision, and can only be extracted by special equipment or through image analysis. Specifically, the hidden copyright mark is at least an embedded copyright mark.

[0051] In this embodiment, on the one hand, the copyright identifier is used to identify the uniqueness of the source of the multimedia to be processed, and on the other hand, it is used to provide access to more resources. For example, if the copyright identification is a webpage QR code, the URL address can be loaded in the webpage QR code, and the user can log in to the URL address and access more resources by scanning the webpage QR code.

[0052] In this embodiment, optionally, step S102 may further include: detecting the subject on the multimedia to determine the location of the non-subject; correspondingly, incorporating the copyright identifier into the multimedia includes: The copyright identification is merged into a non-subject position on the multimedia. In this embodiment, the main body on the multimedia mainly includes people and some subject materials (basketball, cars, etc.). Therefore, when the main body is detected, it is usually detected by face recognition whether there are people in a picture or a certain video frame. ; In addition, by finding extreme points on the spatial scale, and extracting their position, scale, and rotation invariant number to detect and describe local features in multimedia, or directly based on accelerated robust features (Speeded Up Robust Features, referred to as SURF ) To detect and describe local features in multimedia. Further, the edge of the subject is obtained through multi-level edge detection processing.

[0053] Of course, in other embodiments, it can also be added between step S101 and step S102: detecting the subject on the multimedia to determine the position of the non-subject, and correspondingly, in step S102, the copyright identification is merged into The position of the non-main body on the multimedia, thereby preventing the copyright mark from blocking the main body of the multimedia.

[0054] figure 2 This is a schematic flow diagram of the multimedia processing method in the second embodiment of this application; figure 2 As shown, it includes:

[0055] S201. Determine multimedia to be processed, where the multimedia includes pictures or video frames.

[0056] In this embodiment, step S201 is similar to step S101 in the first embodiment. For details, please refer to the first embodiment.

[0057] S202: Extract attribute information of the multimedia, and generate a copyright identifier according to the attribute information;

[0058] In an embodiment of the present application, the multimedia attribute information includes any one or a combination of size, color space, shooting location, and shooting time. Specifically, for example, if the multimedia is a picture, the attribute information includes the size of the picture, the color space of the picture, the shooting location of the picture, and the shooting time of the picture.

[0059] In a specific application scenario, if the multimedia is a picture, the EXIF ​​information of the picture can be extracted, and the EXIF ​​information can be analyzed to obtain the size of the picture, the color space of the picture, and the shooting location of the picture And attribute information such as the shooting time of the picture.

[0060] In this embodiment, if the copyright identification is a two-dimensional code, the above attribute information can be loaded on the two-dimensional code to provide reference information that can prove the only source of the picture from multiple dimensions.

[0061] S203. Incorporate a copyright identifier into the multimedia. After the copyright identifier is identified, it can be used to provide an access to the resource address bound to the multimedia.

[0062] In this embodiment, when merging the copyright mark into the multimedia in step S203, it may include determining the least important pixel area in the multimedia; and then embedding the copyright mark into the least important pixel. Area, which can add hidden copyright signs in multimedia.

[0063] Of course, in other embodiments, a step may be added between step S202 and step S203: determining the least important pixel area in the multimedia, so that when merging in step S203, the copyright mark is embedded To the least important pixel area.

[0064] Further, in this embodiment, the least important pixel area in the multimedia can be determined by specifically determining the positions of the non-subjects, and the pixels corresponding to the positions of these non-subjects are the least important pixels. area.

[0065] Alternatively, in this embodiment, when the copyright mark is incorporated into the multimedia in step S203, it may include performing brightness processing on any selected pixel area in the multimedia so that the average brightness of the multimedia remains unchanged ; And then embed the copyright mark in the arbitrarily selected pixel area, so as to realize the increase of hidden copyright mark in the multimedia.

[0066] Of course, in other embodiments, a step may be added between step S202 and step S203: brightness processing is performed on any selected pixel area in the multimedia, so that when merging in step S203, the The copyright mark is embedded in the arbitrarily selected pixel area.

[0067] Further, in this embodiment, for example, for a certain picture or a video frame, brightness processing is performed on any selected pixel area in the multimedia to realize the embedding of the copyright mark in the arbitrary The selected pixel area can be implemented through the following technical processing procedures:

[0068] Randomly select N pairs of pixels (Ri, Ci) on the picture or video frame, and then add the brightness value of each Ri point to the first set number, and the brightness value of each Ci point is reduced by the second set number, the first The value of the set number and the second set number may be the same or different. When the values ​​of the first set number and the second set number are the same, for example, the two set numbers are specifically 1.

[0069] In addition, alternatively, the picture or video frame may be transformed to the frequency domain through Fourier transform or wavelet transform, and the copyright identification can be added in the frequency domain. Specifically, the technical process for implementing watermarking in the frequency domain is as follows:

[0070] Transform the signals corresponding to the multimedia and copyright identification from time domain to frequency domain to obtain multimedia frequency domain information and copyright identification frequency domain information respectively; synthesize the multimedia frequency domain information and copyright identification frequency domain information; combine the synthesized frequency domain information Perform an inverse transform to transform from the frequency domain to the time domain.

[0071] When transforming the signal corresponding to the copyright mark from the time domain to the frequency domain, exemplarily a random transform may be used; when transforming the signal corresponding to the multimedia from the time domain to the frequency domain, a two-dimensional discrete Fourier transform may be used . Synthesis of multimedia frequency domain information and copyright identification frequency domain information Exemplarily, by introducing an energy function alpha, copyright identification frequency domain information and multimedia frequency domain information can be synthesized to obtain F=F1+alpha f2', F1 represents multimedia frequency domain information , F2' represents frequency domain information of copyright identification, and F represents synthesized frequency domain information. The inverse Fourier transform can be specifically used when inversely transforming the synthesized frequency domain information. image 3 This is a schematic flow chart of the multimedia processing method in the third embodiment of this application; image 3 As shown, it includes:

[0072] S301. Determine multimedia to be processed, where the multimedia includes pictures or video frames.

[0073] In this embodiment, step S301 is similar to step S101 in the first embodiment. For details, please refer to the first embodiment.

[0074] S302: Extract attribute information of the multimedia, and generate a copyright identifier according to the attribute information;

[0075] In this embodiment, step S302 is similar to step S202 described above. For details, please refer to the second embodiment above.

[0076] S303. Generate a multimedia digest according to the attribute information, where the multimedia digest is used to verify the integrity of the multimedia when performing copyright authentication.

[0077] In this embodiment, if the multimedia is a picture, the multimedia summary is specifically a picture summary. When generating the multimedia digest, the MD5 algorithm or the SHA algorithm (Security Hash Algorithm) can be specifically used to implement it.

[0078] S304. Incorporate a copyright identifier into the multimedia. After the copyright identifier is identified, it can be used to provide an access to the resource address bound to the multimedia.

[0079] On the basis of any of the foregoing embodiments, the multimedia processing method further includes: when accessing the resource of the resource address, capturing social actions by accessing the social component in the page.

[0080] Optionally, in an embodiment of the present application, the resource of the resource address includes the complete content of the multimedia, and the multimedia to be processed is a multimedia fragment. Specifically, for example, if the multimedia to which the copyright identification needs to be added is a video frame in a complete video stream, the access can be accessed by accessing the corresponding resource address, so that the complete video stream can be accessed. In order to facilitate the realization of social attributes, social components of social functions can be displayed in the web page, such as like, share, and send private messages.

[0081] The implementation of the multimedia processing method in the above embodiment can directly depend on the smart phone. For example, by installing a third-party application on the smart phone, the user registers on the third-party application, so the multimedia processing is directly tied to the user. Therefore, the logical attribute functions of users, resources, and social interactions are realized.

[0082] Figure 4 This is a schematic diagram of the structure of the multimedia processing device in the fourth embodiment of this application; Figure 4 As shown, it includes:

[0083] The first program unit 401 is used to determine multimedia to be processed, where the multimedia includes pictures or video frames;

[0084] The second program unit 402 is used to incorporate a copyright identifier into the multimedia. After the copyright identifier is identified, it can be used to provide access to the resource address bound to the multimedia.

[0085] Optionally, in an embodiment of the present application, the multimedia processing device further includes: a third program unit (not shown in the figure), configured to generate a copyright identifier according to the attribute information.

[0086] Optionally, in an embodiment of the present application, the third program unit is further configured to generate a visible copyright mark or a hidden copyright mark according to the attribute information.

[0087] Optionally, in an embodiment of the present application, the visible copyright identifier is at least a two-dimensional code, and the hidden copyright identifier is at least an embedded copyright identifier.

[0088] Optionally, in an embodiment of the present application, the multimedia processing device further includes: a fourth program unit (not shown in the figure) for determining the least important pixel area in the multimedia; correspondingly Preferably, the second program unit is further configured to embed the copyright identification into the least important pixel area.

[0089] Optionally, in an embodiment of the present application, the multimedia processing device further includes: a fifth program unit (not shown in the figure), configured to perform brightness on any selected pixel area in the multimedia The processing makes the average brightness of the multimedia remain unchanged; correspondingly, the second program unit is further used to embed the copyright mark in the arbitrarily selected pixel area.

[0090] Optionally, in an embodiment of the present application, the multimedia processing device further includes: a sixth program unit (not shown in the figure), configured to detect the subject on the multimedia to determine the non-subject Location; Correspondingly, the second program unit is further used to merge the copyright identification into a non-subject location on the multimedia.

[0091] Optionally, in an embodiment of the present application, the multimedia attribute information includes any one or a combination of size, color space, shooting location, and shooting time.

[0092] Optionally, in an embodiment of the present application, the multimedia processing device further includes: a sixth program unit, configured to generate a multimedia abstract according to the attribute information, and the multimedia abstract is used for performing copyright authentication Verify the integrity of the multimedia.

[0093] Optionally, in an embodiment of the present application, the multimedia processing device further includes: a seventh program unit (not shown in the figure), which is used to capture by accessing the page when accessing the resource of the resource address Social action.

[0094] In the above embodiment, the expressions such as the first program unit, the second program unit, the third program unit, the fourth program unit, the fifth program unit, the sixth program unit, the seventh program unit, etc. are not limited to the number. In fact, These program units can be reused with each other.

[0095] In addition, the above-mentioned first program unit, second program unit, third program unit, fourth program unit, fifth program unit, sixth program unit, and seventh program unit may be based on a distributed structure, for example, some program units are located at the front end, Some program units are located in the background.

[0096] An embodiment of the present application also provides a storage medium that stores executable instructions, and the executable instructions implement the following steps when executed by a processor:

[0097] Determine the multimedia to be processed, the multimedia including pictures or video frames;

[0098] The copyright identifier is incorporated into the multimedia, and the copyright identifier can be used to provide access to the resource address bound to the multimedia after being identified.

[0099] An embodiment of the present application also provides an electronic product, including a storage medium and a processor, the storage medium stores executable instructions, and the executable instructions implement the following steps when executed by the processor:

[0100] Determine the multimedia to be processed, the multimedia including pictures or video frames;

[0101] The copyright identifier is incorporated into the multimedia, and the copyright identifier can be used to provide access to the resource address bound to the multimedia after being identified.

[0102] Optionally, the steps implemented by the foregoing executable instructions when executed by the processor are included in any of the foregoing method embodiments.

[0103] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in One place, or it can be distributed to multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement without creative work.

[0104] Through the description of the above implementation manners, those skilled in the art can clearly understand that each implementation manner can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions can be embodied in the form of software products, which can be stored in a computer-readable storage medium, and the computer-readable record The medium includes any mechanism for storing or transmitting information in a form readable by a computer (eg, a computer). For example, machine-readable media include read-only memory (ROM), random access memory (RAM), disk storage media, optical storage media, flash storage media, electrical, optical, acoustic, or other forms of propagated signals (for example, carrier waves) , Infrared signal, digital signal, etc.), the computer software product includes a number of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the various embodiments or some parts of the embodiments Methods.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: It is still possible to modify the technical solutions recorded in the foregoing embodiments, or equivalently replace some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and spirit of the technical solutions of the embodiments of the application. range.

[0106] Those skilled in the art should understand that the embodiments of the embodiments of the present application may be provided as methods, devices (equipment), or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0107] The embodiments of this application are described with reference to the flowcharts and/or block diagrams of the methods, devices (equipment) and computer program products according to the embodiments of this application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated for use In the process Figure one Process or multiple processes and/or boxes Figure one A device with functions specified in a block or multiple blocks.

[0108] These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device is implemented in the process Figure one Process or multiple processes and/or boxes Figure one Functions specified in a box or multiple boxes.

[0109] These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. Instructions are provided to implement the process Figure one Process or multiple processes and/or boxes Figure one Steps of functions specified in a box or multiple boxes.