Method and related equipment for video recording during picture preview and image analysis

Abstract

The embodiment of the invention belongs to the technical field of image processing in artificial intelligence, and relates to a video recording method and related equipment during picture preview and image analysis, and the method comprises the steps: obtaining each frame of collected image data according to picture sampling analysis when a user terminal carries out picture preview and picture sampling analysis; storing each frame of collected image data in a preset image frame data set in sequence; calling a configured codec MediaCodec, and sequentially inputting each frame of image data in the image frame data set into the codec MediaCodec for coding and decoding operation to obtain coding and decoding data; and calling the configured synthesizer MediaMuyer, and carrying out data encapsulation operation on the coding and decoding data according to the synthesizer MediaMuyer to obtain target video data. According to the invention, the picture frame data collected in the camera image analysis process is subjected to transcoding and coding output and is repackaged into the video in the mp4 format, so that the problem that camera preview, image analysis and video recording are carried out at the same time is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology in artificial intelligence, and in particular to a method, apparatus, computer equipment, and storage medium for video recording during screen preview and image analysis. Background Art

[0002] Currently, on Android or Hongmeng OS phones, using the phone's built-in camera for screen preview (Preview), video capture (VideoCapture), and image acquisition and analysis (ImageAnalysis) can be implemented separately, or any combination of two functions can be developed. For example, the following function combination:

[0003] 1. Realize camera screen preview and video recording;

[0004] 2. Realize camera screen preview and simultaneous image analysis;

[0005] 3. Realize video recording while collecting and analyzing camera images;

[0006] However, in some actual use cases, if you need to record a video during screen preview and image analysis, referring to the current official Android developer documentation and some common practices, it is not possible to do so. Therefore, the existing technical solutions cannot provide the function of recording a video during screen preview and image analysis. Summary of the Invention

[0007] The purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for video recording during screen preview and image analysis, so as to solve the problem that the existing technical solutions cannot realize the screen preview, image analysis and video recording functions at the same time.

[0008] In order to solve the above technical problems, the embodiment of the present application provides a method for recording video during screen preview and image analysis, which adopts the following technical solution:

[0009] When the user terminal performs screen preview and picture sampling analysis, each frame of image data collected is obtained according to the picture sampling analysis;

[0010] Storing each frame of the collected image data in sequence into a preset image frame data set;

[0011] Calling the configured codec MediaCodec, and inputting each frame of image data in the image frame data set into the codec MediaCodec in sequence for encoding and decoding operations to obtain encoded and decoded data;

[0012] The configured synthesizer MediaMuxer is called, and a data encapsulation operation is performed on the encoded and decoded data according to the synthesizer MediaMuxer to obtain target video data.

[0013] Furthermore, before the step of calling the configured codec MediaCodec, the following steps are also included:

[0014] Initialize the MediaCodec codec and MediaMuxer synthesizer components;

[0015] Configure the encoding format and bit rate, set the file save location and input and output file streams, and obtain the configured codec MediaCodec and the configured synthesizer MediaMuxer.

[0016] Furthermore, when the user terminal performs screen preview and picture sampling analysis, the step of obtaining each frame of image data collected according to the picture sampling analysis specifically includes the following steps:

[0017] Acquiring original audio data according to the microphone of the user terminal;

[0018] After the step of calling the configured synthesizer MediaMuxer and performing a data encapsulation operation on the encoded and decoded data according to the synthesizer MediaMuxer to obtain target video data, the following steps are also included:

[0019] The target video data and the original audio data are combined to obtain target audio and video data.

[0020] Furthermore, after the step of obtaining the original audio data according to the microphone of the user terminal, the following step is also included:

[0021] The original audio data is denoised according to a preset audio denoising tool.

[0022] Furthermore, after the step of obtaining the original audio data according to the microphone of the user terminal, the following step is also included:

[0023] The original audio data is input into the ByteBuffer of the codec MediaCodec for splitting and processing to obtain PCM audio data.

[0024] In order to solve the above technical problems, the embodiment of the present application further provides a device for recording video during screen preview and image analysis, which adopts the following technical solution:

[0025] An image acquisition module, configured to acquire each frame of image data collected according to the image sampling analysis when the user terminal performs screen preview and image sampling analysis;

[0026] An image storage module, used for sequentially storing each frame of the collected image data into a preset image frame data set;

[0027] The encoding and decoding module is used to call the configured codec MediaCodec and input each frame of image data in the image frame data set into the codec MediaCodec in sequence for encoding and decoding operations to obtain encoded and decoded data;

[0028] The data encapsulation module is used to call the configured synthesizer MediaMuxer and perform a data encapsulation operation on the encoded and decoded data according to the synthesizer MediaMuxer to obtain target video data.

[0029] Furthermore, the device further comprises:

[0030] An initialization module is used to initialize the MediaCodec codec and the MediaMuxer synthesizer components;

[0031] The configuration module is used to configure the encoding format and bit rate, set the file save location and input and output file streams, and obtain the configured codec MediaCodec and the configured synthesizer MediaMuxer.

[0032] Furthermore, the image acquisition module includes:

[0033] An audio acquisition submodule, configured to acquire original audio data from a microphone of the user terminal;

[0034] The device further comprises:

[0035] The merging module is used to merge the target video data and the original audio data to obtain target audio and video data.

[0036] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0037] The system comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for performing video recording during screen preview and image analysis as described above are implemented.

[0038] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0039] The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for performing video recording during screen preview and image analysis as described above.

[0040] The present application provides a method for recording video during screen preview and image analysis, comprising: when a user terminal performs screen preview and image sampling analysis, obtaining each frame of image data collected according to the image sampling analysis; storing each frame of image data collected in sequence into a preset image frame data set; calling a configured codec MediaCodec, and inputting each frame of image data in the image frame data set into the codec MediaCodec in sequence for encoding and decoding operations to obtain encoded and decoded data; calling a configured synthesizer MediaMuxer, and performing a data encapsulation operation on the encoded and decoded data according to the synthesizer MediaMuxer to obtain target video data. Compared with the existing technology, the present application transcodes, encodes, and re-encapsulates the image frame data collected during the camera image analysis process into an MP4 format video, thereby achieving the effect of indirect video recording and solving the problem of simultaneous camera preview, image analysis, and video recording. Since it adopts a GPU hard encoding and decoding method, it is highly efficient and occupies less system resources, and has little impact on the performance of Android Hongmeng devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0043] Figure 2 This is a flowchart of the method for recording video during screen preview and image analysis provided in Example 1 of the present application;

[0044] Figure 3 Schematic diagram of the structure of the device for recording video during screen preview and image analysis provided in the second embodiment of the present application;

[0045] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0047] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0048] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0050] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0051] Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0052] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0053] It should be noted that the method for performing video recording during screen preview and image analysis provided in the embodiment of the present application is generally executed by a server / terminal device. Accordingly, the device for performing video recording during screen preview and image analysis is generally set in the server / terminal device.

[0054] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0055] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for recording a video during screen preview and image analysis according to the present application. The method for recording a video during screen preview and image analysis includes: step S201, step S202, step S203, step S204, step S205, step S206, and step S207.

[0056] In step S201, when the user terminal performs screen preview and picture sampling analysis, each frame of image data collected is obtained according to the above picture sampling analysis.

[0057] In the embodiment of the present application, it is necessary to create an Activity page, write relevant layout and function codes to implement the preview of the camera screen and the callback listener ImageAnalysis.Analyzer() of image analysis. After the page is opened, the screen preview and image sampling analysis will be performed at the same time.

[0058] In step S202, each frame of image data collected is sequentially stored in a preset image frame data set.

[0059] In the embodiment of the present application, the image frame data collected in the previous image analysis callback method is stored in an image frame data set (eg, LinkedList<>) of a suitable size.

[0060] In step S203 , the configured codec MediaCodec is called, and each frame of image data in the above image frame data set is sequentially input into the above codec MediaCodec for encoding and decoding operations to obtain encoded and decoded data.

[0061] In the embodiments of this application, MediaCodec is an API for audio and video encoding and decoding on the Android platform. It allows developers to encode and decode multimedia data on Android devices and is suitable for processing audio and video files. MediaCodec includes encoding and decoding. Encoding is to compress the original audio or video data into a format that can be transmitted or stored. Decoding is to restore the encoded data to the original data for playback or further processing.

[0062] In an embodiment of the present application, it is necessary to start an independent thread to take out the frame data from the image frame data set in the previous step in a first-in-first-out order, determine whether to transcode the format according to needs, and input it into MediaCodec for encoding and decoding MediaCodec.queueInputBuffer(). After the encoding and decoding is completed, the output data MediaCodec.getOutputBuffer() is obtained.

[0063] In step S204, the configured synthesizer MediaMuxer is called, and a data encapsulation operation is performed on the encoded and decoded data according to the synthesizer MediaMuxer to obtain target video data.

[0064] In an embodiment of the present application, the synthesizer MediaMuxer is a class on the Android platform for synthesizing audio and video data streams into a multimedia file, typically a video file (such as MP4) or an audio file (such as AAC). MediaMuxer allows encoded audio and video streams to be merged into a complete multimedia file, which is very useful for tasks such as creating video editing applications, recording screens, and merging live streams. The synthesizer MediaMuxer refers to the merging of multimedia streams, that is, MediaMuxer allows one or more audio streams and video streams to be merged into a multimedia file. You can add encoded audio and video data to MediaMuxer and specify their timestamps to ensure that they are correctly aligned in time.

[0065] In an embodiment of the present application, after obtaining the output data MediaCodec.getOutputBuffer() in the previous step, MediaMuxer.writeSampleData() is called to encapsulate the ByteBuffer data into mp4 format, that is, the target video data, and output it to the local storage device.

[0066] In an embodiment of the present application, a method for recording video during screen preview and image analysis is provided, comprising: when a user terminal performs screen preview and image sampling analysis, obtaining each frame of image data collected according to the above image sampling analysis; storing each frame of image data collected in sequence into a preset image frame data set; calling a configured codec MediaCodec, and inputting each frame of image data in the above image frame data set into the above codec MediaCodec for encoding and decoding operations to obtain encoded and decoded data; calling a configured synthesizer MediaMuxer, and performing data encapsulation operations on the above encoded and decoded data according to the above synthesizer MediaMuxer to obtain target video data. Compared with the existing technology, the present application transcodes, encodes, and re-encapsulates the image frame data collected during the camera image analysis process into an MP4 format video, thereby achieving the effect of indirect video recording and solving the problem of simultaneous camera preview, image analysis, and video recording. Since it adopts a GPU hard encoding and decoding method, it is highly efficient and occupies less system resources, and has little impact on the performance of Android Hongmeng devices.

[0067] In some optional implementations of this embodiment, before step S203, the following steps are further included:

[0068] Initialize the MediaCodec codec and MediaMuxer components.

[0069] Configure the encoding format and bit rate, set the file save location and input and output file streams, and obtain the configured codec MediaCodec and the configured synthesizer MediaMuxer.

[0070] In some optional implementations of this embodiment, the above step S201 specifically includes the following steps: obtaining original audio data according to the microphone of the above user terminal;

[0071] After step S204, the following step is also included: the target video data and the original audio data are merged to obtain target audio and video data.

[0072] In the embodiment of the present application, the confluence operation can be performed using the MediaMuxer API provided by the system, which provides the ability to encapsulate audio and video data into an MP4 container; by calling the addTrack(@NonNullMediaFormat format) function of MediaMuxer, an audio or video track is added and the corresponding ID is returned. Data can then be written to the corresponding track using this ID; after adding one audio track and one video track, the start() function is called to start writing data to the Muxer; then the writeSampleData function is called to write the encoded audio and video data to the Muxer; finally, the original audio data and image metadata are encapsulated and output to the final MP4 file.

[0073] In some optional implementations of this embodiment, after the above step of obtaining the original audio data according to the microphone of the user terminal, the following step is further included:

[0074] The above original audio data is denoised according to a preset audio denoising tool.

[0075] In the embodiments of this application, audio denoising is a signal processing technique used to reduce or eliminate background noise, hum, or interfering sounds in recordings or audio files to improve audio quality. The goal of denoising is to make the sound clearer and easier to understand, and to eliminate interfering sounds to improve the listening experience.

[0076] In an embodiment of the present application, the preset audio denoising tool may be Adobe Audition, Audacity, iZotope RX, etc. It should be understood that the examples of the preset audio denoising tools here are only for ease of understanding and are not used to limit this application.

[0077] In some optional implementations of this embodiment, after the above step of obtaining the original audio data according to the microphone of the user terminal, the following step is further included:

[0078] The original audio data is input into the ByteBuffer of the codec MediaCodec for splitting and processing to obtain PCM audio data.

[0079] In an embodiment of the present application, by inputting the original audio data into ByteBuffer for splitting processing, PCM audio data for speech recognition can be obtained, which facilitates subsequent audio applications.

[0080] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target video data, the above-mentioned target video data can also be stored in a node of a blockchain.

[0081] The blockchain referred to in this application is a new application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of this information (to prevent counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, the platform product service layer, and the application service layer.

[0082] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0083] Basic artificial intelligence technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technology mainly includes several major directions, such as computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning. Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0084] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0085] Example 2

[0086] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for recording video during picture preview and image analysis. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0087] like Figure 3 As shown, the apparatus 200 for recording video during screen preview and image analysis in this embodiment includes: an image acquisition module 210, an image storage module 220, a codec module 230, and a data encapsulation module 240. Among them:

[0088] The image acquisition module 210 is used to obtain each frame of image data collected according to the image sampling analysis when the user terminal performs screen preview and image sampling analysis;

[0089] The image storage module 220 is used to sequentially store each frame of the collected image data into a preset image frame data set;

[0090] The encoding and decoding module 230 is used to call the configured codec MediaCodec and input each frame of image data in the above image frame data set into the above codec MediaCodec in sequence for encoding and decoding operations to obtain encoded and decoded data;

[0091] The data encapsulation module 240 is used to call the configured synthesizer MediaMuxer and perform a data encapsulation operation on the above-mentioned encoded and decoded data according to the above-mentioned synthesizer MediaMuxer to obtain target video data.

[0092] In the embodiment of the present application, it is necessary to create an Activity page, write relevant layout and function codes to implement the preview of the camera screen and the callback listener ImageAnalysis.Analyzer() of image analysis. After the page is opened, the screen preview and image sampling analysis will be performed at the same time.

[0093] In the embodiment of the present application, the image frame data collected in the previous image analysis callback method is stored in an image frame data set (eg, LinkedList<>) of a suitable size.

[0094] In the embodiments of this application, MediaCodec is an API for audio and video encoding and decoding on the Android platform. It allows developers to encode and decode multimedia data on Android devices and is suitable for processing audio and video files. MediaCodec includes encoding and decoding. Encoding is to compress the original audio or video data into a format that can be transmitted or stored. Decoding is to restore the encoded data to the original data for playback or further processing.

[0095] In an embodiment of the present application, it is necessary to start an independent thread to take out the frame data from the image frame data set in the previous step in a first-in-first-out order, determine whether to transcode the format according to needs, and input it into MediaCodec for encoding and decoding MediaCodec.queueInputBuffer(). After the encoding and decoding is completed, the output data MediaCodec.getOutputBuffer() is obtained.

[0096] In an embodiment of the present application, the synthesizer MediaMuxer is a class on the Android platform for synthesizing audio and video data streams into a multimedia file, typically a video file (such as MP4) or an audio file (such as AAC). MediaMuxer allows encoded audio and video streams to be merged into a complete multimedia file, which is very useful for tasks such as creating video editing applications, recording screens, and merging live streams. The synthesizer MediaMuxer refers to the merging of multimedia streams, that is, MediaMuxer allows one or more audio streams and video streams to be merged into a multimedia file. You can add encoded audio and video data to MediaMuxer and specify their timestamps to ensure that they are correctly aligned in time.

[0097] In an embodiment of the present application, after obtaining the output data MediaCodec.getOutputBuffer() in the previous step, MediaMuxer.writeSampleData() is called to encapsulate the ByteBuffer data into mp4 format, that is, the target video data, and output it to the local storage device.

[0098] In this embodiment, a device 200 for recording video during screen preview and image analysis is provided, including: an image acquisition module 210, which is used to obtain each frame of image data collected according to the above-mentioned picture sampling analysis when the user terminal performs screen preview and picture sampling analysis; an image storage module 220, which is used to store each frame of image data collected in the above-mentioned image frame data set in sequence; a codec module 230, which is used to call the configured codec MediaCodec, and input each frame of image data in the above-mentioned image frame data set in sequence into the above-mentioned codec MediaCodec for encoding and decoding operations to obtain encoded and decoded data; a data encapsulation module 240, which is used to call the configured synthesizer MediaMuxer, and perform data encapsulation operations on the above-mentioned encoded and decoded data according to the above-mentioned synthesizer MediaMuxer to obtain target video data. Compared with the existing technology, this application transcodes, encodes and outputs the picture frame data collected during the camera image analysis process, and re-encapsulates it into MP4 format video. In this way, the effect of indirect video recording is achieved, and the difficult problem of camera preview, image analysis and video recording are solved at the same time. Since it adopts GPU hard encoding and decoding, it is highly efficient and takes up less system resources, and has little impact on the performance of Android Hongmeng devices.

[0099] In some optional implementations of this embodiment, the apparatus 200 for recording video during screen preview and image analysis further includes: an initialization module and a configuration module, wherein:

[0100] Initialization module, used to initialize the above codec MediaCodec and synthesizer MediaMuxer components;

[0101] The configuration module is used to configure the encoding format and bit rate, set the file save location and input and output file streams, and obtain the above-configured codec MediaCodec and the above-configured synthesizer MediaMuxer.

[0102] In some optional implementations of this embodiment, the image acquisition module includes: an audio acquisition submodule, configured to acquire original audio data according to the microphone of the user terminal;

[0103] The above-mentioned device also includes: a merging module, which is used to merge the above-mentioned target video data and the above-mentioned original audio data to obtain target audio and video data.

[0104] In the embodiment of the present application, the confluence operation can be performed using the MediaMuxer API provided by the system, which provides the ability to encapsulate audio and video data into an MP4 container; by calling the addTrack(@NonNullMediaFormat format) function of MediaMuxer, an audio or video track is added and the corresponding ID is returned. Data can then be written to the corresponding track using this ID; after adding one audio track and one video track, the start() function is called to start writing data to the Muxer; then the writeSampleData function is called to write the encoded audio and video data to the Muxer; finally, the original audio data and image metadata are encapsulated and output to the final MP4 file.

[0105] In some optional implementations of this embodiment, the audio acquisition submodule further includes: a denoising unit, configured to perform denoising processing on the original audio data according to a preset audio denoising tool.

[0106] In the embodiments of this application, audio denoising is a signal processing technique used to reduce or eliminate background noise, hum, or interfering sounds in recordings or audio files to improve audio quality. The goal of denoising is to make the sound clearer and easier to understand, and to eliminate interfering sounds to improve the listening experience.

[0107] In an embodiment of the present application, the preset audio denoising tool may be Adobe Audition, Audacity, iZotope RX, etc. It should be understood that the examples of the preset audio denoising tools here are only for ease of understanding and are not used to limit this application.

[0108] In some optional implementations of this embodiment, the audio acquisition submodule further includes: an audio splitting unit, configured to input the original audio data into the ByteBuffer of the codec MediaCodec for splitting processing to obtain PCM audio data.

[0109] In an embodiment of the present application, by inputting the original audio data into ByteBuffer for splitting processing, PCM audio data for speech recognition can be obtained, which facilitates subsequent audio applications.

[0110] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0111] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected through a system bus. It should be noted that the figure only shows the computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0112] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0113] The memory 310 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, optical disk, etc. In some embodiments, the memory 310 may be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 may also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory 310 may also include both the internal storage unit of the computer device 300 and its external storage device. In this embodiment, the memory 310 is typically used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for methods of recording video during screen preview and image analysis. In addition, the memory 310 can also be used to temporarily store various types of data that have been output or will be output.

[0114] In some embodiments, the processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 320 is generally used to control the overall operation of the computer device 300. In this embodiment, the processor 320 is used to execute computer-readable instructions or process data stored in the memory 310, such as computer-readable instructions for executing the method for performing video recording during screen preview and image analysis.

[0115] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0116] The computer device provided in this application transcodes, encodes and outputs the picture frame data collected during the camera image analysis process, and repackages it into an MP4 format video. In this way, the effect of indirect video recording is achieved, and the difficult problem of camera preview, image analysis and video recording are solved simultaneously. Since it adopts GPU hard encoding and decoding, it is highly efficient and takes up little system resources, and has little impact on the performance of Android Hongmeng devices.

[0117] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the method for video recording during screen preview and image analysis as described above.

[0118] The computer-readable storage medium provided in this application transcodes, encodes and outputs the picture frame data collected during the camera image analysis process, and repackages it into an MP4 format video. In this way, the effect of indirect video recording is achieved, and the difficult problem of camera preview, image analysis and video recording are solved simultaneously. Since it adopts GPU hard encoding and decoding, it is highly efficient and takes up little system resources, and has little impact on the performance of Android Hongmeng devices.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0120] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A method for video recording during screen preview and image analysis, characterized in that: The steps include: When the user terminal performs screen preview and picture sampling analysis, each frame of image data collected is obtained according to the picture sampling analysis; The collected image data is sequentially stored in a preset image frame data set; Calling the configured codec MediaCodec, and inputting each frame of image data in the image frame data set into the codec MediaCodec in sequence for encoding and decoding operations to obtain encoded and decoded data; The configured synthesizer MediaMuxer is called, and a data encapsulation operation is performed on the encoded and decoded data according to the synthesizer MediaMuxer to obtain target video data.

2. The method for video recording during screen preview and image analysis according to claim 1, characterized in that: Before the step of calling the configured codec MediaCodec, the following steps are also included: Initialize the MediaCodec codec and the MediaMuxer synthesizer components; Configure the encoding format and bit rate, set the file save location and input and output file streams, and obtain the configured codec MediaCodec and the configured synthesizer MediaMuxer.

3. The method for video recording during screen preview and image analysis according to claim 1, characterized in that: When the user terminal performs screen preview and picture sampling analysis, the step of acquiring each frame of image data collected according to the picture sampling analysis specifically includes the following steps: Acquiring original audio data according to the microphone of the user terminal; After the step of calling the configured synthesizer MediaMuxer and performing a data encapsulation operation on the encoded and decoded data according to the synthesizer MediaMuxer to obtain the target video data, the following steps are also included: The target video data and the original audio data are combined to obtain target audio and video data.

4. The method for video recording during screen preview and image analysis according to claim 3, characterized in that: After the step of obtaining the original audio data according to the microphone of the user terminal, the following steps are also included: The original audio data is denoised according to a preset audio denoising tool.

5. The method for video recording during screen preview and image analysis according to claim 3, characterized in that: After the step of obtaining the original audio data according to the microphone of the user terminal, the following steps are also included: The original audio data is input into the ByteBuffer of the codec MediaCodec for splitting and processing to obtain pcm audio data.

6. A device for recording video during picture preview and image analysis, characterized in that: include: An image acquisition module, used to acquire each frame of image data acquired according to the image sampling analysis when the user terminal performs screen preview and image sampling analysis; An image storage module, used for sequentially storing each frame of the collected image data into a preset image frame data set; The encoding and decoding module is used to call the configured codec MediaCodec, and input each frame of image data in the image frame data set into the codec MediaCodec in sequence for encoding and decoding operations to obtain encoded and decoded data; The data encapsulation module is used to call the configured synthesizer MediaMuxer, and perform a data encapsulation operation on the encoded and decoded data according to the synthesizer MediaMuxer to obtain target video data.

7. The device for recording video during screen preview and image analysis according to claim 6, characterized in that: The device also includes: An initialization module, used to initialize the codec MediaCodec and the synthesizer MediaMuxer components; The configuration module is used to configure the encoding format and bit rate, and set the file save location and input and output file streams to obtain the configured codec MediaCodec and the configured synthesizer MediaMuxer.

8. The device for recording video during screen preview and image analysis according to claim 6, characterized in that: The image acquisition module comprises: An audio acquisition submodule, used to obtain original audio data according to the microphone of the user terminal; The device also includes: The confluence module is used to perform a confluence operation on the target video data and the original audio data to obtain target audio and video data.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for performing video recording during screen preview and image analysis as described in any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for performing video recording during picture preview and image analysis as described in any one of claims 1 to 5.

Images