Image data processing method, device, equipment and program product
By directly reading image data using the address cache area in image data processing, the memory and CPU burden problems caused by multiple data replication in the prior art are solved, and processing efficiency is improved.
Patent Information
- Application Number
- CN202411997310.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-04-11
AI Technical Summary
The prior art requires multiple data replications during image data processing, resulting in increased memory resource usage and increased CPU computing burden, affecting overall efficiency.
The frame receiving thread obtains the first address of the image data to be processed and sends it to the first address cache area. The image processing thread obtains the address from the cache area and directly reads and processes the image data to avoid data copying.
It reduces the use of memory resources, reduces the computing workload of CPU, and improves the overall efficiency of image data processing.
Smart Images
Figure CN119814955A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing, and in particular to image data processing methods, devices, equipment and program products. Background Art
[0002] In the field of image processing such as video surveillance, it is usually necessary to process image data in real time. For example, in a real-time monitoring system, after the camera captures the video image, it is generally stored in the physical memory in YUV format. When the image is detected and recognized, the YUV data (image data in YUV format) in the physical memory is usually copied to the cache, and then the YUV data is copied from the cache for image detection and recognition, such as processing by a neural network engine.
[0003] Since the image data processing process requires multiple data replications, it not only consumes precious memory resources, but also increases the computational burden of the CPU, affecting the overall efficiency of image data processing. Summary of the invention
[0004] In view of this, embodiments of the present application provide methods, devices, equipment and program products to solve the problems in the prior art.
[0005] A first aspect of an embodiment of the present application provides an image data processing method, the method comprising:
[0006] Acquire the image data to be processed through the frame receiving thread, and determine a first address for storing the image data to be processed of a single frame;
[0007] Sending the first address of the image data to be processed to the first address buffer area through the frame receiving thread;
[0008] The first address of the image data to be processed is obtained from the first address buffer area through the image processing thread, and the image data to be processed is read and processed through the first address.
[0009] In combination with the first aspect, in a first possible implementation manner of the first aspect, reading and processing the to-be-processed image data through the first address includes:
[0010] Reading the image data to be processed according to the first address through the image processing thread, processing the image data to be processed according to a predetermined image processing method, determining the image data to be encoded, sending the storage address of the image data to be encoded to the second address buffer area, and then obtaining the address of the next frame of image data to be processed from the first address buffer area for processing;
[0011] The second address of the image data to be encoded is asynchronously obtained from the second address buffer area through the encoding thread, and the image data to be encoded is encoded according to the second address.
[0012] In combination with the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, processing the image data to be processed according to a predetermined image processing method to determine the image data to be encoded includes:
[0013] Performing detection processing on the image data to be processed by the image processing thread to determine whether the image data to be processed includes an object of interest;
[0014] When it is detected that the image data to be processed includes an object of interest, the object of interest in the image data to be processed is cropped to obtain cropped image data, the cropped image data is identified, and the cropped image data or the image data to be processed is determined to be encoded image data.
[0015] In combination with the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, after detecting and processing the image data to be processed by the image processing thread to determine whether the image data to be processed includes an object of interest, the method further includes:
[0016] When it is detected that the image data to be processed does not include an object of interest, the cache space occupied by the image data to be processed is released by the image processing thread, and then the address of the next frame of image data to be processed is obtained from the first address cache area for detection.
[0017] In combination with the first possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the image data to be processed is image data in a YUV format;
[0018] The method further comprises: performing encoding processing on the image data to be encoded according to the second address, comprising:
[0019] Reading the image data to be encoded in YUV format according to the second address through the encoding thread;
[0020] The encoded image data in the YUV format is compressed and encoded by the encoding thread to obtain compressed image data, and then the cache space occupied by the image data to be processed is released.
[0021] In combination with the first aspect, in a fifth possible implementation manner of the first aspect, sending the first address of the to-be-processed image data to the first address buffer area through a frame receiving thread includes:
[0022] Sending the first address of the image data to be processed to the first address buffer area through the frame receiving thread, and determining the sending result of the first address;
[0023] When the sending result is successful, executing the step of sending the storage address of the next frame of image data to be processed to the first address buffer area;
[0024] When the sending result is a sending failure, the storage space occupied by the image data to be processed is released.
[0025] In combination with any one of the first aspect to the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, before sending the first address of the to-be-processed image data to the first address buffer area through the frame receiving thread, the method further includes:
[0026] The frame receiving thread encodes the to-be-processed image data received in sequence according to a video data encoding method to obtain video data in a video encoding format.
[0027] A second aspect of an embodiment of the present application provides an image data processing device, the device comprising:
[0028] A data acquisition unit, used for acquiring the image data to be processed through a frame receiving thread, and determining a first address for storing the image data to be processed in a single frame;
[0029] A storage address sending unit, used for sending the first address of the image data to be processed to the first address buffer area through a frame receiving thread;
[0030] The image data processing unit is used to obtain the first address of the image data to be processed from the first address buffer area through the image processing thread, and read and process the image data to be processed through the first address.
[0031] A third aspect of an embodiment of the present application provides an image data processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image data processing device implements a method as described in any one of the first aspects.
[0032] A fourth aspect of the embodiments of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method in the first aspect or its various implementations.
[0033] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.
[0034] The sixth aspect of the embodiment of the present application provides a chip for implementing the methods in each implementation of the first aspect. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the method in the first aspect or its implementation.
[0035] Compared with the prior art, the embodiments of the present application have the following beneficial effects: for the received image data to be processed, the embodiments of the present application first determine the first address for storing the image data to be processed, and send the first address to the first address cache area for caching. When the image processing thread processes the image to be processed, it only needs to read the image to be processed through the first address, and does not need to cache the data to be processed to the first cache area, thereby reducing the occupation of memory resources. Data can be read directly through the first address, which effectively reduces the computing workload of the CPU and is conducive to improving the overall processing efficiency of image data. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0037] Figure 1 It is a schematic diagram of an implementation scenario of an image data processing method provided in an embodiment of the present application;
[0038] Figure 2 It is a schematic diagram of an implementation flow of an image data processing method provided in an embodiment of the present application;
[0039] Figure 3 This is a schematic diagram of a ring buffer area caching image data to be processed provided by an embodiment of the present application;
[0040] Figure 4 This is a schematic diagram of an implementation flow of an address reference provided in an embodiment of the present application;
[0041] Figure 5 This is a schematic diagram of an asynchronous processing flow provided by an embodiment of the present application;
[0042] Figure 6 A schematic diagram of an image data processing device provided in an embodiment of the present application;
[0043] Figure 7 It is a schematic diagram of an image data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0044] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0045] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.
[0046] In the field of image processing such as video surveillance, real-time processing of image data is crucial. The video stream captured by the camera is usually stored directly in the physical memory in YUV format. In order to perform subsequent image detection and recognition processing, these YUV data often need to be copied to the cache. Subsequently, the data is copied from the cache to the processing module again to perform analysis and recognition tasks.
[0047] However, in this process, data replication requires additional memory space, which is particularly significant in high-resolution video stream processing. In addition, frequent data replication requires the CPU to perform a large number of data transfer operations, which not only takes up CPU cycles but may also cause cache consistency issues. Multiple data replications extend the total data processing time, especially in real-time monitoring systems that require fast response, which may increase latency and affect system performance.
[0048] In order to solve the above problems, the present application proposes an image data processing method. Figure 1The present invention is a schematic diagram of one implementation scenario of the method. In the implementation scenario, an image data acquisition device and an image data processing device are included. The image data acquisition device is used to acquire image data to be processed. The image data to be processed is usually raw image data, such as image data in YUV format. The image data acquisition device can be a camera, including a security monitoring camera, a traffic law enforcement camera, an industrial automation camera, etc. The image data processing device is used to process the data acquired by the image acquisition device, including detection processing and / or recognition processing. The image data processing device and the image data acquisition device can be independent physical devices, or they can also be integrated into one device. During operation, the image data processing device can include a frame receiving thread, an image processing thread, or it can also include an encoding thread. The frame receiving thread is used to obtain image data to be processed, and the image data to be processed can be cached in a circular buffer area to improve the utilization efficiency of the buffer area. Determine the first address of the image data to be processed of a single frame, and send the first address to the first address buffer area. The image processing thread can read the image data to be processed from the first address of the image data to be processed in the first address buffer area based on the first address. After the image data to be processed is processed, the image data to be processed cached in the physical memory is released, thereby improving the reliability of data processing and reducing memory usage by blocking. In order to reduce the image processing time occupied by the encoding thread during the encoding process, a second address buffer area can be opened, and the second address of the image data to be encoded determined by the image processing thread can be stored in the second address buffer area, so that the encoding thread can asynchronously process the image data to be encoded, thereby effectively improving the rotation efficiency of the first address buffer area and improving the frame rate of image data processing.
[0049] Figure 2 A schematic diagram of an implementation flow of an image data processing method provided in an embodiment of the present application is described in detail as follows:
[0050] In S201, the image data to be processed is acquired through a frame receiving thread, and a first address for storing a single frame of the image data to be processed is determined.
[0051] The image data processing device in the embodiment of the present application may be a real-time image data processing device or a non-real-time image data processing device. When the image data processing device is a real-time image data processing device, when a single frame of image data is received in real time, the frame of image data may be used as the image data to be processed, and the first address of the frame of image data to be processed may be determined. When the image data processing device is a non-real-time image data processing device, single frames of image data may be sequentially received from the stored multiple frames of image data as the image data to be processed.
[0052] The image data to be processed received by the frame receiving thread can be cached in the physical memory. In order to facilitate the management of the image data to be processed, the physical memory can set a ring buffer (RINGBUF). Figure 3 A schematic diagram of a ring buffer area for caching image data to be processed provided in an embodiment of the present application. The number of cache elements included in the ring buffer area can be preset. For example, Figure 3 The number of cache elements set in is 5, namely YUV1, YUV2, YUV3, YUV4 and YUV5. Five image data to be processed (i.e., data of five frames of images) can be cached in sequence, so that the image processing thread can process the image data through the cached image data to be processed. Figure 3 As shown, after sending the image data to be processed to the cache elements YUV1, YUV2, YUV3, YUV4 and YUV5 in sequence, the image data to be processed is cached in YUV1, YUV2, YUV3, YUV4 and YUV5 respectively, and the addresses of the image data to be processed are respectively address 1, address 2, address 3, address 4 and address 5. If new image data to be processed is received, when the newly received image data to be processed is sent to the circular buffer area, the earliest cached cache element, such as YUV1, can be detected to see whether the cached image data to be processed is released. If YUV1 has been released, it can be sent successfully, otherwise the sending is unsuccessful, indicating that the circular buffer area has no free cache space to cache the new image data to be processed, and the application layer does not occupy this frame of image data, so space can be freed up to facilitate the SDK to put in new image data, and the storage space occupied by the frame of image data to be processed can be released. The image processing thread can read the cached image data to be processed in the storage space according to the first address of the image data to be processed, including address 1, address 2, address 3, address 4 or address 5. Among them, the release of cache elements is related to the processing process in the image processing thread. After completing the detection and / or recognition processing of the image data to be processed, including detecting that the image does not include the target of interest, or detecting that the image includes the target of interest, and encoding the image data where the target of interest is located, the cache space of the cache element can be released.
[0053] In S202, the first address of the image data to be processed is sent to the first address buffer area through the frame receiving thread.
[0054] In a possible implementation, before the frame receiving thread sends the first address of the image data to be processed to the first address buffer area, the image data to be processed sequentially received by the frame receiving thread can also be encoded according to the video data encoding method to obtain video data in a video encoding format. For example, image data in YUV format can be encoded into image data in H.265 format, so that when a target of interest is detected according to the detection algorithm, or when a specific target is identified through the recognition algorithm, the video data of a specific time period can be read from the video data for easy playback or downloading for playback.
[0055] When sending the first address to the first address buffer, the frame receiving thread can detect whether the first address is sent (POST) successfully. If the first address is sent successfully, that is, the first address is cached in the cache element in the first address buffer, it means that the current image data to be processed has been cached successfully, and the frame receiving processing of the next frame of image data to be processed can be executed, and the storage address of the next frame of image data to be processed is stored in the first address buffer. If the first address fails to be sent, it means that there is no free cache space in the circular buffer to cache the new address, and the application layer does not occupy this frame of image data. Space can be freed up to facilitate the SDK to put in new image data, and the storage space occupied by the frame of image data to be processed can be released.
[0056] Figure 4 A schematic diagram of an implementation flow of address reference provided in an embodiment of the present application is described in detail as follows:
[0057] In S401, image data to be processed is obtained.
[0058] The image data to be processed may be original image data, such as image data in a YUV format. The image data to be processed may be real-time image data or non-real-time image data.
[0059] In S402, video encoding processing is performed on the image data to be processed to obtain image data in a video format.
[0060] For the application of detection and recognition algorithms, the encoded video data can be stored. When an object of interest is detected or a specific object is identified, the video data of the relevant period can be extracted for subsequent analysis and verification. This video data management method not only facilitates the retrieval and use of video content, but also improves the intelligence level of the video surveillance system. For example, the image data to be processed can be encoded in the H.265 format, but it is not limited to this and can also be encoded in other video encoding formats.
[0061] In S403, the first address of the image data to be processed is sent to the first address buffer area.
[0062] The first address of the image data to be processed can be cached in the first address cache area. The embodiment of the present application is different from the traditional data caching method of the ring cache area, and only needs to cache the first address of the storage location of the image data to be processed.
[0063] In S404, it is detected whether the first address is sent successfully.
[0064] If the first address is sent successfully, it can jump to S401 to continue to obtain the next frame of image data to be processed and continue to perform the address reference operation. If the sending fails, it means that there is no free cache space in the first address cache area to cache the new address, and the application layer does not occupy this frame of image data. S405 can be executed to release the storage space occupied by the image data to be processed. By releasing the storage space occupied by the image data to be processed, space can be freed up for the SDK to put in new image data. At the same time, the cache space occupied by the first address can be released.
[0065] When the first address is sent successfully, the first address is cached in the first address cache area. Based on the cached first address, the image processing thread can read the first address from the first address cache area, obtain the image data to be processed according to the first address, and process the image data to be processed by executing the image processing flow.
[0066] In S203, the first address of the image data to be processed is obtained from the first address buffer area through the image processing thread, and the image data to be processed is read and processed through the first address.
[0067] When the image processing thread processes the image data to be processed, there is no need to specially allocate data cache space for the thread. The image processing can be performed on the image data to be processed only through the address information of the first address cache area.
[0068] When the to-be-processed image data is read and processed through the first address, it may include a processing process of an image processing thread and an encoding process of an encoding thread.
[0069] During the processing of the image processing thread, the image data to be processed can be read based on the first address, and the image data to be processed can be processed by a predetermined image processing algorithm, including a detection method and / or a recognition method, to determine the target of interest in the image data to be processed. If the image data to be processed includes the target of interest, the image data to be processed can be determined to be image data to be encoded.
[0070] For example, an image detection method can be used to detect whether the image data to be processed includes a target of interest. For example, the target of interest can be a human face, or it can be a vehicle. When it is determined that the image data to be processed includes a target of interest, the image data to be processed can be further cropped to obtain a cropped image including the target of interest. The cropped image can be further processed for recognition to determine the category or group to which the cropped image belongs. For example, a similarity comparison analysis can be performed between the cropped image and the stored human face, or the license plate number in the cropped image can be identified, the cropped image can be classified, and it can be determined that the image data to be processed is image data to be encoded. Alternatively, the embodiment of the present application can also determine that the image data to be processed is image data to be encoded when it is detected that the image data to be processed includes a target of interest.
[0071] The second address of the image data to be encoded can be sent to the second address buffer. At this time, the address of the next frame of image data to be processed can be obtained through the first address buffer for processing, and the cache space occupied by the first address can be released. The encoding thread can asynchronously obtain the second address in the second address buffer to encode the image data to be encoded, such as encoding the original image data in YUV format into image data in a compressed format such as JPG. The specific asynchronous processing flow can be as follows: Figure 5 As shown, including:
[0072] In S501, the image processing thread obtains image data to be processed through a first address.
[0073] The image processing thread may obtain a first address in the first address buffer area, and read a single frame of to-be-processed image data cached in the physical memory according to the obtained first address, such as original image data in a YUV format.
[0074] In S502, it is detected whether the image data to be processed includes an object of interest.
[0075] In the process of processing the image data to be processed, the embodiment of the present application does not return immediately after the image data to be processed is placed, but occupies the image data to be processed in a blocking manner, and returns after the detection and recognition are completed. That is, the original image data to be processed is occupied in a blocking manner so that the upper layer can reliably perform detection and recognition, avoiding the upper layer from releasing the image data to be processed in advance before the detection and recognition are completed, which can help ensure the correctness of the data processing results and reduce the probability of system crashes.
[0076] The target of interest in the image data to be processed can be detected by an image detection algorithm such as a neural network model, for example, detecting targets of interest such as cars and pedestrians in the image data to be processed. If no target of interest is detected, the storage space occupied by the frame of image data to be processed can be released, cache space can be provided for newly added image data to be processed, and S501 is returned to obtain a new frame of image data to be processed. When the storage space of the image data to be processed in the physical memory is released, the cache space occupied by the first address of the image data to be processed in the first address cache area can be released. If a target of interest is detected, S503 can be executed to crop and identify the target of interest.
[0077] The cropping process can crop the region including the target of interest from the image data to be processed according to the detected position of the target of interest. For example, the cropped image can be obtained according to the detected position of a pedestrian or the detected position of a car.
[0078] For the detected target of interest, the recognition algorithm can be used to perform recognition processing to determine the group or category to which the target of interest belongs. For example, when the target of interest is a pedestrian, the specific features of the pedestrian can be identified, including facial features, clothing features, etc. When the target of interest is a car, the license plate number, vehicle type, etc. of the car can be identified.
[0079] When it is detected that the image data to be processed includes an object of interest, the image data to be processed may be used as image data to be encoded.
[0080] In S504, the second address of the image data to be processed where the object of interest exists is sent to the second address buffer area, and the process returns to S501 to process the new image data to be processed.
[0081] The second address buffer can cache the second address according to the buffer space, so that the encoding thread can asynchronously read the image data to be encoded for encoding. Since the image processing and encoding process uses the second address buffer for asynchronous processing, the frame rate of the processed image data can be effectively improved, and the cache utilization efficiency can be improved.
[0082] For example, the processing time of the image processing thread for a single frame of image data to be processed, that is, the detection time plus the recognition time is 50ms, and the encoding time is 50ms. Then the serial processing and encoding time of the image data to be processed is 100ms, and the next frame of image data to be processed can be processed after 100ms, and the frame rate is 10 at this time. If the two are executed separately and asynchronously, after the image processing process is completed, that is, after the detection and recognition are completed, the new frame of image data to be processed can be processed, and the time to complete the image data to be processed is reduced to 50ms, and the frame rate is increased to 20 at this time, so that the cache space can be effectively used to improve the frame rate of image processing.
[0083] In S505, the data to be encoded is read through the second address.
[0084] The encoding thread can obtain the second address through the second address buffer area, and read the data to be encoded through the second address.
[0085] In S506, the image data to be encoded is compressed to obtain compressed image data.
[0086] The data to be encoded may be original image data, such as image data in YUV format. The image data to be encoded may be compressed to obtain data in image formats such as JPG.
[0087] In S507, the storage space occupied by the image data to be encoded is released.
[0088] After the compression encoding of the image data to be encoded is completed, the storage space occupied by the image data to be encoded can be released, thereby providing cache space for the newly added image data to be processed.
[0089] The embodiment of the present application caches the address of the image data to be processed to the first address cache area of the image processing thread, so that the image processing thread can read the image to be processed through the first address, and there is no need to cache the data to be processed in the first cache area, thereby reducing the occupation of memory resources. The data can be read directly through the first address, which effectively reduces the computing workload of the CPU and is conducive to improving the overall processing efficiency of image data.
[0090] Through the blocking design, after the image processing thread detection and recognition are completed, the image encoding processing is performed asynchronously, which can effectively improve the data rotation efficiency, increase the frame rate of image processing, and further improve the processing efficiency of memory and CPU through address reference.
[0091] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0092] Figure 6 A schematic diagram of an image data processing device provided in an embodiment of the present application, the device comprising:
[0093] The data acquisition unit 601 is used to acquire the image data to be processed through the frame receiving thread, and determine a first address for storing the image data to be processed of a single frame;
[0094] A storage address sending unit 602 is used to send the first address of the image data to be processed to the first address buffer area through a frame receiving thread;
[0095] The image data processing unit 603 is configured to obtain the first address of the image data to be processed from the first address buffer area through an image processing thread, and read and process the image data to be processed through the first address.
[0096] Figure 6 The image data processing device shown in Figure 2 The image data processing method shown corresponds to.
[0097] Figure 7 Schematic diagram of an image data processing device provided in an embodiment of the present application. Figure 7 As shown, the image data processing device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as an image data processing program. When the processor 70 executes the computer program 72, the steps in the above-mentioned various image data processing method embodiments are implemented. Alternatively, when the processor 70 executes the computer program 72, the functions of the modules / units in the above-mentioned various device embodiments are implemented.
[0098] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 72 in the image data processing device 7.
[0099] The image data processing device 7 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The image data processing device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will appreciate that Figure 7 It is only an example of the image data processing device 7 and does not constitute a limitation of the image data processing device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the image data processing device may also include input and output devices, network access devices, buses, etc.
[0100] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0101] The memory 71 may be an internal storage unit of the image data processing device 7, such as a hard disk or memory of the image data processing device 7. The memory 71 may also be an external storage device of the image data processing device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the image data processing device 7. Further, the memory 71 may also include both an internal storage unit and an external storage device of the image data processing device 7. The memory 71 is used to store the computer program and other programs and data required by the image data processing device. The memory 71 may also be used to temporarily store data that has been output or is to be output.
[0102] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0103] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0104] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0105] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0108] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.
[0109] In addition, an embodiment of the present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the methods in the above-mentioned implementation modes.
[0110] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for processing image data, characterized in that: The method comprises: Acquire the image data to be processed through the frame receiving thread, and determine a first address for storing the image data to be processed of a single frame; Sending the first address of the image data to be processed to the first address buffer area through the frame receiving thread; The first address of the image data to be processed is obtained from the first address buffer area through the image processing thread, and the image data to be processed is read and processed through the first address.
2. The method according to claim 1, characterized in that Reading and processing the to-be-processed image data through the first address includes: Reading the image data to be processed according to the first address through the image processing thread, processing the image data to be processed according to a predetermined image processing method, determining the image data to be encoded, sending the storage address of the image data to be encoded to the second address buffer area, and then obtaining the address of the next frame of image data to be processed from the first address buffer area for processing; The second address of the image data to be encoded is asynchronously obtained from the second address buffer area through the encoding thread, and the image data to be encoded is encoded according to the second address.
3. The method according to claim 2, characterized in that Processing the image data to be processed according to a predetermined image processing method to determine the image data to be encoded includes: Performing detection processing on the image data to be processed by the image processing thread to determine whether the image data to be processed includes an object of interest; When it is detected that the image data to be processed includes an object of interest, the object of interest in the image data to be processed is cropped to obtain cropped image data, the cropped image data is identified, and the cropped image data or the image data to be processed is determined to be encoded image data.
4. The method according to claim 3, characterized in that After detecting and processing the image data to be processed by the image processing thread to determine whether the image data to be processed includes an object of interest, the method further includes: When it is detected that the image data to be processed does not include an object of interest, the cache space occupied by the image data to be processed is released by the image processing thread, and then the address of the next frame of image data to be processed is obtained from the first address cache area for detection.
5. The method according to claim 2, characterized in that: The image data to be processed is image data in YUV format; The method further comprises: performing encoding processing on the image data to be encoded according to the second address, comprising: Reading the image data to be encoded in YUV format according to the second address through the encoding thread; The encoded image data in the YUV format is compressed and encoded by the encoding thread to obtain compressed image data, and then the cache space occupied by the image data to be processed is released.
6. The method according to claim 1, characterized in that Sending the first address of the image data to be processed to the first address buffer area through the frame receiving thread includes: Sending the first address of the image data to be processed to the first address buffer area through the frame receiving thread, and determining the sending result of the first address; When the sending result is successful, executing the step of sending the storage address of the next frame of image data to be processed to the first address buffer area; When the sending result is a sending failure, the storage space occupied by the image data to be processed is released.
7. The method according to any one of claims 1 to 6, characterized in that: Before sending the first address of the to-be-processed image data to the first address buffer area through the frame receiving thread, the method further includes: The frame receiving thread encodes the to-be-processed image data received in sequence according to a video data encoding method to obtain video data in a video encoding format.
8. An image data processing device, characterized in that: The device comprises: A data acquisition unit, used for acquiring the image data to be processed through a frame receiving thread, and determining a first address for storing the image data to be processed in a single frame; A storage address sending unit, used for sending the first address of the image data to be processed to the first address buffer area through a frame receiving thread; The image data processing unit is used to obtain the first address of the image data to be processed from the first address buffer area through the image processing thread, and read and process the image data to be processed through the first address.
9. An image data processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image data processing device implements the method according to any one of claims 1 to 7.
10. A computer program product comprising computer program instructions, characterized in that When the computer program is executed, the method according to any one of claims 1 to 7 is performed.