Image processing apparatus and method

Abstract

The invention discloses an image processing device and method, and the device comprises a first memory which is used for storing the to-be-processed data of a plurality of first images; the scaling control component is used for reading the to-be-processed data of the plurality of first images from the first memory; determining an image scaling coefficient corresponding to each first image; sending the to-be-processed data of each first image and the corresponding zooming coefficient to a zooming component; the zooming component is used for performing zooming processing on the to-be-processed data of the corresponding first image based on the zooming coefficient to obtain a target image; wherein the target image comprises the target object in each first image, and each target object is displayed in the target image according to the target proportion.

Description

Technical Field

[0001] The present application relates to, but is not limited to, the field of computer technology, and in particular to an image processing device and method. Background Art

[0002] In a video conference scenario, the image data of multiple participants in the conference need to be reasonably displayed on the user's display. However, due to the large amount of image data generated in a video conference and the possibility of various data transmission delays, the data processing speed is slow and requires a lot of computing resources. Therefore, how to process the images of multiple participants has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present application at least provides an image processing device and method.

[0004] The technical solution of this application is implemented as follows:

[0005] In one aspect, the present application provides an image processing device, the device comprising:

[0006] A first memory, used for storing a plurality of to-be-processed data of first images;

[0007] A zoom control component is used to read the to-be-processed data of a plurality of first images from the first memory; determine the image zoom coefficient corresponding to each first image; and send the to-be-processed data of each first image and its corresponding image zoom coefficient to the zoom component;

[0008] The scaling component is used to perform scaling processing on the to-be-processed data of the corresponding first image based on the image scaling factor to obtain a target image; wherein the target image includes a target object in each first image, and each target object is displayed in the target image according to a target ratio.

[0009] In some embodiments, the first memory includes at least one first data storage area; the arrangement order of the at least one first data storage area corresponds to the order in which the zoom control component reads the image data from the first memory; wherein,

[0010] Each first data storage area is used to store the data to be processed of at least one first image; wherein the scaled processed images corresponding to each first image are arranged in an array in the target image, and the scaled processed images corresponding to at least one first image stored in the same first data storage area are arranged along a first direction in the target image.

[0011] In some embodiments, the device further includes: a second memory and a read control component, wherein:

[0012] A second memory, used for storing a plurality of first images to be processed;

[0013] A read control component is used to read the idle state information of the first memory; when the idle state information indicates that the first memory has idle storage space not less than a specified threshold, read the to-be-processed data of at least one first image from the second memory, and store the read to-be-processed data of at least one first image in the first memory.

[0014] In some embodiments, the read control component is used to store the read data to be processed of each first image into the corresponding first data storage area based on the reading order of the read data to be processed of at least one first image from the second memory and the arrangement order of at least one first data storage area.

[0015] In some embodiments, the plurality of first images include a plurality of first sub-images; the to-be-processed data of the plurality of first sub-images are stored in different first data storage areas;

[0016] Zoom control component for:

[0017] Reading at least a portion of the to-be-processed data of each first sub-image from each first data storage area according to the arrangement order of at least one first data storage area;

[0018] Sending at least a portion of the to-be-processed data of each first sub-image to a scaling component for performing scaling processing;

[0019] The scaled images corresponding to the multiple first sub-images are arranged along a second direction in the target image; and the second direction is different from the first direction.

[0020] In some embodiments, the plurality of first images include a second sub-image and a third sub-image; the first data storage area corresponding to the second sub-image is sorted before the first data storage area corresponding to the third sub-image;

[0021] The zoom control component is also used to:

[0022] In response to a scaling processing completion signal output by the scaling component for the first portion of data of the second sub-image, the second portion of data of the third sub-image is read from the first memory and an image scaling factor corresponding to the third sub-image is determined; wherein the data amount of the first portion of data is determined based on the image scaling factor corresponding to the second sub-image; and the data amount of the second portion of data is determined based on the image scaling factor corresponding to the third sub-image;

[0023] The read second portion of data and its corresponding image scaling factor are sent to the scaling component.

[0024] In some embodiments, the zoom component is further configured to:

[0025] In response to the completion of the scaling process for the first portion of the data of the second sub-image, first indication information is generated; the first indication information is used to indicate processing position information for at least a portion of the image data of the received second sub-image.

[0026] In some embodiments, the scaling component includes a third memory; the third memory is used to store processed data of at least one first image corresponding to at least one row of pixels along the second direction in the target image;

[0027] The zoom control component is also used to:

[0028] In response to the scaling component completing the scaling process on the first data set of at least one first image, the second data set stored in the third memory is updated using the first data set; wherein the second data set represents image data of at least one first image whose processing order is before the first data set.

[0029] In some embodiments, each first image has corresponding original size information;

[0030] The zoom control component is also used to:

[0031] Get the preset target size information for the target image;

[0032] Based on the original size information and the target size information of each first image, an image scaling factor corresponding to each first image is determined.

[0033] In some embodiments, the apparatus further comprises:

[0034] A driver, the driver is used to read the target image from the scaling component to send the target image to a corresponding application for display output.

[0035] On the other hand, the present application also provides an image processing method, the method comprising:

[0036] Using a zoom control component in the electronic device, read the to-be-processed data of a plurality of first images from the first memory, and determine an image zoom coefficient corresponding to each first image;

[0037] Using a zoom component in an electronic device, the corresponding first image data to be processed is zoomed based on a zoom factor to obtain a target image, wherein the target image includes a target object in each first image, and each target object is displayed in the target image at a target ratio.

[0038] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0040] Figure 1 A schematic diagram of a video conferencing image processing method;

[0041] Figure 2A A video conference image including multiple participants;

[0042] Figure 2B For Figure 2A A plurality of cutout images obtained by performing cutout processing on the video conference image in;

[0043] Figure 2C To use the conference application Figure 2B A stitched image obtained by performing stitching processing on the multiple cutout images in;

[0044] Figure 3 A schematic diagram of the structure of an image processing device provided by the present application;

[0045] Figure 4 A schematic diagram of the structure of another image processing device provided by the present application;

[0046] Figure 5 A schematic diagram of the corresponding relationship between the storage mode of multiple first images in the first memory and the display mode of the corresponding scaled images in the target image in the image processing device provided by the present application;

[0047] Figure 6 A schematic diagram of a state machine of a zoom control component in an image processing device provided by the present application;

[0048] Figure 7 A schematic diagram of the implementation flow of an image processing method provided in this application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0050] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0051] The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged with a specific order or sequence where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0053] In a conference scenario, multiple participants of a video conference participate in the video conference through multiple terminal devices. Figure 1 As shown, multiple participants 121 participate in the video conference through multiple terminal devices 120. In this case, multiple terminal devices 120 transmit the locally collected video data to a server (not shown in the figure) or terminal devices 110 of other participants through a communication network 130; because different terminal devices 120 are in different collection environments, the size ratios of the participants in the collected original images are different. If they are directly presented in the conference application of the terminal devices of other participants, there will be a problem of different display ratios of multiple participants. Therefore, it is necessary to use the conference application running in the server or terminal device 110 to cut, scale and splice the images collected by each terminal device 120, so as to generate a conference screen 111 containing multiple participants with a uniform display ratio.

[0054] In another conference scenario, multiple participants of a video conference participate in the video conference through the same terminal device, that is, there are multiple participants in the same conference venue. Figure 2A As shown in FIG. 1 , when a conference room includes multiple participants, a camera device at the conference room is used to capture an image 210 containing the multiple participants. Due to the different distances from the camera device, the display sizes of participants at different positions in the image are different, which affects the presentation effect. To solve this technical problem, it is necessary to use a conference application to cut the image 210 output by the image capture device to obtain a cutout image 220 corresponding to each participant in the image 210, as shown in FIG. Figure 2B As shown; thereafter, the conference application performs beautification, scaling, and splicing processing on each cutout image 220 to obtain an image 230, in which the display size of each participant in the image 230 is similar, such as Figure 2C As shown; thereafter, the spliced ​​image 230 is transmitted to other venues through a communication network for display output.

[0055] In both of the above conference scenarios, software processing methods are used to cut, clip, and splice the image data of multiple participants to obtain processed images that are arranged in an orderly manner and have similar display sizes. However, the transmission of image data from the image acquisition device (for example, a camera) to the local conference application or the conference application in the remote device for processing is time-consuming; in addition, the splicing of image data using the conference application is software processing, which has a slow processing speed, further reducing the image processing speed and bringing a bad user experience. In particular, in real-time communication scenarios (for example, conference scenarios) that require high image processing speeds, the long delay problem will seriously affect the communication effect.

[0056] Based on this, the present application provides an image processing device, which can be set in electronic devices with data processing capabilities such as conference machines, servers, laptops, tablet computers, desktop computers, smart TVs, set-top boxes, mobile devices (such as mobile phones, portable video players, personal digital assistants, dedicated messaging devices, portable gaming devices), etc.

[0057] Figure 3 A schematic diagram of the structure of an image processing device provided in this application is shown in FIG. Figure 3 As shown, the device 300 includes a first memory 310, a zoom control component 320 and a zoom component 330; wherein:

[0058] The first memory 310 is used to store a plurality of first image data to be processed.

[0059] Here, the first memory 310 is a hardware memory in the image processing apparatus 300 for storing image data to be scaled.

[0060] In some embodiments, the first memory 310 may be a cache or a high-speed buffer corresponding to the zoom control component 320 and the zoom component 330 .

[0061] In some embodiments, the first memory 310 may also be an auxiliary memory installed in the image processing apparatus.

[0062] The first image refers to an image to be processed that includes a target object. In some embodiments, the first image may be a cutout image obtained by performing a cutout process on an original image that includes multiple target objects; the first image may also be a cutout image obtained by performing a cutout process on multiple target objects in multiple original images, where each original image includes one target object.

[0063] In some embodiments, the first image has corresponding object identification information; the object identification information is used to identify the state of the target object in the first image. For example, when the upper layer application is a conference application, a designated target object among multiple target objects can be identified as a conference host through the upper layer application; for another example, the speaking state of multiple target objects can be detected to determine the current speaker from the multiple target objects, and the current speaker can be identified using the object identification information; and so on. The object identification information can also be used as reference information for subsequent scaling of the image.

[0064] The data to be processed of the first image may refer to all image data corresponding to the first image, or may refer to part of the image data of the first image.

[0065] Here, the data to be processed of the first image stored in the first memory 310 is partial image data of the first image, which means, for example, that the pixel data of the first row to the Nth row of the first image is stored in the first memory 310. In this way, a storage space of a specified size appears in the first memory 310, or after performing scaling processing on the pixel data of the first row to the Nth row of the first image, the pixel data of the N+1th row to the Mth row of the first image is stored in the first memory 310 as new data to be processed of the first image. Here, N and M are both integers greater than 1.

[0066] The zoom control component 320 is used to read the unprocessed data of the multiple first images from the first memory 310; determine the image zoom coefficient corresponding to each of the first images; and send the unprocessed data of each of the first images and its corresponding image zoom coefficient to the zoom component.

[0067] Here, the zoom control component 320 refers to the hardware in the image processing device 300 for controlling the image zooming process.

[0068] In some embodiments, the zoom control component 320 reads the to-be-processed data of the first image from the first memory 310 according to the image processing status of the zoom component 330 , and sends the read to-be-processed data to the zoom component 330 .

[0069] In some embodiments, the zoom control component 320 communicates with a corresponding application (e.g., a conference application) to obtain the display size of a target image corresponding to the current multiple first images, thereby determining an image zoom factor corresponding to each first image based on the display size of the target image and the size information of each first image.

[0070] In some embodiments, user configuration information is received through the above application or an electronic device running the above application, and the display size of the target image is determined based on the user configuration information.

[0071] In some embodiments, the size information of the display area for displaying the target image in the running interface of the above application is obtained, and the display size of the target image is determined based on the size information of the display area. For example, when the display area is a square area with an aspect ratio of 1:1, the display size of the target image is set to a square image with an aspect ratio of 1:1; when the display area is a rectangular area with an aspect ratio of 2:1, the display size of the target image is set to a square image with an aspect ratio of 2:1 accordingly. In some embodiments, in response to the user's adjustment operation on the size information of the display area, the display size of the target image is adjusted.

[0072] In some embodiments, the zoom control component 320 communicates with a corresponding application (eg, a conference application) to obtain an image zoom factor corresponding to each first image.

[0073] In some embodiments, each of the first images has corresponding original size information.

[0074] Thus, the zoom control component 320 is also used for:

[0075] Acquire target size information preset for the target image;

[0076] An image scaling factor corresponding to each of the first images is determined based on the original size information of each of the first images and the target size information.

[0077] Here, the zoom control component 320 obtains preset target size information for the target image from the upper layer application, and then can determine the image zoom coefficient of each first image according to the target size information of the target image and the original size information of each first image.

[0078] In some embodiments, the zoom control component 320 obtains the quantity information of multiple first images from the upper-layer application to determine the size information of each first image in the target image based on the target size information of the target image and the quantity information of the multiple first images; then, the zoom control component 320 determines the image zoom factor of each first image based on the size information of each first image in the target image and the original size information.

[0079] In some embodiments, the zoom control component 320 includes a configuration unit for storing multiple sets of image zoom factors. In some embodiments, the configuration unit in the zoom control component 320 can be implemented as multiple registers, each register being used to store a set of image zoom factors.

[0080] In this way, when the zoom control component 320 reads the data to be processed from the first memory 310 and sends it to the zoom component 330 , the zoom control component 320 inputs the image zoom coefficient corresponding to the data to be processed to the zoom component 330 .

[0081] In some embodiments, the image scaling factor corresponding to the first image includes scaling factors in at least two directions, for example, a scaling factor in a horizontal direction and a scaling factor in a vertical direction.

[0082] The scaling component 330 is used to perform scaling processing on the to-be-processed data of the corresponding first image based on the scaling factor to obtain a target image; wherein the target image includes each target object in the first image, and each target object is displayed in the target image according to a target ratio.

[0083] Here, the scaling component 330 is hardware for performing scaling processing on image data.

[0084] After the scaling component 330 receives the first image and its corresponding image scaling factor from the scaling control component 320 , the scaling component performs scaling processing on the first image.

[0085] In some embodiments, the scaling component 330 performs scaling processing on the first images in units of row pixels or column pixels of each first image.

[0086] For example, a reduction process is performed on the first image, that is, the number of pixels in the first image is reduced.

[0087] In some embodiments, the target pixels displayed in the target image can be determined from the original pixels of the first image based on the reduction factor of the first image, and the determined target pixels are filled into the target image. For example, when the rows and columns of the first image are reduced by 2 times, one pixel is selected as the target pixel for every other pixel in the original pixels of the first image.

[0088] In some embodiments, the target pixels to be filled in the target image can be recalculated based on the reduction factor of the first image and the original pixels of the first image. For example, if the first image includes M*N original pixels and the reduced image corresponding to the first image includes m*n target pixels, then first, the original pixels of the first image are divided into multiple small blocks of (M / m)*(N / n) size that do not intersect with each other; then, the average value of each small block is calculated, and the average value is used as the target pixel to be filled in the target image.

[0089] For another example, when the method is executed on the first image, the filling position of each original pixel in the first image in the target image is determined according to the magnification corresponding to the first image, and at the same time, the calculated pixel value is interpolated between the original pixels according to the specified interpolation algorithm.

[0090] In some embodiments, the specified interpolation algorithm may include a nearest neighbor interpolation algorithm, a bilinear interpolation algorithm, a bicubic interpolation algorithm, an edge-based image interpolation algorithm, a region-based image interpolation algorithm, and the like.

[0091] Here, the scaling component 330 performs scaling processing on the multiple first images to obtain a target image, and the target image includes the target object in each first image, that is, the scaling component performs scaling processing on the multiple first images and stitches the scaling processing results corresponding to each first image into a target image.

[0092] In some embodiments, the display ratio of each target object in the target image is the same or similar. For example, the display height of each target object in the target image is the same or similar; for another example, the display position of each target object in the target image is the same or similar. Here, the display ratio is similar, which means that the difference in the display ratio of multiple target objects in the target image is within a preset difference threshold range.

[0093] In some embodiments, a corresponding display size is set for each target object using an application program, so that the display ratios of different target objects in the target image are the same or different. For example, the display size of a specified target object (e.g., a conference host in a conference scene) among multiple target objects in the target image is set to be larger than the display sizes of other target objects in the target image; for another example, the display size of the current speaker among multiple target objects in the target image is set to be larger than the display sizes of other target objects, while the display sizes of the other target objects are the same.

[0094] In the image processing device provided by the present application, a zoom control component is used to read the to-be-processed data of multiple first images from the first memory, and the image zoom coefficient corresponding to each first image is determined, and the to-be-processed data of each first image and its corresponding image zoom coefficient are sent to the zoom component; the zoom component is used to perform zoom processing on the to-be-processed data of the corresponding first image based on the image zoom coefficient to obtain a target image; wherein the target image includes a target object in each first image, and each target object is displayed in the target image at a target ratio. In this way, the zoom processing of multiple first images is completed and a target image is generated through each hardware component in the image processing device, and the target object in each first image is displayed in the target image at a target ratio, so that the zoom processing and splicing processing of the image are completed by the hardware circuit, and the processed target image is given to the corresponding upper-layer application for display output, without the upper-layer application performing processing related to image zooming and splicing, which can greatly improve the image processing speed.

[0095] In some embodiments, the first memory 310 includes at least one first data storage area; the arrangement order of the at least one first data storage area corresponds to the order in which the zoom control component reads the image data from the first memory 310; wherein,

[0096] Each of the first data storage areas is used to store data to be processed of at least one first image; wherein the scaled processed images corresponding to each of the first images are arranged in an array in the target image, and the scaled processed images corresponding to at least one first image stored in the same first data storage area are arranged along a first direction in the target image.

[0097] Here, at least one first data storage area in the first memory 310 is arranged in sequence, and the arrangement order of the at least one first data storage area corresponds to the order in which the zoom control component 320 reads the image data from the first memory 310. Figure 3As shown, the first memory 310 includes three first data storage areas arranged in sequence, namely, the first data storage area 311, the first data storage area 312 and the first data storage area 313. When the zoom control component 320 reads the data to be processed from the first memory 310, the data is read in positive or reverse order according to the arrangement order of the address segments of the first data storage area 311, the first data storage area 312 and the first data storage area 313. In addition, in the case where the first memory 310 uses a linked list structure to store the data to be processed, the zoom control component 320 can read the data from each first data storage area in sequence according to the pointer in the linked list, without considering the positional relationship of the arrangement order of the actual storage area address segments of each first data storage area. For example, first, read the data in the first data storage area 311; then, based on the data pointer in the first data storage area 311 pointing to the first data storage area 313, read the data in the first data storage area 313; then, based on the data pointer in the first data storage area 313 pointing to the first data storage area 312, read the data in the first data storage area 312.

[0098] In some embodiments, when the zoom control component 320 reads the data to be processed from the first memory 310, the zoom control component 320 determines the first image to be processed next according to the first image currently processed by the zoom component 330, and sends a select data source command to the first memory 310 based on the storage address information of the data to be processed of the first image to be processed next; if there is data to be processed in the first data storage area specified by the zoom control component 320, the first memory 310 returns a reply message indicating that the data is valid to the zoom control component 320; thereafter, the zoom control component 320 sends a ready command to the first memory 310, and directly reads the data to be processed from the first memory 310.

[0099] Each first data storage area is used to store data to be processed of at least one first image, that is, in the process of generating a target image using the image processing device 300, each first data storage area can be used to cache data to be processed of one or more first images.

[0100] In some embodiments, the scaled processed images corresponding to each first image are arranged in an array in the target image, which may include the scaled processed images corresponding to each first image being arranged in a linear array, a rectangular array, a circular array, a Z-shaped array, an F-shaped array, or any other type of array arrangement in the target image.

[0101] In this way, for example, when the target image corresponds to a rectangular array arrangement and the first direction is the longitudinal direction of the rectangular array, the scaled processed image corresponding to at least one first image stored in the same first data storage area is arranged along the longitudinal direction of the rectangular array in the target image; for another example, when the target image corresponds to a circular array arrangement and the first direction is the radial extension direction of the circular array, the scaled processed image corresponding to at least one first image stored in the same first data storage area is arranged along the radial extension direction of the circular array in the target image; for another example, when the target image corresponds to a Z-shaped array arrangement and the first direction is the longitudinal direction of the Z array, the scaled processed image corresponding to at least one first image stored in the same first data storage area is arranged along the longitudinal direction of the Z-shape in the target image.

[0102] like Figure 3 As shown, when the target image corresponds to a rectangular array arrangement and the first direction is the longitudinal direction of the rectangular array, the scaled processed images corresponding to at least one first image stored in the first data storage area 311 are arranged in the same column (for example, the first column) in the target image; and by analogy, the scaled processed images corresponding to at least one first image stored in the first data storage area 312 and the first data storage area 313 are respectively arranged in the same column (for example, the second column and the third column) in the target image.

[0103] It can be seen from the above that in the above embodiment, by dividing the first memory into at least one first data storage area, the zoomed images of the plurality of first images are arranged in an orderly manner in the target image.

[0104] In some embodiments, Figure 4 As shown, the present application provides an image processing device 400, which includes a second memory 410, a read control component 420, a first memory 430, a zoom control component 440 and a zoom component 450; wherein:

[0105] The second memory 410 is used to store a plurality of first images to be processed;

[0106] The read control component 420 is used to read the idle state information of the first memory 430; when the idle state information indicates that the first memory 430 has free storage space that is not less than a specified threshold, read the to-be-processed data of at least one first image from the second memory 410, and store the read to-be-processed data of the at least one first image in the first memory 430.

[0107] Here, the second memory 410 refers to a memory for storing complete image data of a plurality of first images.

[0108] In some embodiments, the second memory 410 may be a memory of the image processing device 400 , for example, a Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRA).

[0109] In some embodiments, the first image stored in the second memory 410 may be an image including a plurality of target objects. Thus, before the scaling component 450 is used to scale the first image, a cutout process needs to be performed on the first image to determine a cutout image of each target object in the first image, and the scaling component 450 is used to scale each cutout image.

[0110] In some embodiments, the first image stored in the second memory 410 may be an image obtained by performing a matting process on an original image including a plurality of target objects.

[0111] In some embodiments, the first image stored in the second memory 410 may be image data acquired by a local image acquisition device, or may be image data acquired by an image acquisition device of another electronic device through a data communication path.

[0112] For example, when the image processing device 400 is a conference machine, the camera of the conference machine can be used to collect video data of the scene, and the target object in each frame of the video data can be cut out, and the image containing the target object obtained after the cutout processing is stored as the first image in the second memory 410; for another example, when the image processing device 400 is a personal computer (PC), the PC obtains image data from multiple image acquisition devices through a wired or wireless network, and cuts out the target object in the obtained image data, and the image containing the target object obtained after the cutout processing is stored as the first image in the second memory 410.

[0113] The read control component 420 refers to hardware in the image processing apparatus for controlling the process of reading data from the second memory 410 .

[0114] In some embodiments, the read control component 420 reads the idle state information of the first memory 430 in real time or periodically to determine whether to read the processed data from the second memory 410 and store the read processed data to the first memory 430 according to the idle state information of the first memory 430 .

[0115] Here, the free storage space of the specified threshold refers to the minimum storage space for the read control component 420 to store data in the first memory 430 .

[0116] In some embodiments, the free storage space of the specified threshold may be determined in units of the storage space occupied by a row of pixels of the first image with the smallest data volume, or may be determined based on the minimum data volume that the scaling component can process.

[0117] In some embodiments, the read control component 420 reads the data to be processed from the second memory 410 in units of images, that is, the read control component 420 reads the complete data to be processed of at least one first image from the second memory 410 each time.

[0118] In some embodiments, the read control component 420 reads the data to be processed from the second memory 410 in units of at least one row of pixels.

[0119] In some embodiments, a burst mode is set for the read control component 420 to read a larger amount of data to be processed from the second memory 430 .

[0120] In some embodiments, when the read control component 420 determines that the first memory 430 has free storage space that is not less than a specified threshold based on the idle status information of the read first memory 430, the read control component 420 sends a data read command to the second memory 410, and carries the address information of the data to be processed to be read in the data read command; thereafter, the read control component 420 reads the data to be processed of at least one first image from the corresponding storage address of the second memory 410.

[0121] In the above embodiment, when the second memory 430 is a memory (for example, DDR SDRA) and the first memory 410 is a cache, since the transmission rate and the operating frequency of the memory chip are both low, the to-be-processed data of at least one first image in the first memory 430 are pre-read to the first memory 430 for caching through the read control component 420, so that the scaling control component 440 can obtain the to-be-processed data at a higher frequency, thereby reducing the data transmission delay caused by the low access rate of the second memory 410.

[0122] In some embodiments, the read control component 420 is used to store the to-be-processed data of each first image read to the corresponding first data storage area based on the reading order of the to-be-processed data of the at least one first image read from the second memory 410 and the arrangement order of the at least one first data storage area.

[0123] In some embodiments, the reading order of the to-be-processed data of the at least one first image by the read control component 420 may be determined based on the storage order and / or storage time of the at least one first image in the second memory 410 .

[0124] In some embodiments, the reading order of the to-be-processed data of at least one first image by the read control component 420 may be determined based on the identification information of at least one first image. For example, the read control component 420 obtains the reading order information corresponding to the plurality of first images from the upper layer application, and then reads the to-be-processed data of the plurality of first images in sequence according to the obtained reading order information based on the identification information of the plurality of first images.

[0125] Here, when the read control component 420 reads the to-be-processed data of the first image from the second memory 410 in units of at least one row of pixel data, the read control component 420 stores the to-be-processed data belonging to the same first image in the same first data storage area.

[0126] For example, Figure 4 As shown, after the read control component 420 detects that the first data storage area 431 has free storage space that is not less than the specified threshold, the first row of pixel data corresponding to image A is read from the second memory 410 and stored in the first data storage area 431; thereafter, after the read control component 420 detects that the first data storage area 432 has free storage space that is not less than the specified threshold, the first row of pixel data corresponding to image B is read from the second memory 410 and stored in the first data storage area 432; when the read control component 420 detects again that the first data storage area 431 has free storage space that is not less than the specified threshold, the second row of pixel data corresponding to image A is read from the second memory 410 and stored in the first data storage area 431.

[0127] In some embodiments, the plurality of first images include a plurality of first sub-images; and the to-be-processed data of the plurality of first sub-images are stored in different first data storage areas.

[0128] For example, when the first sub-images of the first images are image C, image D and image E, the to-be-processed data of the first sub-images are stored in different first data storage areas, that is, for example, image C, image D and image E are stored in the first data storage areas, respectively. Figure 3 The first data storage area 311, the first data storage area 312 and the first data storage area 313 are shown in FIG.

[0129] Thus, the zoom control component 320 is used to:

[0130] Reading at least a portion of the to-be-processed data of each first sub-image from each first data storage area according to the arrangement order of the at least one first data storage area;

[0131] Sending at least part of the to-be-processed data of each of the first sub-images to the scaling component 330 for performing scaling processing;

[0132] The scaled images corresponding to the multiple first sub-images are arranged along a second direction in the target image; and the second direction is different from the first direction.

[0133] Here, the zoom control component 320 reads the data to be processed from at least one first data storage area in sequence according to the arrangement order of at least one first data storage area in the first memory 310, and sends the data to be processed and the corresponding image zoom factor to the zoom component, so that the zoom component 330 performs image zoom processing based on the data to be processed and the corresponding image zoom factor.

[0134] Continuing with the above example, after the zoom control component 320 reads the data to be processed of image C from the first data storage area 311, the zoom control component 320 sends the data to be processed of image C to the zoom component 330, and configures the image zoom coefficient corresponding to image C for the zoom component 330, so that the zoom component 330 performs zoom processing on the data to be processed of image C based on the configured image zoom coefficient; thereafter, the zoom control component 320 processes the data to be processed of image D and image E in the first data storage areas 312 and 313 in sequence according to the same steps.

[0135] Here, in the target image after scaling, the plurality of first sub-images stored in different first data storage areas are arranged along the second direction. For example, when the scaled images in the target image are arranged in a rectangular array, image C, image D, and the scaled image corresponding to image D may be arranged in the horizontal or vertical direction of the rectangular array in the target image.

[0136] At the same time, the second direction is different from the first direction described above, that is, the arrangement direction of the scaled processed image corresponding to at least one first image stored in the same first data storage area in the target image is different from the arrangement direction of the scaled processed image corresponding to at least one first image stored in different first data storage areas in the target image. For example, when the scaled processed images in the target image are arranged in a rectangular array, the first direction can represent the longitudinal direction of the rectangular array, and the second direction can represent the transverse direction of the rectangular array.

[0137] like Figure 5 As shown, the first memory includes three first data storage areas, namely, the first data storage area 510, the first data storage area 520 and the first data storage area 530; wherein, before generating the target image 500, the first data storage area 510 is used to store the data to be processed of image 1 and image 4, the first data storage area 520 is used to store the data to be processed of image 2 and image 5, and the first data storage area 530 is used to store the data to be processed of image 3 and image 6.

[0138] After the scaling process by the scaling component, the scaled images corresponding to the images stored in the same first data storage area are arranged vertically in the target image 500, that is, the scaled images corresponding to image 1 and image 4 are in the same column in the target image 500, the scaled images corresponding to image 2 and image 5 are in the same column in the target image 500, and the scaled images corresponding to image 3 and image 6 are in the same column in the target image 500; at the same time, the scaled images corresponding to the images stored in different first data storage areas are arranged horizontally in the target image 500, that is, the scaled images corresponding to image 1, image 2 and image 3 are in the same row in the target image 500, and the scaled images corresponding to image 4, image 5 and image 6 are in the same row in the target image 500.

[0139] Here, since the target image processed by the scaling component needs to be sent to the corresponding rendering component (for example, a graphics processing unit (GPU)) for rendering, when the rendering component renders the target image according to pixel rows, the scaling control component sequentially reads the data to be processed from the first data storage areas 510, 520 and 530 and sends them to the scaling component for scaling processing. After that, the scaling component arranges the pixels in the generated target image in pixel rows. After forming part of the pixel row data of the target image, the pixels of the generated target image can be sent to the GPU for rendering in a timely manner according to the pixel rows, without waiting for the reception to obtain all the data of the target image before rendering. In this way, a pipelined image processing process is formed from the scaling control component, the scaling component to the rendering component, and each component completes a part of the image processing task, which can significantly improve the image processing efficiency.

[0140] In this way, through the arrangement order of at least one first data storage area in the first memory and the data reading order of the zoom control component, the arrangement of the zoomed images corresponding to the multiple first images in the target image can be determined, thereby achieving the purpose of determining the image output effect by using hardware equipment.

[0141] In some embodiments, Figure 4 As shown, the upper layer application sends the display format information related to the target image to the read control component 420; wherein the display format information at least includes the number of first images corresponding to each row in the target image along the second direction. For example, when the second direction is the horizontal direction of the target image, the display format information at least includes the number of first images corresponding to each row in the target image.

[0142] In this way, the read control component 420 can determine at least one first data storage area from the multiple data storage areas of the first memory 430 based on the display format information, so as to use the at least one first data storage area to store the to-be-processed data of the multiple first images. For example, when the display format information indicates that each row in the target image corresponds to three first images, the read control component 420 determines three from the multiple data storage areas of the first memory 430 as the first data storage areas.

[0143] Thus, in the above embodiment, after the zoom control component 440 reads at least part of the data to be processed of each first sub-image from each first data storage area according to the arrangement order of at least one first data storage area and sends it to the zoom component 450, the zoom component 450 can arrange the zoomed processed images of the multiple first sub-images along the second direction in the target image based on the multiple first sub-images stored in different first storage areas.

[0144] In some embodiments, the plurality of first images include a second sub-image and a third sub-image; and the first data storage area corresponding to the second sub-image is sorted before the first data storage area corresponding to the third sub-image.

[0145] For example, Figure 3 As shown, the second sub-image is an image stored in the first data storage area 311 , and the third sub-image is an image stored in the first data storage area 312 .

[0146] The zoom control component 320 is further used for:

[0147] In response to a scaling processing completion signal outputted by the scaling component 330 for the first portion of data of the second sub-image, the second portion of data of the third sub-image is read from the first memory 310 and an image scaling factor corresponding to the third sub-image is determined; wherein the data amount of the first portion of data is determined based on the image scaling factor corresponding to the second sub-image; and the data amount of the second portion of data is determined based on the image scaling factor corresponding to the third sub-image;

[0148] The read second portion of data and its corresponding image scaling factor are sent to the scaling component.

[0149] Here, after receiving the signal that the scaling component 330 completes the scaling process for the first part of the data of the second sub-image, the scaling control component 320 reads the second part of the data of the third sub-image stored in the next first data storage area and determines the corresponding image scaling factor according to the arrangement order of at least one first data storage area in the first memory 310. In this way, the cross-reading of the data to be processed in at least one first data storage area is realized, and then the data to be processed in at least one first data storage area is cross-processed by using the scaling component 330.

[0150] Meanwhile, the amount of the to-be-processed data read by the zoom control component 320 is determined based on the corresponding image zoom factor.

[0151] For example, the zoom factor corresponding to the second sub-image is 2, and the zoom factor corresponding to the third sub-image is 1 / 2, that is, the second sub-image is enlarged by 2 times, and the third sub-image is reduced by 2 times. In this way, in order to enable the zoom component 330 to generate the target image in units of rows of pixels, the zoom control component can read 1 row of pixel data read from the second sub-image as the first part of data, and read 4 rows of pixel data from the third sub-image as the second part of data, so that the zoom component 330 can generate 2 rows of pixel data in the target image based on the first part of data and the second part of data.

[0152] In some embodiments, the scaling component 330 is further configured to:

[0153] In response to completion of the scaling process for the first portion of data of the second sub-image, first indication information is generated; the first indication information is used to indicate processing position information for at least a portion of the received image data of the second sub-image.

[0154] Since the target image includes the scaled data of multiple first images, the scaling component 330 needs to switch to the data to be processed of different first images for scaling processing during the process of generating the target image. For example, after the scaling component 330 receives the first part of the data of the second sub-image from the scaling control component 320 and performs scaling processing, the scaling component 330 switches to processing the second part of the data of the third sub-image received from the scaling control component 320.

[0155] In this way, in order to enable the scaling component 330 to accurately locate the processing position of the second sub-image when it continues to process the next part of the data of the second sub-image, the scaling component 330 generates information for indicating the current processing position, that is, the first indication information, for at least part of the image data of the received second sub-image, so that the processing can be continued from the breakpoint position when the next part of the data of the second sub-image is subsequently processed.

[0156] In some embodiments, the first indication information may be row count information (V-count) for indicating the number of rows that have been processed.

[0157] In some embodiments, the scaling component 330 includes a third memory; the third memory is used to store processed data of at least one first image corresponding to at least one row of pixels along the second direction in the target image.

[0158] Here, the scaling component 330 cross-processes the to-be-processed data of the plurality of first images according to the arrangement order of at least one first data storage area in the first memory 310, so that the scaled processed images of the plurality of first images stored in different first data storage areas can be arranged along the second direction (for example, horizontally) in the target image. Therefore, a third memory is provided in the scaling component 330 to store the processed data of at least one first image data corresponding to at least one row of pixels in the target image along the second direction received by the scaling component 330 from the scaling control component 320, so that when the scaling component 330 generates the next row of pixels in the target image along the second direction, the processed data stored in the third memory can be referred to.

[0159] Thus, the zoom control component 320 is also used for:

[0160] In response to the scaling component 330 completing the scaling process on the first data set of at least one first image, the second data set stored in the third memory is updated using the first data set; wherein the second data set represents image data of at least one first image whose processing order is before the first data set.

[0161] Here, the first data set and the second data set respectively represent image data of at least one first image corresponding to at least one row of pixels in the target image, and the image data corresponding to the second data set is scaled before the image data corresponding to the first data set.

[0162] In this way, when performing scaling processing on the first data set, the scaling component 330 needs to refer to the second data set stored in the third memory so that at least one row of pixels in the target image generated based on the first data set can match at least one row of pixels in the target image generated based on the second data set.

[0163] At the same time, in response to the zoom component 330 completing the zoom processing on the first data set, the zoom control component 320 uses the first data set to update the second data set in the third storage area, that is, clears the second data set in the third memory, and stores the first data set to the third memory, so that the first data set can be used as a reference when the zoom component 330 generates the next row of pixels in the target image.

[0164] In some embodiments, the third memory is further used to store pre-scaling data corresponding to previously processed pixels in the same row of pixels in the target image.

[0165] For example, Figure 5 As shown, when the scaling component 330 generates the first row of pixel points of the target image 550, the scaling component 330 first performs scaling processing on part of the to-be-processed data of image 1 in the first data storage area 510, and stores the processed image data of image 1 in the third memory; thereafter, when the scaling component 330 processes part of the to-be-processed data of image 2 in the second data storage area 520, the image data of image 1 stored in the third memory can be used as a reference.

[0166] Next, combine Figure 6 , the state machine of the zoom control component in the image processing device provided in this application is described.

[0167] like Figure 6 As shown, when there is no image processing task currently, the zoom control component is in an idle state 610;

[0168] After receiving the image processing task, the zoom control component starts to read the to-be-processed data of the first image from the first memory (frame_start), and enters the cleaning state 620 (clr) to clean up the parameter configuration information of the component, for example, the parameter configuration information for the image zoom factor;

[0169] After reading the data to be processed of the first image, the zoom control component determines the image zoom coefficient corresponding to the first image, and enters the update image zoom coefficient state 630 (update coef) to update the image zoom coefficient corresponding to the first image to the zoom component;

[0170] During the period when the scaling component performs scaling processing on the to-be-processed data of the first image, the scaling control component enters the scaling component processing state 640 (scaler_proc) to wait for a signal indicating that the scaling component has completed scaling processing on the to-be-processed data of the current first image;

[0171] After the scaling component completes the scaling process of the to-be-processed data of the current first image, the scaling component sends a processing completion signal (part_le) to the scaling control component, so that the scaling control component reads the to-be-processed data of the next first image from the first memory again and re-enters the cleaning state 620;

[0172] After the zoom component generates a complete frame of the target image, the zoom component sends a signal (o_fe) indicating that the processing of the entire frame of the image is completed to the zoom control component, and the zoom control component enters the end state 650 (end).

[0173] In some embodiments, the image processing device provided by the present application further includes a driver; the driver is used to read the target image from the scaling component to send the target image to a corresponding upper-layer application for display output.

[0174] Here, in the image processing device provided by the present application, hardware is used to perform scaling and stitching processing on multiple first images, so that the driver can directly obtain the scaled and stitched target image from the hardware (ie, the scaling component), and then send the target image to the upper-level application.

[0175] In some embodiments, the driver reads and outputs the target image from the scaling component in units of at least one row of pixels.

[0176] In some embodiments, in response to the number of pixels of the target image obtained after processing by the scaling component being not less than a number threshold, the driver reads and outputs the target image from the scaling component; wherein the number threshold represents the minimum number of pixels of the target image that the driver reads from the scaling component.

[0177] On the other hand, the present application also provides an image processing method, which can be executed by a processor of a computer device equipped with the above-mentioned image processing apparatus. The computer device may refer to a conference machine, a server, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device) and other devices with image processing capabilities.

[0178] Figure 7 FIG7 is a schematic diagram of an implementation flow of an image processing method provided by the present application. The method includes the following steps S701 to S702:

[0179] Step S701, using a zoom control component in an electronic device, reading a plurality of first image data to be processed from a first memory, and determining an image zoom coefficient corresponding to each of the first images;

[0180] Step S702, utilizing the zoom component in the electronic device to perform zoom processing on the corresponding first image's processed data based on the zoom factor to obtain a target image; wherein the target image includes each target object in the first image, and each target object is displayed in the target image at a target ratio.

[0181] In some embodiments, the first memory includes at least one first data storage area; the arrangement order of the at least one first data storage area corresponds to the order in which the zoom control component reads the image data from the first memory; wherein,

[0182] Each of the first data storage areas is used to store data to be processed of at least one first image; the scaled processed images corresponding to each of the first images are arranged in an array in the target image, and the scaled processed images corresponding to at least one first image stored in the same first data storage area are arranged along a first direction in the target image.

[0183] In some embodiments, the image processing method further includes:

[0184] Using a read control component to read idle state information of the first memory;

[0185] When the idle state information indicates that the first memory has free storage space not less than a specified threshold, the read control component is used to read the to-be-processed data of at least one first image from the second memory, and the read to-be-processed data of the at least one first image is stored in the first memory; the second memory is used to store multiple first images to be processed.

[0186] In some embodiments, the using the read control component to read the to-be-processed data of at least one first image from the second memory, and storing the read to-be-processed data of the at least one first image to the first memory, comprises:

[0187] Based on the reading order of the read control component reading the to-be-processed data of the at least one first image from the second memory and the arrangement order of the at least one first data storage area, the read control component is used to store the to-be-processed data of each read first image into the corresponding first data storage area.

[0188] In some embodiments, the plurality of first images include a plurality of first sub-images; the to-be-processed data of the plurality of first sub-images are stored in different first data storage areas;

[0189] The method of using a zoom control component in an electronic device to read the to-be-processed data of a plurality of first images from a first memory includes:

[0190] Reading at least a portion of the to-be-processed data of each first sub-image from each first data storage area according to the arrangement order of the at least one first data storage area;

[0191] The scaled images corresponding to the multiple first sub-images are arranged along a second direction in the target image; and the second direction is different from the first direction.

[0192] In some embodiments, the plurality of first images include a second sub-image and a third sub-image; the first data storage area corresponding to the second sub-image is sorted before the first data storage area corresponding to the third sub-image;

[0193] The method further comprises:

[0194] In response to a scaling processing completion signal output by the scaling component for the first portion of data of the second sub-image, the scaling control component is used to read the second portion of data of the third sub-image from the first memory and determine an image scaling factor corresponding to the third sub-image; wherein the data amount of the first portion of data is determined based on the image scaling factor corresponding to the second sub-image; and the data amount of the second portion of data is determined based on the image scaling factor corresponding to the third sub-image;

[0195] The zoom control component is used to send the read second portion of data and its corresponding image zoom coefficient to the zoom component.

[0196] In some embodiments, the method further comprises:

[0197] In response to completion of the scaling process for the first portion of data of the second sub-image, the scaling component is used to generate first indication information; the first indication information is used to indicate processing position information for at least a portion of the received image data of the second sub-image.

[0198] In some embodiments, the scaling component includes a third memory; the third memory is used to store processed data of at least one first image corresponding to at least one row of pixels along the second direction in the target image;

[0199] The method further comprises:

[0200] In response to the zoom component completing the zoom processing of the first data set of at least one first image, the zoom control component uses the first data set to update the second data set stored in the third memory; wherein the second data set represents the image data of at least one first image whose processing order is before the first data set.

[0201] In some embodiments, each of the first images has corresponding original size information;

[0202] The method further comprises:

[0203] The zoom control component is used to obtain preset target size information for the target image; and based on the original size information of each first image and the target size information, an image zoom coefficient corresponding to each first image is determined.

[0204] In some embodiments, the method further comprises:

[0205] The target image is read from the scaling component using a driver program to send the target image to a corresponding application for display output.

[0206] The description of the above method embodiment is similar to the description of the above device embodiment, and has similar beneficial effects as the device embodiment. In some embodiments, for technical details not disclosed in the method embodiment of the present application, please refer to the description of the device embodiment of the present application for understanding.

[0207] It should be noted that in the embodiment of the present application, if the above-mentioned image processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0208] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0209] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0210] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0211] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0212] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0213] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0214] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0215] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0216] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0217] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0218] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0219] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0220] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An image processing device, comprising: A first memory, used for storing a plurality of to-be-processed data of first images; A zoom control component, configured to read the to-be-processed data of the plurality of first images from the first memory; determining an image scaling factor corresponding to each of the first images; Sending the to-be-processed data of each first image and its corresponding image scaling factor to a scaling component; A scaling component is used to perform scaling processing on the to-be-processed data of the corresponding first image based on the image scaling factor to obtain a target image; wherein the target image includes each target object in the first image, and each target object is displayed in the target image according to a target ratio.

2. The device according to claim 1, wherein the first memory comprises at least one first data storage area; the arrangement order of the at least one first data storage area corresponds to the order in which the zoom control component reads the image data from the first memory; wherein Each of the first data storage areas is used to store data to be processed of at least one first image; wherein the scaled processed images corresponding to each of the first images are arranged in an array in the target image, and the scaled processed images corresponding to at least one first image stored in the same first data storage area are arranged along a first direction in the target image.

3. The device according to claim 2, further comprising: A second memory and a read control component, wherein The second memory is used to store a plurality of first images to be processed; The read control component is used to read the idle state information of the first memory; when the idle state information indicates that the first memory has free storage space not less than a specified threshold, read the to-be-processed data of at least one first image from the second memory, and store the read to-be-processed data of the at least one first image in the first memory.

4. The device according to claim 3, wherein the read control component is used to store the to-be-processed data of each first image read into the corresponding first data storage area based on the reading order of the to-be-processed data of the at least one first image read from the second memory and the arrangement order of the at least one first data storage area.

5. The device according to claim 2, wherein the plurality of first images include a plurality of first sub-images; and the to-be-processed data of the plurality of first sub-images are stored in different first data storage areas; The zoom control component is used to: Reading at least a portion of the to-be-processed data of each first sub-image from each first data storage area according to the arrangement order of the at least one first data storage area; Sending at least a portion of the to-be-processed data of each of the first sub-images to the scaling component for performing scaling processing; in, The scaled images corresponding to the multiple first sub-images are arranged along a second direction in the target image; the second direction is different from the first direction.

6. The device according to claim 2, wherein the plurality of first images include a second sub-image and a third sub-image; the first data storage area corresponding to the second sub-image is arranged before the first data storage area corresponding to the third sub-image; The zoom control component is further used for: In response to a scaling processing completion signal output by the scaling component for the first portion of data of the second sub-image, the second portion of data of the third sub-image is read from the first memory and an image scaling coefficient corresponding to the third sub-image is determined; wherein, The data volume of the first part of data is determined based on the image scaling factor corresponding to the second sub-image; the data volume of the second part of data is determined based on the image scaling factor corresponding to the third sub-image; The read second portion of data and its corresponding image scaling factor are sent to the scaling component.

7. The apparatus according to claim 6, wherein the scaling component is further configured to: In response to completion of the scaling process for the first portion of data of the second sub-image, first indication information is generated; the first indication information is used to indicate processing position information for at least a portion of the received image data of the second sub-image.

8. The device according to any one of claims 2 to 7, wherein the scaling component comprises a third memory; the third memory is used to store processed data of at least one first image corresponding to at least one row of pixels along the second direction in the target image; The zoom control component is further used for: In response to the scaling component completing the scaling process of the first data set of at least one first image, the second data set stored in the third memory is updated using the first data set; wherein, The second data set represents image data of at least one first image that precedes the first data set in a processing order.

9. The device according to claim 1, wherein each of the first images has corresponding original size information; The zoom control component is further used for: Acquire target size information preset for the target image; An image scaling factor corresponding to each of the first images is determined based on the original size information of each of the first images and the target size information.

10. The apparatus according to claim 1, further comprising: A driver, wherein the driver is used to read the target image from the scaling component to send the target image to a corresponding application for display output.

11. An image processing method, the method comprising: Using a zoom control component in the electronic device, read the to-be-processed data of a plurality of first images from a first memory, and determine an image zoom coefficient corresponding to each of the first images; Using the scaling component in the electronic device, scaling processing is performed on the corresponding first image's processed data based on the scaling factor to obtain a target image; wherein the target image includes each target object in the first image, and each target object is displayed in the target image according to a target ratio.

Images