Image processing methods, apparatus and storage media

Abstract

This application provides an image processing method, apparatus, and storage medium, relating to the field of artificial intelligence technology. It enables the determination of the sliding window's sliding distance during data processing based on the image recognition model's image frame processing rate and the frame extraction rate during data processing, thereby altering the input image frame sequence and fully utilizing the computational power of the image recognition model. The method includes: acquiring the image extraction rate of an image extractor and the image processing rate of an image processing model; the image extractor extracting images from a video to be processed; adjusting the sliding step size of the image sampling window of the image frame sequence according to the image extraction rate and the image processing rate; the image frame sequence comprising multiple frames extracted by the image extractor; and matching the number of images acquired by the adjusted image sampling window to the image processing rate of the image processing model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus and storage medium. Background Technology

[0002] With the development of artificial intelligence and deep learning technologies, the performance of image processing models has been significantly improved. Deep learning algorithms, such as convolutional neural networks (CNNs), can autonomously learn and extract complex features of images, thereby achieving efficient recognition of image content.

[0003] Currently, image processing models typically use fixed-window-size data frame processing for input preprocessing. However, since the performance of computing devices deploying image processing models often cannot maintain a stable value, resource waste and low utilization rates occur during the operation of image processing models due to performance fluctuations. Summary of the Invention

[0004] This application provides an image processing method, apparatus, and storage medium that can fully utilize the computing resources of an image processing model by changing the sliding distance of image sampling.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, this application provides an image processing method, the method comprising:

[0007] The image extraction rate of the image extractor and the image processing rate of the image processing model are obtained; the image extractor is used to extract images from the video to be processed; the sliding step size of the image sampling window of the image frame sequence is adjusted based on the image extraction rate and the image processing rate; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window is matched with the image processing rate of the image processing model.

[0008] Based on the above technical means, this application can change the number of images input to the image processing model by adjusting the sliding step size of the image sampling window, thereby matching the processing rate of the image processing model and realizing the full utilization of the computing resources of the image processing model.

[0009] In one possible implementation, the sliding step size of the image sampling window is adjusted according to the image processing rate and / or the window length of the image sampling window.

[0010] Based on the above technical means, this application can obtain the sliding step size of the image sampling window, and the number of images collected by the image sampling window through the sliding step size can be matched with the image processing rate.

[0011] In one possible implementation, when the image processing rate is greater than the image extraction rate, the sliding step size of the image sampling window is adjusted according to the image extraction rate and the window length of the image sampling window; the adjusted sliding step size is the ratio of the window length to the image extraction rate.

[0012] Based on the above technical means, this application can obtain the sliding step size of the image sampling window when the image processing rate is greater than the image extraction rate. The image sampling window can input an image into the image processing model multiple times through this sliding step size, thereby making full use of the idle resources of the image processing model and improving the accuracy of the image processing model.

[0013] In one possible implementation, when the image processing rate is less than or equal to the image extraction rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0014] Based on the above technical means, this application can obtain the sliding step size of the image sampling window when the image processing rate is less than or equal to the image extraction rate. The number of images collected by the image sampling window through this sliding step size can match the processing rate of the image processing model, thus avoiding the accumulation of input data.

[0015] In one possible implementation, when the image processing rate equals the image extraction rate, the adjusted sliding step size equals the window length; when the image processing rate is less than the image extraction rate, the adjusted sliding step size is N times the window length, where N is greater than or equal to 2.

[0016] Based on the above technical means, this application can obtain the sliding step size of the image sampling window with an image processing rate equal to or less than the image extraction rate, thereby making the number of images collected by the image sampling window match the processing rate of the image processing model.

[0017] In one possible implementation, after extracting multiple frames of images from the image frame sequence based on the adjusted sliding step size, some images in the multiple frames are deleted, and the deleted images are input into the image processing model.

[0018] Based on the above technical means, this application can reduce the number of images input to the image processing model per unit time by deleting some images when the image processing rate is lower than the image extraction rate, so that the number of images input to the image processing model per unit time matches the image processing rate.

[0019] In one possible implementation, target images are deleted from multiple frames of images based on preset deletion rules. The preset deletion rules include at least one of the following: deleting images with a resolution lower than a preset resolution; deleting images with a similarity to other images higher than a preset similarity; extracting at least one target image from multiple frames of images at preset intervals and deleting at least one target image.

[0020] Based on the above technical means, this application can delete unnecessary images according to preset deletion rules, thereby reducing the image processing pressure on the image processing model and minimizing the errors caused by image deletion.

[0021] Secondly, this application provides an image processing apparatus, the apparatus comprising:

[0022] The communication unit is used to obtain the image extraction rate of the image extractor and the image processing rate of the image processing model; the image extractor is used to extract images from the video to be processed.

[0023] The processing unit is used to adjust the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window is matched with the image processing rate of the image processing model.

[0024] In one possible implementation, the processing unit is further configured to adjust the sliding step size of the image sampling window according to the image processing rate and / or the window length of the image sampling window.

[0025] In one possible implementation, the processing unit is further configured to adjust the sliding step size of the image sampling window according to the image extraction rate and the window length of the image sampling window when the image processing rate is greater than the image extraction rate; the adjusted sliding step size is the ratio of the window length to the image extraction rate.

[0026] In one possible implementation, the processing unit is further configured to adjust the sliding step size of the image sampling window based on the window length of the image sampling window when the image processing rate is less than or equal to the image extraction rate.

[0027] In one possible implementation, the processing unit is further configured to adjust the sliding step size to be equal to the window length when the image processing rate is equal to the image extraction rate; and to adjust the sliding step size to be N times the window length when the image processing rate is less than the image extraction rate, where N is greater than or equal to 2.

[0028] In one possible implementation, the processing unit is further configured to, after extracting multiple frames of images from the image frame sequence based on the adjusted sliding step size, delete some images from the multiple frames of images and input the deleted images into the image processing model.

[0029] In one possible implementation, the processing unit is further configured to delete target images from multiple frames of images based on preset deletion rules; the preset deletion rules include at least one of the following: deleting images with a resolution lower than a preset resolution; deleting images with a similarity to other images higher than a preset similarity; extracting at least one target image from multiple frames of images at preset intervals and deleting at least one target image.

[0030] Thirdly, this application provides a computer-readable storage medium storing instructions that, when executed on a terminal, cause the terminal to perform the image processing method as described in the first aspect and any possible implementation thereof.

[0031] Fourthly, this application provides a computer program product containing instructions that, when run on an image processing apparatus, cause the image processing apparatus to perform the image processing method as described in the first aspect and any possible implementation thereof. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the structure of an image processing system provided in an embodiment of this application;

[0033] Figure 2 A flowchart of an image processing method provided in this application embodiment Figure 1 ;

[0034] Figure 3 This application provides an illustration of an image processing method. Figure 1 ;

[0035] Figure 4 This application provides an illustration of an image processing method. Figure 2 ;

[0036] Figure 5 This application provides an illustration of an image processing method. Figure 3 ;

[0037] Figure 6 A flowchart of an image processing method provided in this application embodiment Figure 2 ;

[0038] Figure 7 This application provides an illustration of an image processing method. Figure 4 ;

[0039] Figure 8 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0040] The image processing method, apparatus, and storage medium provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0041] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0042] The terms "first" and "second," etc., used in the specification and drawings of this application are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of objects.

[0043] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0044] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0045] In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0046] The following explanations of the terms used in the embodiments of this application are provided to facilitate the reader's understanding.

[0047] (1) Image sampling service based on image sampling window: Image sampling service based on image sampling window can also be image sampling service based on sliding window. The image sampling service samples multiple image frame sequences or continuous video frames by setting a sliding window on multiple consecutive image frame sequences or continuous video frames to obtain image frame sequences or continuous video frames. In this application, the image frame sequence after image extraction service can be sampled according to the preset image sampling window size, sliding step size and preset activity rules to obtain the image frame sequence of the input image processing model.

[0048] The above is a brief introduction to some of the concepts involved in the embodiments of this application.

[0049] Currently, image processing models typically use fixed-window-size data frame processing for input preprocessing. However, since the performance of computing devices deploying image processing models often cannot maintain a stable value, resource waste and low utilization rates occur during the operation of image processing models due to performance fluctuations.

[0050] In view of this, embodiments of this application provide an image processing method that can determine the sliding distance of the sliding window in the data processing process based on the image recognition model's processing rate of image frames and the frame extraction rate in the data processing stage, thereby changing the input image frame sequence and realizing full utilization of the image recognition model's computing power.

[0051] For example, Figure 1 The diagram shows a schematic representation of an image processing system 10 according to an embodiment of this application. The image processing system 10 may include at least one scheduler 101, at least one log terminal 102, at least one image processing model 103, at least one image extractor 104, and at least one parameter configuration terminal 105. The scheduler 101, log terminal 102, image processing model 103, image extractor 104, and parameter configuration terminal 105 can be communicatively connected. Figure 1 The document shows only one scheduler 101, one log terminal 102, one image processing model 103, one image extractor 104, and one parameter configuration terminal 105. This application embodiment does not impose any limitation on the number of schedulers 101, log terminals 102, image processing models 103, image extractors 104, and parameter configuration terminals 105.

[0052] Scheduler 101 is used to receive the image processing rate and obtain the image extraction rate. It is also used to determine the sliding step size of the sampling window based on the relationship between the image processing rate and the image extraction rate, and send the sliding step size of the sampling window to the parameter configuration terminal 105.

[0053] Log terminal 102 is used to obtain the working log of the image processing model. It is also used to determine the image processing rate of the image processing model based on the working log of the image processing model, and send the image processing rate to scheduler 101.

[0054] Image processing model 103 is used to process the received image frame sequence and output the processing results.

[0055] Image extractor 104 is used to perform image extraction processing on the original video and / or multiple images to obtain a sequence of image frames for image sampling. It is also used to sample the sequence of image frames to obtain multiple sequences of image frames for input image processing models.

[0056] The parameter configuration terminal 105 is used to receive the sliding step size of the image sampling window and adjust the sliding step size of the image sampling window in the image extractor.

[0057] The application scenarios of the embodiments in this application are not limited. The system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0058] like Figure 2 As shown in the embodiment of this application, an image processing method is proposed, which includes:

[0059] S201, The scheduler obtains the image extraction rate of the image extractor and the image processing rate of the image processing model.

[0060] Here, image extraction rate refers to the number of images extracted per unit time during the video extraction process. Image processing rate refers to the number of images that the image processing model can process per unit time. The image processing model can be an image recognition model, an image classification model, or other image processing-related artificial intelligence models.

[0061] In one example, the image extraction rate is a preset fixed value, which can be obtained through configured preset parameters. The image processing rate can be obtained by analyzing the working log of the image processing model within a preset time period at preset intervals. For example, the image processing rate can be 30 frames / s or 60 frames / s, without limitation.

[0062] The image processing model's work log contains the completion times of multiple image processing steps. The log terminal can determine the image processing rate of the image processing model based on the completion times of multiple image processing steps within a preset time period. The preset time period and preset cycle can be set according to user needs; they can be 1 minute, 10 minutes, or other values ​​without restriction.

[0063] S202. The scheduler adjusts the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate.

[0064] The image frame sequence comprises multiple frames extracted by the image extractor. The number of images acquired by the adjusted image sampling window matches the image processing rate of the image processing model. The sliding step size of the image sampling window can also be the distance moved from the starting position of the image sampling window.

[0065] In one example, the sliding step size of the image sampling window is adjusted based on the image extraction rate and / or the window length of the image sampling window.

[0066] Specifically, when the image processing rate is greater than the image extraction rate, the sliding step size of the image sampling window is adjusted according to the image extraction rate and the window length of the image sampling window.

[0067] When the image processing rate is less than or equal to the image extraction rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0068] For specific details, please refer to situations 1 through 3.

[0069] Case 1: The image processing rate is greater than the image extraction rate.

[0070] In one example, when the image processing rate is greater than the image extraction rate, the sliding step size of the image sampling window is adjusted based on the image extraction rate and the window length of the image sampling window.

[0071] The adjusted sliding step size is the ratio of the image sampling window length to the image extraction rate. The image sampling window length refers to the number of images that the image sampling window can acquire at one time.

[0072] For example, if the image extraction rate of the image extractor is v1, then the time for the image extractor to extract one image frame is c = 1 / v1. If the image processing rate of the image processing model is v2, then the time for the image processing model to process one image frame is r = 1 / v2. The window length of the image sampling window is w. If v2 > v1 (i.e., c < r), the sliding step size of the image sampling window is d = w / v1 = cw.

[0073] For example, such as Figure 3 As shown, the image extraction rate of the image extractor is v1 = 30 frames / s, the image processing rate of the image processing model is 40 frames / s, and the window length of the image sampling window is w = 4 frames. Then, the time to extract one frame is c = 1 / 30, the time to process one frame is r = 1 / 40, r < c, the moving distance of the sliding window is d = w / v2 = cw = 4 / 30 frames, and the image sampling window a moves a distance of 4 / 30 frames to reach the position of the image sampling window b.

[0074] Case 2: Image processing rate equals image extraction rate.

[0075] In one example, when the image processing rate equals the image extraction rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0076] The adjusted sliding step size is equal to the window length.

[0077] For example, if the image extraction rate of the image extractor is v1, then the time to extract one image frame is c = 1 / v1. If the image processing rate of the image processing model is v2, then the time to process one image frame is r = 1 / v2. The window length of the image sampling window is w. If v1 = v2 (i.e., c = r), then the sliding step size of the image sampling window is d = w.

[0078] For example, such as Figure 4 As shown, the image extraction rate of the image extractor is v1 = 30 frames / s, the image processing rate of the image processing model is v2 = 30 frames / s, the window length of the image sampling window is w = 4 frames, then the time to extract 1 frame is c = 1 / 30, the time to process 1 frame is r = 1 / 30, r = c, the moving distance of the sliding window is d = w = 4 frames, and the image sampling window a moves a distance of 4 frames to reach the position of the image sampling window b.

[0079] Case 3: The image processing rate is less than the image extraction rate.

[0080] In one example, when the image processing rate is less than the image extraction rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0081] The adjusted sliding step size is N times the window length, where N is greater than or equal to 2.

[0082] For example, if the image extraction rate of the image extractor is v1, then the time for the image extractor to extract one image frame is c = 1 / v1. If the image processing rate of the image processing model is v2, then the time to process one image frame is r = 1 / v2. The window length of the image sampling window is w, N = 2. If v1 > v2 (i.e., c < r), the sliding step size of the image sampling window is d = 2w.

[0083] Furthermore, after extracting multiple frames from the image frame sequence based on the adjusted sliding step size, some images in the multiple frames are deleted, and the remaining images are sequentially arranged to form an image frame sequence which is then input into the image processing model. See Implementation Example 2 for details.

[0084] Based on the above technical means, this application can change the number of images input to the image processing model by adjusting the sliding step size of the image sampling window, thereby matching the processing rate of the image processing model and realizing the full utilization of the computing resources of the image processing model.

[0085] In another embodiment (Embodiment 2), when the image processing rate is less than the image extraction rate, after adjusting the sliding step size of the image sampling window, the method of this application embodiment may further include: deleting the target image in multiple frames of images based on a preset deletion rule.

[0086] The preset deletion rules include at least one of rule 1 to rule 3.

[0087] Rule 1: Delete images with a resolution lower than the preset resolution.

[0088] In one example, such as Figure 5 As shown, with the adjusted sliding step size d being twice the window length and the preset resolution being 1980*1080 pixels, image sampling window a slides twice the window length to the position of image sampling window b, deleting image frames with a preset resolution lower than 1980*1080 pixels. Image frames 2, 4, 6, and 8 have resolutions less than 1980*1080 pixels, while the remaining image frames are all greater than or equal to 1980*1080 pixels. Therefore, image frames 2, 4, 6, and 8 are deleted. The image frame sequence of the input image processing model then consists of image frames 1, 3, 5, and 7.

[0089] Rule 2: Delete images whose similarity to other images is higher than the preset similarity.

[0090] In one example, such as Figure 5 As shown, with the adjusted sliding step size being twice the window length and a preset similarity of 80%, image sampling window a slides twice the window length to the position of image sampling window b. Image frame 2 has a similarity greater than 80% with image frame 1, therefore image frame 2 is deleted. Image frame 3 has a similarity less than 80% with image frame 1, therefore image frame 3 is not deleted. Image frame 4 has a similarity greater than 80% with image frame 3, therefore image frame 4 is deleted. This process of deleting images continues until the end of image sampling window b. Therefore, the image frame sequence input to the image processing model consists of image frame 1, image frame 3, image frame 5, and image frame 7.

[0091] Rule 3: Extract at least one target image from multiple frames of images at preset intervals, and delete at least one target image.

[0092] In one example, such as Figure 5 As shown, with the adjusted sliding step size being twice the window length and the preset interval being 1, image sampling window a slides twice the window length to the position of image sampling window b, and the deleted images are image frame 2, image frame 4, image frame 6, and image frame 8, respectively. Therefore, the image frame sequence input to the image processing model consists of image frame 1, image frame 3, image frame 5, and image frame 7.

[0093] Based on the above technical means, this application can delete some images according to preset deletion rules, thereby reducing the number of images input to the image processing model per unit time and avoiding the phenomenon of image backlog in the input image processing model.

[0094] In another embodiment (Example 3), as Figure 6 As shown, an image processing method is provided in an embodiment of this application, the method including S601-S603.

[0095] S601, The image extractor acquires the preprocessed original video, performs image extraction service, and obtains the first image frame sequence.

[0096] The first image frame sequence consists of multiple image frames from the original video. Image preprocessing includes conventional data preprocessing procedures such as data cleaning, image enhancement, and background separation; for details, please refer to existing technologies.

[0097] In one example, such as Figure 7 As shown, when the original video is a video with a frame rate of 60 frames per second and the preset image extraction rate is 30 frames per second, the image extraction service acquires the pre-processed original video and performs image extraction processing on it. The image extraction service outputs the corresponding image frame sequence of the video at a rate of 30 frames per second according to the preset image extraction rate.

[0098] S602, The image extractor samples the first image frame sequence to obtain multiple second image frame sequences.

[0099] The image sampling service is an image frame sequence processing service based on an image sampling window. The second image frame sequence is the image frame sequence obtained through the image sampling service and is used as input to the image processing model.

[0100] In one example, such as Figure 7As shown, the image sampling service acquires the preprocessed image frame sequence, performs sampling processing on the preprocessed image frame sequence according to the preset window size and sliding step size, obtains multiple images for input image processing model, and arranges these multiple images according to the extraction order to obtain the image frame sequence.

[0101] In the image sampling service, the sliding distance of the image sampling window is periodically adjusted by the parameter configuration terminal. The parameter configuration terminal can receive the sliding step size of the sliding window sent by the scheduler and adjust the sliding step size parameter in the image sampling service to the latest received sliding step size.

[0102] S603. The image processing model receives the second image frame sequence and processes the second image frame sequence.

[0103] In one example, such as Figure 7 As shown, the image processing model receives a second image frame sequence as input data and processes the sampled image frame sequence. This image frame sequence contains multiple images. After the image processing model completes processing one image, it records the completion time in the work log.

[0104] The work log can be retrieved by the log terminal at preset intervals to analyze the image processing rate of the image processing model within a preset time period. The images processed by the image processing model can then be provided for use in other subsequent services.

[0105] Based on the above technical features, this application can add an image sampling service to the image frame sequence of the input image processing model. Since changing the image extraction service is more complex than changing the image sampling service, the number of image frames in the input image processing model can be matched with the processing rate of the image processing model by simply changing the image sampling service, without having to adjust the more complex image extraction service.

[0106] It is understood that the above-described image processing method can be implemented by an image processing device. To achieve the above functions, the image processing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments disclosed in this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments disclosed in this application.

[0107] The embodiments disclosed in this application can divide the image processing apparatus generated according to the above method examples into functional modules. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in the embodiments disclosed in this application is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0108] Figure 8 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. Figure 8 As shown, the image processing device 80 can be used to perform... Figures 1-7 The image processing method shown is illustrated. The image processing apparatus 80 includes a communication unit 801 and a processing unit 802.

[0109] The communication unit 801 is used to acquire the image extraction rate of the image extractor and the image processing rate of the image processing model; the image extractor is used to extract images from the video to be processed.

[0110] The processing unit 802 is used to adjust the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window is matched with the image processing rate of the image processing model.

[0111] In one possible implementation, the processing unit 802 is further configured to adjust the sliding step size of the image sampling window according to the image processing rate and / or the window length of the image sampling window.

[0112] In one possible implementation, the processing unit 802 is further configured to adjust the sliding step size of the image sampling window according to the image sampling rate and the window length of the image sampling window when the image processing rate is greater than the image extraction rate; the adjusted sliding step size is the ratio of the window length to the image extraction rate.

[0113] In one possible implementation, the processing unit 802 is further configured to adjust the sliding step size of the image sampling window based on the window length of the image sampling window when the image processing rate is less than or equal to the image extraction rate.

[0114] In one possible implementation, the processing unit 802 is further configured to, when the image processing rate is equal to the image extraction rate, adjust the sliding step size to be equal to the window length; and when the image processing rate is less than the image extraction rate, adjust the sliding step size to be N times the window length, where N is greater than or equal to 2.

[0115] In one possible implementation, the processing unit 802 is further configured to delete some images from the multiple frames of images after extracting multiple frames of images from the image frame sequence based on the adjusted sliding step size, and input the deleted images into the image processing model.

[0116] In one possible implementation, the processing unit 802 is further configured to delete target images from multiple frames of images based on preset deletion rules; the preset deletion rules include at least one of the following: deleting images with a resolution lower than a preset resolution; deleting images with a similarity to other images higher than a preset similarity; extracting at least one target image from multiple frames of images at preset intervals and deleting at least one target image.

[0117] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0118] This disclosure also provides a computer-readable storage medium storing instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the image processing method provided in the embodiments of this disclosure described above.

[0119] This disclosure also provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the image processing method provided in the above-described embodiments of this disclosure.

[0120] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires; portable computer disks; hard disks; random access memory (RAM); read-only memory (ROM); erasable programmable read-only memory (EPROM); registers; hard disks; optical fibers; portable compact disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination thereof; or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may reside in an application-specific integrated circuit (ASIC). In the embodiments of this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized by, The method comprises: obtaining an image extraction rate of an image extractor and an image processing rate of an image processing model; the image extractor is used to extract images from a video to be processed; based on the image extraction rate and the image processing rate, adjusting a sliding step of an image sampling window of an image frame sequence; the image frame sequence comprises multiple images extracted by the image extractor; the number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model; the method further comprises: in a case where the image processing rate is greater than the image extraction rate, adjusting the sliding step of the image sampling window according to the image extraction rate and the window length of the image sampling window; the adjusted sliding step is a ratio of the window length to the image extraction rate; in a case where the image processing rate is less than or equal to the image extraction rate, adjusting the sliding step of the image sampling window based on the window length of the image sampling window; the adjusted sliding step is greater than or equal to the window length.

2. The method of claim 1, wherein in a case where the image processing rate is equal to the image extraction rate, the adjusted sliding step is equal to the window length; in a case where the image processing rate is less than the image extraction rate, the adjusted sliding step is N times the window length, and N is greater than or equal to 2.

3. The method of claim 2, wherein, in a case where the adjusted sliding step is N times the window length, the method further comprises: after extracting the multiple images from the image frame sequence based on the adjusted sliding step, deleting part of the multiple images and inputting the deleted images into the image processing model.

4. The method of claim 3, wherein, the method further comprises: based on a preset deletion rule, deleting a target image in the multiple images; the preset deletion rule comprises at least one of the following: deleting an image whose resolution is lower than a preset resolution; deleting an image whose image similarity with other images is higher than a preset similarity; extracting at least one target image from the multiple images at a preset interval, and deleting the at least one target image.

5. An image processing apparatus characterized by comprising: The device comprises a communication unit and a processing unit; the communication unit is configured to obtain an image extraction rate of an image extractor and an image processing rate of an image processing model; the image extractor is used to extract images from a video to be processed; the processing unit is configured to adjust a sliding step of an image sampling window of an image frame sequence according to the image extraction rate and the image processing rate; the image frame sequence comprises multiple images extracted by the image extractor; the number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model; The processing unit is further configured to, when the image processing rate is greater than the image extraction rate, adjust a sliding step of the image sampling window according to the image extraction rate and a window length of the image sampling window; and the adjusted sliding step is a ratio of the window length and the image extraction rate. The processing unit is further configured to, when the image processing rate is less than or equal to the image extraction rate, adjust a sliding step of the image sampling window based on a window length of the image sampling window; and the adjusted sliding step is greater than or equal to the window length.

6. A computer readable storage medium characterized by, The computer readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer executes the image processing method as claimed in any one of claims 1-4.

7. A computer program product, characterised in that, The computer program product comprises computer program instructions, and when the computer program instructions are executed by a processor, the image processing method as claimed in any one of claims 1-4 is implemented.

Images