Image processing method and device and storage medium

Abstract

The invention provides an image processing method and device and a storage medium, relates to the technical field of artificial intelligence, and can determine the sliding distance of a sliding window in a data processing process according to the processing rate of an image recognition model for image frames and the frame extraction rate in a data processing stage, so that an input image frame sequence is changed, and the image processing efficiency is improved. And full utilization of the computing power of the image recognition model is realized. The method comprises the following steps: acquiring an image extraction rate of an image extractor and an image processing rate of an image processing model; the image extractor is used for extracting an image from a to-be-processed video; adjusting a sliding step length of an image sampling window of the image frame sequence according to the image extraction rate and the image processing rate; the image frame sequence comprises multiple frames of images extracted by the image extractor; the number of the images collected by the adjusted image sampling window is matched with the image processing rate of the image processing model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image processing method, device and storage medium. Background Art

[0002] With the development of artificial intelligence and deep learning technology, the performance of image processing models has been significantly improved. Deep learning algorithms such as convolutional neural networks (CNN) can autonomously learn and extract complex features of images, thereby achieving efficient recognition of image content.

[0003] Currently, the input preprocessing of image processing models usually uses a fixed window size for data frame processing. However, since the performance of the computing equipment deployed with the image processing model usually cannot maintain a stable constant value, as the performance of the computing equipment fluctuates, the image processing model will waste resources and have low utilization rate during its operation. Summary of the invention

[0004] The present application provides an image processing method, device and storage medium, which can fully utilize the computing power resources of the image processing model by changing the sliding distance of image sampling.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides an image processing method, the method comprising:

[0007] The image extraction rate of the image extractor and the image processing rate of the image processing model are obtained; the image extractor is used to extract images from the video to be processed; based on the image extraction rate and the image processing rate, the sliding step size of the image sampling window of the image frame sequence is adjusted; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model.

[0008] According to the above-mentioned technical means, the present application can change the number of images input into the image processing model by adjusting the sliding step size of the image sampling window, thereby matching the processing rate of the image processing model and achieving full utilization of the computing power resources of the image processing model.

[0009] In a possible implementation manner, the sliding step size of the image sampling window is adjusted according to the image processing rate and / or the window length of the image sampling window.

[0010] According to the above technical means, the present application can obtain the sliding step size of the image sampling window, and the number of images collected by the image sampling window through the sliding step size can match the image processing rate.

[0011] In one possible implementation, when the image processing rate is greater than the image decimation rate, the sliding step size of the image sampling window is adjusted according to the image decimation rate and the window length of the image sampling window; the adjusted sliding step size is the ratio of the window length to the image decimation rate.

[0012] According to the above-mentioned technical means, the present application can obtain the sliding step of the image sampling window when the image processing rate is greater than the image extraction rate. Through this sliding step, the image sampling window can input an image into the image processing model multiple times, thereby making full use of the idle resources of the image processing model, and at the same time improving the accuracy of the image processing model.

[0013] In a possible implementation, when the image processing rate is less than or equal to the image decimation rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0014] According to the above-mentioned technical means, the present application can obtain the sliding step size of the image sampling window when the image processing rate is less than or equal to the image extraction rate. The number of images collected by the image sampling window through the sliding step size can match the processing rate of the image processing model, thereby avoiding the accumulation of input data.

[0015] In one possible implementation, when the image processing rate is equal to the image extraction rate, the adjusted sliding step size is equal to the window length; when the image processing rate is less than the image extraction rate, the adjusted sliding step size is N times the window length, where N is greater than or equal to 2.

[0016] According to the above technical means, the present application can respectively obtain the sliding step size of the image sampling window where the image processing rate is equal to the image extraction rate or the image processing rate is less than the image extraction rate, so that the number of images collected by the image sampling window matches the processing rate of the image processing model.

[0017] In a possible implementation, after extracting multiple frames of images from the image frame sequence based on the adjusted sliding step size, part of the images in the multiple frames of images are deleted, and the deleted images are input into the image processing model.

[0018] According to the above-mentioned technical means, the present application can reduce the number of images input into the image processing model per unit time by deleting some images when the image processing rate is lower than the image extraction rate, so that the number of images input into the image processing model per unit time matches the image processing rate.

[0019] In a possible implementation, a target image in multiple frames of images is deleted based on a preset deletion rule; the preset deletion rule includes at least one of the following: deleting images whose image resolution is lower than a preset resolution; deleting images whose image similarity with other images is higher than a preset similarity; extracting at least one frame of the target image from multiple frames of images at a preset interval, and deleting at least one frame of the target image.

[0020] According to the above technical means, the present application can delete unnecessary images according to preset deletion rules, and can also minimize the errors caused by deleting images while reducing the image processing pressure of the image processing model.

[0021] In a second aspect, the present application provides an image processing device, the device comprising:

[0022] A communication unit, used for acquiring an image extraction rate of an image extractor and an image processing rate of an image processing model; the image extractor is used for extracting an image from a video to be processed;

[0023] The processing unit is used to adjust the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model.

[0024] In a possible implementation manner, the processing unit is further configured to adjust the sliding step size of the image sampling window according to the image processing rate and / or the window length of the image sampling window.

[0025] In one possible implementation, the processing unit is also used to adjust the sliding step size of the image sampling window according to the image decimation rate and the window length of the image sampling window when the image processing rate is greater than the image decimation rate; the adjusted sliding step size is the ratio of the window length to the image decimation rate.

[0026] In a possible implementation, the processing unit is further configured to adjust the sliding step size of the image sampling window based on the window length of the image sampling window when the image processing rate is less than or equal to the image extraction rate.

[0027] In one possible implementation, the processing unit is also used to adjust the sliding step size to be equal to the window length when the image processing rate is equal to the image extraction rate; when the image processing rate is less than the image extraction rate, the adjusted sliding step size is N times the window length, and N is greater than or equal to 2.

[0028] In a possible implementation, the processing unit is further used to delete part of the images in the multiple frames after extracting multiple frames from the image frame sequence based on the adjusted sliding step size, and input the deleted images into the image processing model.

[0029] In one possible implementation, the processing unit is further used to delete the target image from the multiple frames of images based on a preset deletion rule; the preset deletion rule includes at least one of the following: deleting images whose image resolution is lower than a preset resolution; deleting images whose image similarity to other images is higher than a preset similarity; extracting at least one frame of the target image from the multiple frames of images at a preset interval, and deleting at least one frame of the target image.

[0030] In a third aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a terminal, the terminal executes the image processing method described in the first aspect and any possible implementation of the first aspect.

[0031] In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed on an image processing device, enables the image processing device to execute the image processing method as described in the first aspect and any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0033] Figure 2 A process of an image processing method provided in an embodiment of the present application Figure 1 ;

[0034] Figure 3 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 1 ;

[0035] Figure 4 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 2 ;

[0036] Figure 5 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 3 ;

[0037] Figure 6 A process of an image processing method provided in an embodiment of the present application Figure 2 ;

[0038] Figure 7 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 4 ;

[0039] Figure 8 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The image processing method, device and storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0041] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0042] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0043] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0044] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0045] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0046] The following explains the terms involved in the embodiments of the present application to facilitate the reader's understanding.

[0047] (1) Image sampling service based on image sampling window: The image sampling service based on image sampling window can also be an image sampling service based on sliding window. The image sampling service samples multiple image frame sequences or continuous video frames by setting a sliding window on multiple continuous image frame sequences or continuous video frames to obtain image frame sequences or continuous video frames. In the present application, image sampling can be performed on the image frame sequence that has passed through the image extraction service according to the preset image sampling window size, sliding step size, and preset activity rules to obtain the image frame sequence that is input into the image processing model.

[0048] The above is a brief introduction to some concepts involved in the embodiments of this application.

[0049] Currently, the input preprocessing of image processing models usually uses a fixed window size for data frame processing. However, since the performance of the computing equipment deployed with the image processing model usually cannot maintain a stable constant value, as the performance of the computing equipment fluctuates, the image processing model will waste resources and have low utilization rate during its operation.

[0050] In view of this, an embodiment of the present application provides an image processing method, which can determine the sliding distance of the sliding window in the data processing process according to the processing rate of the image recognition model for the image frames and the frame extraction rate in the data processing stage, so that the input image frame sequence changes, thereby achieving full utilization of the computing power of the image recognition model.

[0051] For example, Figure 1 1 shows a schematic diagram of the structure of an image processing system 10 provided in an embodiment of the present application. The image processing system 10 may include at least one scheduler 101, at least one log terminal 102, at least one image processing model 103, at least one image extractor 104, and at least one parameter configuration terminal 105. The scheduler 101, the log terminal 102, the image processing model 103, the image extractor 104, and the parameter configuration terminal 105 may be communicatively connected. Figure 1 Only one scheduler 101, one log terminal 102, one image processing model 103, one image extractor 104 and one parameter configuration terminal 105 are shown. The embodiment of the present application does not impose any restriction on the number of schedulers 101, log terminals 102, image processing models 103, image extractors 104 and parameter configuration terminals 105.

[0052] The scheduler 101 is used to receive the image processing rate and obtain the image decimation rate, and is also used to determine the sliding step of the sampling window according to the relationship between the image processing rate and the image decimation rate, and send the sliding step of the sampling window to the parameter configuration terminal 105 .

[0053] The log terminal 102 is used to obtain the work log of the image processing model and to determine the image processing rate of the image processing model according to the work log of the image processing model and send the image processing rate to the scheduler 101 .

[0054] The image processing model 103 is used to process the received image frame sequence and output the processing result.

[0055] The image extractor 104 is used to perform image extraction processing on the original video and / or multiple images to obtain a segment of image frame sequence for image sampling, and is also used to sample the image frame sequence to obtain multiple segments of image frame sequences for inputting into the image processing model.

[0056] The parameter configuration terminal 105 is used to receive the sliding step size of the image sampling window and adjust the sliding step size of the image sampling window in the image extractor.

[0057] The application scenarios of the embodiments of the present application are not limited. The system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0058] like Figure 2 As shown, the embodiment of the present application proposes an image processing method, which includes:

[0059] S201. The scheduler obtains the image extraction rate of the image extractor and the image processing rate of the image processing model.

[0060] The image extraction rate is the number of images extracted per unit time during the process of the image extractor extracting the video to be processed. The image processing rate is the number of images that can be processed by the image processing model per unit time. The image processing model can be an image recognition model or an image classification model or other artificial intelligence model related to image processing.

[0061] In one example, the image extraction rate is a preset fixed value, which can be obtained by configuring preset parameters. The image processing rate can be obtained by analyzing the work log of the image processing model within a preset time period at every preset period. For example, the image processing rate can be 30 frames / s or 60 frames / s, which is not limited here.

[0062] The working log of the image processing model includes the time when multiple image processing is completed. The log terminal can determine the image processing rate of the image processing model according to the time when multiple image processing is completed within the preset time period. The preset time period and the preset cycle can be set according to the needs of the user, which can be 1 minute, 10 minutes, or other values ​​without limitation.

[0063] S202: The scheduler adjusts the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate.

[0064] The image frame sequence includes multiple frames of images extracted by the image extractor. The number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model. The sliding step size of the image sampling window can also be the moving distance of the starting position of the image sampling window.

[0065] In one example, the sliding step size of the image sampling window is adjusted according to the image decimation rate and / or the window length of the image sampling window.

[0066] Specifically, when the image processing rate is greater than the image decimation rate, the sliding step size of the image sampling window is adjusted according to the image decimation rate and the window length of the image sampling window.

[0067] When the image processing rate is less than or equal to the image decimation rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0068] For details, please refer to Case 1 to Case 3.

[0069] Case 1: The image processing rate is greater than the image extraction rate.

[0070] In one example, when the image processing rate is greater than the image decimation rate, the sliding step size of the image sampling window is adjusted based on the image decimation rate and the window length of the image sampling window.

[0071] The adjusted sliding step size is the ratio of the window length of the image sampling window to the image decimation rate. The window length of the image sampling window refers to the number of images that can be collected by the image sampling window at one time.

[0072] For example, if the image extraction rate of the image extractor is v1, then the time it takes for the image extractor to extract an image frame is c = 1 / v1. If the image processing rate of the image processing model is v2, then the time it takes for the image processing model to process an image frame is r = 1 / v2. The window length of the image sampling window is w. If v2>v1 (that is, c<r), the sliding step length of the image sampling window is d = w / v1 = cw.

[0073] For example, Figure 3 As shown, the image extraction rate of the image extractor is v1 = 30 frames / s, the image processing rate of the image processing model is 40 frames / s, and the window length of the image sampling window is w = 4 frames. The time for extracting one frame is c = 1 / 30, and the time for processing one frame is r = 1 / 40, r < c. The moving distance of the sliding window is d = w / v2 = cw = 4 / 30 frames, and the image sampling window a slides a distance of 4 / 30 frames to reach the position of the image sampling window b.

[0074] Case 2: The image processing rate is equal to the image decimation rate.

[0075] In one example, when the image processing rate is equal to the image extraction rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0076] Among them, the adjusted sliding step size is equal to the window length.

[0077] For example, if the image extraction rate of the image extractor is v1, then the time to extract an image frame is c = 1 / v1. If the image processing rate of the image processing model is v2, then the time to process an image frame is r = 1 / v2. The window length of the image sampling window is w. If v1 = v2 (that is, c = r), the sliding step length of the image sampling window is d = w.

[0078] For example, Figure 4 As shown, the image extraction rate of the image extractor is v1 = 30 frames / s, the image processing rate of the image processing model is v2 = 30 frames / s, and the window length of the image sampling window is w = 4 frames. The time for extracting one frame is c = 1 / 30, the time for processing one frame is r = 1 / 30, r = c, the moving distance of the sliding window is d = w = 4 frames, and the image sampling window a slides a distance of 4 frames to reach the position of the image sampling window b.

[0079] Case 3: The image processing rate is less than the image extraction rate.

[0080] In one example, when the image processing rate is less than the image extraction rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

[0081] The adjusted sliding step size is N times the window length, where N is greater than or equal to 2.

[0082] For example, if the image extraction rate of the image extractor is v1, then the time for the image extractor to extract an image frame is c = 1 / v1. If the image processing rate of the image processing model is v2, then the time for processing an image frame is r = 1 / v2. The window length of the image sampling window is w, N = 2, if v1>v2 (that is, c<r), the sliding step length of the image sampling window is d = 2w.

[0083] Furthermore, after extracting multiple frames of images from the image frame sequence based on the adjusted sliding step size, some images in the multiple frames of images are deleted, and the remaining images are sequentially composed into an image frame sequence and input into the image processing model.

[0084] According to the above-mentioned technical means, the present application can change the number of images input into the image processing model by adjusting the sliding step size of the image sampling window, thereby matching the processing rate of the image processing model and achieving full utilization of the computing power resources of the image processing model.

[0085] In another embodiment (embodiment two), when the above-mentioned image processing rate is less than the image extraction rate, after adjusting the sliding step size of the image sampling window, the method of the embodiment of the present application may also include: deleting the target image in the multiple frames of images based on a preset deletion rule.

[0086] The preset deletion rule includes at least one of rule 1 to rule 3.

[0087] Rule 1: Delete images whose resolution is lower than the preset resolution.

[0088] In one example, Figure 5 As shown in the figure, when the adjusted sliding step d is twice the window length and the preset resolution is 1980*1080 pixels, the image sampling window a slides twice the window length to the position of the image sampling window b, and the image frames with a preset resolution lower than 1980*1080 pixels are deleted. The resolutions of image frames 2, 4, 6, and 8 are less than 1980*1080 pixels, and the remaining image frames are greater than or equal to 1980*1080 pixels. Therefore, image frames 2, 4, 6, and 8 are deleted. Then the image frame sequence of the input image processing model consists of image frame 1, image frame 3, image frame 5, and image frame 7.

[0089] Rule 2: Delete images whose image similarity with other images is higher than a preset similarity.

[0090] In one example, Figure 5 As shown, when the adjusted sliding step size is twice the window length and the preset similarity is 80%, the image sampling window a slides twice the window length to the position of the image sampling window b, and the similarity between image frame 2 and image frame 1 is greater than 80%, so image frame 2 is deleted. The similarity between image frame 3 and image frame 1 is less than 80%, so image frame 3 is not deleted. The similarity between image frame 4 and image frame 3 is greater than 80%, so image frame 4 is deleted. The images are deleted in sequence until the end position of the image sampling window b. Therefore, the image frame sequence input to the image processing model consists of image frame 1, image frame 3, image frame 5, and image frame 7.

[0091] Rule 3: Extract at least one target image from multiple frames of images at a preset interval and delete at least one target image.

[0092] In one example, Figure 5 As shown in the figure, when the adjusted sliding step is twice the window length and the preset interval is 1, the image sampling window a slides twice the window length to the position of the image sampling window b, and the deleted images are image frame 2, image frame 4, image frame 6, and image frame 8. Therefore, the image frame sequence input to the image processing model consists of image frame 1, image frame 3, image frame 5, and image frame 7.

[0093] According to the above technical means, the present application can delete some images according to preset deletion rules, so that the number of images input into the image processing model per unit time is reduced, thereby avoiding the phenomenon of image backlog input into the image processing model.

[0094] In another embodiment (Embodiment 3), Figure 6 As shown, an image processing method is provided for an embodiment of the present application, and the method includes S601-S603.

[0095] S601: An image extractor obtains an original video that has undergone data preprocessing, performs image extraction service, and obtains a first image frame sequence.

[0096] The first image frame sequence is composed of multiple image frames of the original video. Image preprocessing includes conventional data preprocessing processes such as data cleaning, image enhancement, and background separation. For details, reference may be made to the prior art.

[0097] In one example, Figure 7 As shown in the figure, when the original video is a video with a frame rate of 60 frames / s and the preset image extraction rate is 30 frames / s, the image extraction service obtains the original video after data preprocessing and performs image extraction on the original video. The image extraction service outputs the image frame sequence corresponding to the video at a rate of 30 frames / s according to the preset image extraction rate.

[0098] S602: An image extractor performs sampling processing on a first image frame sequence to obtain a plurality of second image frame sequences.

[0099] The image sampling service is an image frame sequence processing service based on an image sampling window. The second image frame sequence is an image frame sequence obtained through the image sampling service and is used to input into the image processing model.

[0100] In one example, Figure 7As shown, the image sampling service obtains the preprocessed image frame sequence, samples the preprocessed image frame sequence according to a preset window size and a sliding step size, obtains a plurality of images for inputting an image processing model, and arranges the plurality of images according to an extraction order to obtain an image frame sequence.

[0101] The sliding distance of the image sampling window in the image sampling service is periodically adjusted by the parameter configuration terminal. The parameter configuration terminal can receive the sliding step of the sliding window sent by the scheduler, and adjust the sliding step parameter in the image sampling service to the latest received sliding step.

[0102] S603: The image processing model receives the second image frame sequence and processes the second image frame sequence.

[0103] In one example, Figure 7 As shown, the image processing model receives the second image frame sequence as the input data of the model to process the sampled image frame sequence. The image frame sequence includes multiple images. After the image processing model completes processing of an image, the time of completion of the processing is recorded in the work log.

[0104] The work log can be obtained by the log terminal after a preset period, and used to analyze and obtain the image processing rate of the image processing model within a preset time period. The image processed by the image processing model can be provided to other subsequent services.

[0105] According to the above technical features, the present application can add an image sampling service to the image frame sequence of the input image processing model. Since the change of the image extraction service is more complicated than the image sampling service, the number of image frames of the input image processing model can be matched with the processing rate of the image processing model only by changing the image sampling service, without adjusting the more complicated image extraction service.

[0106] It is understandable that the above-mentioned image processing method can be implemented by an image processing device. In order to realize the above-mentioned functions, the image processing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the modules and algorithm steps of each example described in the embodiments disclosed herein, the embodiments disclosed in the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments disclosed in this application.

[0107] The embodiments disclosed in the present application can divide the functional modules of the image processing device generated by the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments disclosed in the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0108] Figure 8 Schematic diagram of the structure of an image processing device provided in an embodiment of the present application. Figure 8 As shown, the image processing device 80 can be used to perform Figure 1-Figure 7 The image processing method shown. The image processing device 80 includes: a communication unit 801 and a processing unit 802.

[0109] The communication unit 801 is used to obtain the image extraction rate of the image extractor and the image processing rate of the image processing model; the image extractor is used to extract images from the video to be processed;

[0110] The processing unit 802 is used to adjust the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model.

[0111] In a possible implementation, the processing unit 802 is further configured to adjust the sliding step size of the image sampling window according to the image processing rate and / or the window length of the image sampling window.

[0112] In one possible implementation, the processing unit 802 is also used to adjust the sliding step size of the image sampling window according to the image decimation rate and the window length of the image sampling window when the image processing rate is greater than the image decimation rate; the adjusted sliding step size is the ratio of the window length to the image decimation rate.

[0113] In a possible implementation, the processing unit 802 is further configured to adjust the sliding step size of the image sampling window based on the window length of the image sampling window when the image processing rate is less than or equal to the image decimation rate.

[0114] In one possible implementation, the processing unit 802 is also used to adjust the sliding step size to be equal to the window length when the image processing rate is equal to the image extraction rate; when the image processing rate is less than the image extraction rate, the adjusted sliding step size is N times the window length, and N is greater than or equal to 2.

[0115] In a possible implementation, the processing unit 802 is further configured to delete part of the images in the multiple frames after extracting the multiple frames from the image frame sequence based on the adjusted sliding step size, and input the deleted images into the image processing model.

[0116] In one possible implementation, the processing unit 802 is further used to delete the target image from the multiple frames of images based on a preset deletion rule; the preset deletion rule includes at least one of the following: deleting images whose image resolution is lower than a preset resolution; deleting images whose image similarity with other images is higher than a preset similarity; extracting at least one frame of the target image from the multiple frames of images at a preset interval, and deleting at least one frame of the target image.

[0117] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0118] The present disclosure also provides a computer-readable storage medium having instructions stored thereon. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method provided by the above-mentioned embodiment of the present disclosure.

[0119] The embodiments of the present disclosure also provide a computer program product including instructions, which, when executed on an electronic device, enables the electronic device to execute the image processing method provided by the embodiments of the present disclosure.

[0120] Among them, the computer readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer readable storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (ASIC). In the embodiments of the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image extraction rate of an image extractor and an image processing rate of an image processing model; the image extractor is used to extract images from a video to be processed; Based on the image extraction rate and the image processing rate, the sliding step size of the image sampling window of the image frame sequence is adjusted; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model.

2. The method according to claim 1, characterized in that The step of adjusting the sliding step of the image sampling window of the image frame sequence includes: The sliding step size of the image sampling window is adjusted according to the image decimation rate and / or the window length of the image sampling window.

3. The method according to claim 2, characterized in that The adjusting the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate comprises: When the image processing rate is greater than the image decimation rate, the sliding step size of the image sampling window is adjusted according to the image decimation rate and the window length of the image sampling window; the adjusted sliding step size is the ratio of the window length to the image decimation rate.

4. The method according to claim 2, characterized in that: The adjusting the sliding step size of the image sampling window of the image frame sequence based on the image extraction rate and the image processing rate comprises: When the image processing rate is less than or equal to the image extraction rate, the sliding step size of the image sampling window is adjusted based on the window length of the image sampling window.

5. The method according to claim 4, characterized in that When the image processing rate is equal to the image extraction rate, the adjusted sliding step size is equal to the window length; When the image processing rate is lower than the image extraction rate, the adjusted sliding step size is N times the window length, where N is greater than or equal to 2.

6. The method according to claim 5, characterized in that When the adjusted sliding step size is N times the window length, the method further includes: After extracting multiple frames of images from the image frame sequence based on the adjusted sliding step size, some images in the multiple frames of images are deleted, and the deleted images are input into the image processing model.

7. The method according to claim 6, characterized in that Deleting some images from the multiple frames of images includes: Based on a preset deletion rule, the target image in the multiple frames of images is deleted; the preset deletion rule includes at least one of the following: Delete images whose resolution is lower than the preset resolution; Deleting images whose image similarity with other images is higher than a preset similarity; At least one frame of target image is extracted from the multiple frames of image at a preset interval, and the at least one frame of target image is deleted.

8. An image processing device, characterized in that: The device comprises: a communication unit and a processing unit; The communication unit is used to obtain the image extraction rate of the image extractor and the image processing rate of the image processing model; the image extractor is used to extract images from the video to be processed; The processing unit is used to adjust the sliding step size of the image sampling window of the image frame sequence according to the image extraction rate and the image processing rate; the image frame sequence includes multiple frames of images extracted by the image extractor; the number of images collected by the adjusted image sampling window matches the image processing rate of the image processing model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the image processing method as claimed in any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises computer program instructions, and when the computer program instructions are executed by a processor, the image processing method as claimed in any one of claims 1 to 7 is implemented.

Images