A chip-based image processing method, device, equipment and storage medium

By introducing an image pyramid module to process images in the vehicle autonomous driving chip, a target area of ​​interest that meets the preset size is generated, which solves the problem of excessive computing power for chip processing images, and improves the ability to turn on the autonomous driving function at the same time.

CN116309415BActive Publication Date: 2024-11-22BEIJING PHIGENT TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310219597.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-01
Publication Date
2024-11-22
Estimated Expiration
2043-03-01

AI Technical Summary

Technical Problem

In the prior art, the vehicle's autonomous driving chip occupies too much processor computing power during the process of processing pictures, resulting in insufficient computing of other functions, which may lead to chip damage.

Method used

By introducing an image pyramid module into the terminal, the target object's area of ​​interest in the original image is determined and inputted to the image pyramid module for processing, and a target area of ​​interest that meets the preset size is generated, thereby reducing the computing burden of the chip.

Benefits of technology

It effectively reduces the processor computing burden of the chip, improves the ability to turn on the automatic driving function while avoiding the risk of chip overload and damage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116309415B_ABST
    Figure CN116309415B_ABST
Patent Text Reader

Abstract

The embodiment of the present invention provides a chip-based image processing method, device, equipment and storage medium, the method comprising: inputting an original image into a chip, determining a first region of interest including a target object in the original image through the chip; inputting the original image into an image pyramid module, generating at least one second region of interest including the target object through the image pyramid module; and obtaining a target region of interest that meets a preset size from the second region of interest according to the first region of interest; performing preset image processing according to the target object in the image corresponding to the target region of interest. After the present invention determines the first region of interest of the target object through the chip, the original image can also be processed through an image pyramid module outside the chip to obtain an image corresponding to the target region of interest, so as to solve the problem that the chip needs to calculate the entire process of processing the image, resulting in a significant occupation of the processor computing power of the chip.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a chip-based image processing method, a chip-based image processing device, an electronic device and a storage medium. Background Art

[0002] With the development of science and technology, vehicles have more and more functions, and the vehicle's automatic driving mode has also appeared in people's lives. In order to improve the performance and experience of the vehicle's automatic driving mode, people are constantly upgrading the automatic driving functions and chips in the vehicle. For example: Horizon chip J5, launched a series of intelligent driving central computers (CPUs) with a computing power of 200-1000T, the industry's highest fps (frameper second) performance, and the lowest power consumption.

[0003] In the prior art, in order to reduce costs, the processor computing power of the chip is reduced. If the chip is used to calculate the entire process of processing images for the vehicle, the processor computing power of the chip will be greatly occupied. As a result, when other functions of autonomous driving need to be calculated through the chip, the chip processor may have insufficient computing power, which may cause damage to the chip. Summary of the invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a chip-based image processing method, a chip-based image processing device, an electronic device and a storage medium that overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problems, an embodiment of the present invention discloses a chip-based image processing method, which is applied to a terminal, and the terminal includes a chip and an image pyramid module:

[0006] Inputting an original image into the chip, and determining a first region of interest including a target object in the original image through the chip;

[0007] Inputting the original image into the image pyramid module, generating at least one second region of interest including a target object through the image pyramid module, wherein the size of at least one second region of interest is different; and obtaining a target region of interest satisfying a preset size from the second region of interest according to the first region of interest;

[0008] Preset image processing is performed according to the target object in the image corresponding to the target region of interest.

[0009] Optionally, the method further comprises: inputting size information of the original image into the chip;

[0010] The determining, by the chip, a first region of interest including a target object in the original image comprises:

[0011] The image region of interest extraction interface is called by the chip to identify the target object in the original image; and a first region of interest having a size smaller than that of the original image and containing the target object is determined.

[0012] Optionally, after inputting the original image into the chip, the method further includes:

[0013] Determining, by the chip, whether the format of the original image is a preset format;

[0014] If the format of the original image is not the preset format, the format conversion interface is called by the chip to convert the format of the original image into the preset format.

[0015] Optionally, generating at least one second region of interest including a target object through the image pyramid module, wherein the size of at least one second region of interest is different; and obtaining a target region of interest satisfying a preset size from the second region of interest according to the first region of interest, comprises:

[0016] generating at least one second region of interest of different sizes including the target object by means of the image pyramid module; if a third region of interest having the same size as the first region of interest exists in the second region of interest, determining whether the size of the third region of interest is consistent with the preset size;

[0017] If the size of the third region of interest is consistent with the preset size, determining the third region of interest as a target region of interest;

[0018] If the size of the third region of interest is inconsistent with the preset size, an image scaling interface is called to scale the size of the image of the third region of interest to the preset size to obtain a target region of interest that meets the preset size.

[0019] Optionally, the generating at least one second region of interest including a target object by the image pyramid module, wherein the size of at least one second region of interest is different; and obtaining a target region of interest satisfying a preset size from the second region of interest according to the first region of interest, further comprises:

[0020] If there is no third region of interest with the same size as the first region of interest in the second regions of interest, determining a corresponding intersection-over-union ratio according to the size of at least one of the second regions of interest and the preset size, and determining a fourth region of interest with the largest corresponding intersection-over-union ratio;

[0021] The size of the fourth region of interest is scaled to the preset size through the image scaling interface to obtain a target region of interest that meets the preset size.

[0022] Optionally, determining the fourth region of interest corresponding to the maximum intersection-over-union ratio includes:

[0023] Calculating the aspect ratio of the first region of interest and the preset size respectively;

[0024] If the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size, determining at least one fifth region of interest in the second region of interest whose width is greater than that of the first region of interest;

[0025] A fourth region of interest with the largest intersection-over-union ratio is determined from the fifth regions of interest.

[0026] Optionally, the determining of the fourth region of interest corresponding to the maximum intersection-over-union ratio further includes:

[0027] If the aspect ratio of the first region of interest is not greater than the aspect ratio of the preset size, determining at least one sixth region of interest in the second region of interest whose height is smaller than that of the first region of interest;

[0028] A fourth region of interest with the largest intersection-over-union ratio is determined from the sixth regions of interest.

[0029] Correspondingly, an embodiment of the present invention further discloses a chip-based image processing system, the system comprising a terminal, the terminal comprising a chip and an image pyramid module:

[0030] The chip is used to acquire an original image and determine a first region of interest including a target object in the original image;

[0031] The image pyramid module is used to obtain the original image, generate at least one second region of interest including a target object, and the size of at least one second region of interest is different; and obtain a target region of interest that meets a preset size from the second region of interest according to the first region of interest; and perform preset image processing according to the target object in the image corresponding to the target region of interest.

[0032] Optionally, the chip is also used to obtain size information of the original image; call the image region of interest extraction interface to identify the target object in the original image; and determine a first region of interest that is smaller than the size information of the original image and contains the target object.

[0033] Optionally, the chip is further used to determine whether the format of the original image is a preset format; if the format of the original image is not the preset format, calling the format conversion interface to convert the format of the original image into the preset format.

[0034] Optionally, the image pyramid module is further used to generate at least one second region of interest of different sizes including a target object; if a third region of interest having the same size as the first region of interest exists in the second region of interest, it is determined whether the size of the third region of interest is consistent with the preset size; if the size of the third region of interest is consistent with the preset size, the third region of interest is determined as a target region of interest; if the size of the third region of interest is inconsistent with the preset size, an image scaling interface is called to scale the size of the image of the third region of interest to the preset size, so as to obtain a target region of interest that meets the preset size.

[0035] Optionally, the image pyramid module is further used to, if there is no third region of interest with the same size as the first region of interest in the second region of interest, determine a corresponding intersection-and-union ratio according to the size of at least one of the second regions of interest and the preset size, and determine a fourth region of interest with the largest corresponding intersection-and-union ratio; and scale the size of the fourth region of interest to the preset size through the image scaling interface to obtain a target region of interest that meets the preset size.

[0036] Optionally, the image pyramid module is further used to calculate the aspect ratio of the first region of interest and the preset size respectively; if the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size, determine at least one fifth region of interest with a width greater than that of the first region of interest in the second region of interest; and determine a fourth region of interest with the largest intersection-over-union ratio from the fifth region of interest.

[0037] Optionally, the image pyramid module is further used to, if the aspect ratio of the first region of interest is not greater than the aspect ratio of the preset size, determine at least one sixth region of interest in the second region of interest whose height is smaller than that of the first region of interest; and determine a fourth region of interest with the largest intersection-over-union ratio from the sixth region of interest.

[0038] Accordingly, an embodiment of the present invention discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the various steps of the above-mentioned chip-based image processing method embodiment when executed by the processor.

[0039] Accordingly, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each step of the above-mentioned chip-based image processing method embodiment is implemented.

[0040] The embodiment of the present invention includes the following advantages: inputting the original image into the chip, and determining the first region of interest including the target object in the original image through the chip; inputting the original image into the image pyramid module, and generating at least one second region of interest including the target object through the image pyramid module, and the size of at least one second region of interest is different; and obtaining a target region of interest that meets a preset size from the second region of interest according to the first region of interest; and performing preset image processing according to the target object in the image corresponding to the target region of interest. In the embodiment of the present invention, after determining the first region of interest of the target object through the chip, the original image can be processed by the image pyramid module outside the chip to obtain the image corresponding to the target region of interest, so as to solve the problem in the prior art that the chip needs to calculate the entire process of processing the image, resulting in a large occupation of the processor computing power of the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flowchart of a chip-based image processing method provided by an embodiment of the present invention;

[0042] Figure 2 is a flowchart of another chip-based image processing method provided by an embodiment of the present invention;

[0043] Figure 3 is a structural block diagram of a flow chart of a chip-based image processing method provided by an embodiment of the present invention;

[0044] Figure 4 It is a structural block diagram of a chip-based image processing system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Reference Figure 1, shows a flowchart of a chip-based image processing method provided by an embodiment of the present invention, the method is applied to a terminal, the terminal includes a chip and an image pyramid module, and the method may specifically include the following steps:

[0047] Step 101: input an original image into the chip, and determine a first region of interest including a target object in the original image through the chip.

[0048] With the development of science and technology, people have more and more demands for car functions, and the car's automatic driving mode has also appeared in people's lives. The automatic driving of cars is inseparable from the research and development of automatic driving chips. In order to improve the automatic driving experience, people are constantly upgrading chips and reducing chip costs. People can reduce the processor computing power of the chip, which may make it impossible to turn on multiple automatic driving functions at the same time.

[0049] Autonomous driving is the synergy of computer technology, perception technology, vehicle networking communication technology, artificial intelligence, high-precision positioning and mapping technology, path planning technology, decision-making and control technology, etc., to achieve the effective and complete driving of the car automatically, safely and in a standardized manner as required. Autonomous driving will realize the networking, automation and intelligence of the car, information sharing and electrification. Autonomous driving can be said to be the ultimate intelligent system.

[0050] Autonomous driving is inseparable from integrated chips. Chips, integrated circuits (ICs), or microcircuits, microchips are a way of miniaturizing circuits (mainly semiconductor devices, but also passive components, etc.) in electronics and are often manufactured on the surface of semiconductor wafers, hereinafter referred to as chips.

[0051] The computing power of the chip processor is defined in the "White Paper on China's Computing Power Development Index" as the computing power of the device by processing data to achieve a specific result output. William D. Nordhau, winner of the 2018 Nobel Prize in Economics, proposed in his article "Computing Process" that "computing power is the amount of information data that a device can process per second based on its internal memory." The core of computing power is various computing chips such as CPU and GPU, and is carried by computers, servers, high-performance computing clusters and various smart terminals. Massive data processing and various digital applications are inseparable from the processing and calculation of computing power. TOPS (Tera Operations Per Second, computing power), 1TOPS represents the processor can perform one trillion (10^12) operations per second FLOPS (floating-point operations per second, floating-point operations per second), the number of floating-point operations performed per second. 1TFLOPS (teraFLOPS, the number of floating-point operations performed per second) is equal to one trillion (=10^12) floating-point operations per second. Because floating-point operations have high performance requirements, floating-point numbers can be converted into integers for calculation while ensuring the accuracy of AI (artificial intelligence), which can greatly reduce the consumption of computing resources and improve computing performance.

[0052] In order to reduce the processor computing power of the chip and utilize other modules outside the chip to replace the chip for calculation as much as possible, such as the image pyramid module, to reduce the computing power of the chip and increase the functions that can be turned on simultaneously for autonomous driving, the present invention can be used with a terminal, and the terminal may include the chip and the image pyramid module outside the chip.

[0053] The original image can be input into the chip, wherein the original image can be an image captured by a car camera or an image obtained from other places. The original image can include multiple objects, such as cars, people, trees, etc. The chip can be used to identify the objects in the original image and determine the target object. The first region of interest in the original image that includes the target object can also be determined. For example, the original image includes cars, people, trees, etc., and the identified objects can include cars and people. Then, the first region of interest with the car as the target object and the first region of interest with the person as the target object can be determined.

[0054] Step 102: input the original image into the image pyramid module, generate at least one second region of interest including a target object through the image pyramid module, and the size of at least one second region of interest is different; and obtain a target region of interest that meets a preset size from the second region of interest according to the first region of interest.

[0055] While the chip determines the first region of interest including the target object, the original image can be input into the image pyramid module outside the chip. After receiving the original image, the image pyramid module can generate at least one second region of interest including the target object. The size of the at least one second region of interest is different. The size of the second region of interest can be a preset fixed value or a preset ratio of the region of interest of the target object. For example, the preset fixed value can be 50*50, 100*100, 150*150, etc. The preset ratio of the region of interest can be 1 / 2, 1 / 4, 1 / 8, etc. The target region of interest that meets the preset size can also be obtained from the second region of interest based on the first region of interest, wherein the preset size can be a size preset by a classification model, and the classification model can be a model for processing the image after obtaining the image corresponding to the target region of interest. Therefore, the region of interest needs to be processed according to the preset size of the classification model. The specific way of presetting the size of the second region of interest can be set according to actual needs, and the embodiment of the present invention is not limited here.

[0056] Step 103: performing preset image processing according to the target object in the image corresponding to the target region of interest.

[0057] After obtaining the target region of interest that meets the preset size, the image corresponding to the target region of interest can be input into the classification model. The classification model can perform preset image processing on the image corresponding to the target region of interest according to the target object in the image corresponding to the target region of interest, wherein the preset image processing can be image classification or image screening. The specific preset image processing process can be set according to actual needs, and the embodiments of the present invention are not limited here.

[0058] The original image is input into the chip, and the chip determines the first region of interest including the target object in the original image; the original image is input into the image pyramid module, and the image pyramid module generates at least one second region of interest including the target object, and the size of the at least one second region of interest is different; and according to the first region of interest, a target region of interest that meets the preset size is obtained from the second region of interest; according to the target object in the image corresponding to the target region of interest, a preset image processing is performed. In an embodiment of the present invention, after the first region of interest of the target object is determined by the chip, the original image can be processed by the image pyramid module outside the chip to obtain the image corresponding to the target region of interest, so as to solve the problem in the prior art that the chip needs to calculate the entire process of processing the image, resulting in a large occupation of the processor computing power of the chip.

[0059] Reference Figure 2, shows a flowchart of another chip-based image processing method provided by an embodiment of the present invention, the method is applied to a terminal, the terminal includes a chip and an image pyramid module, and the method may specifically include the following steps:

[0060] Step 201, inputting an original image into the chip, and determining through the chip whether the format of the original image is a preset format; if the format of the original image is not the preset format, calling the format conversion interface through the chip to convert the format of the original image into the preset format, and determining through the chip a first region of interest including a target object in the original image.

[0061] The chip may include a format conversion function, and the format of the image may be converted by calling a format conversion interface. After the original image is input into the chip, the format of the original image may be judged to determine whether the format of the original image is a preset format, wherein the preset format may be a YUV format. In the process of the autonomous driving chip processing the image, the autonomous driving chip may process the image in the YUV format. If the format of the original image is not in the preset format, it is necessary to call the format conversion interface through the chip, and convert the format of the original image into the preset format through the format conversion function. If the format of the original image is in the preset format, no conversion is required. The chip may be used to identify the object in the original image, and determine the target object therefrom. The first region of interest including the target object in the original image may also be determined. For example, the original image includes a car, a person, a tree, etc., and the identified objects may include a car and a person. Then, the first region of interest with the car as the target object and the first region of interest with the person as the target object may be determined. The subsequent processing steps for each target object are the same. The present invention takes a person as an example.

[0062] In an embodiment of the present invention, the method further includes: inputting size information of the original image into the chip;

[0063] The determining, by the chip, a first region of interest including a target object in the original image comprises:

[0064] The chip may include an image region of interest extraction function, and the image may be identified through an image region of interest extraction interface to obtain a region of interest of a target object. The image region of interest extraction interface may be called by the chip to identify the target object in the original image; and a first region of interest having a size information smaller than that of the original image and containing the target object may be determined.

[0065] The size information of the original image can be input into the chip. After the chip receives the size information of the original image, the image region of interest extraction interface can be called to identify the original image through the image region of interest extraction function to determine the target object in the original image. The first region of interest containing the target object and having a size smaller than the size information of the original image can also be determined based on the size of the original image.

[0066] Step 202: input the original image into the image pyramid module, generate at least one second region of interest including a target object through the image pyramid module, and the size of at least one second region of interest is different; and obtain a target region of interest that meets a preset size from the second region of interest according to the first region of interest.

[0067] In the embodiment of the present invention, generating at least one second region of interest including a target object through the image pyramid module, wherein the size of at least one second region of interest is different; and obtaining a target region of interest satisfying a preset size from the second region of interest according to the first region of interest, comprises:

[0068] generating at least one second region of interest of different sizes including the target object by means of the image pyramid module; if a third region of interest having the same size as the first region of interest exists in the second region of interest, determining whether the size of the third region of interest is consistent with the preset size;

[0069] If the size of the third region of interest is consistent with the preset size, determining the third region of interest as a target region of interest;

[0070] If the size of the third region of interest is inconsistent with the preset size, an image scaling interface is called to scale the size of the image of the third region of interest to the preset size to obtain a target region of interest that meets the preset size.

[0071] The chip may include an image scaling function, and the image in the region of interest may be scaled through the image scaling interface to obtain a region of interest of a preset size and an image in the region of interest. The image pyramid module may be used to generate at least one second region of interest of different sizes including a target object, wherein the size of the second region of interest may be a preset fixed value or a preset ratio of the region of interest, for example, the preset fixed value may be 50*50, 100*100, 150*150, etc., and the preset ratio of the region of interest of the target object may be 1 / 2, 1 / 4, 1 / 8, etc. of the region of interest. It may also be determined whether there is a third region of interest with the same size as the first region of interest in the second region of interest. If there is a third region of interest with the same size as the first region of interest in the second region of interest, it may be determined whether the size of the third region of interest is consistent with the preset size of the classification model; if the size of the third region of interest is consistent with the preset size of the classification model, the third region of interest may be determined as the target region of interest. If the size of the third region of interest is inconsistent with the preset size of the classification model, the image scaling interface may be called to scale the size of the image of the third region of interest to the preset size through the image scaling function, and the target region of interest meeting the preset size may be obtained.

[0072] In the embodiment of the present invention, the step of generating at least one second region of interest including a target object by the image pyramid module, wherein the size of at least one second region of interest is different; and obtaining a target region of interest satisfying a preset size from the second region of interest according to the first region of interest, further comprises:

[0073] If there is no third region of interest with the same size as the first region of interest in the second regions of interest, determining a corresponding intersection-over-union ratio according to the size of at least one of the second regions of interest and the preset size, and determining a fourth region of interest with the largest corresponding intersection-over-union ratio;

[0074] The size of the fourth region of interest is scaled to the preset size through the image scaling interface to obtain a target region of interest that meets the preset size.

[0075] If there is no third region of interest with the same size as the first region of interest in the second region of interest, the intersection-and-union ratio of each second region of interest and the preset size can be determined based on the size of at least one second region of interest and the preset size, so that the fourth region of interest with the largest corresponding intersection-and-union ratio can be determined, wherein the intersection-and-union ratio function calculates the ratio of the intersection and union of two bounding boxes, and the image scaling interface can be called to scale the size of the fourth region of interest to the preset size through the image scaling function to obtain the target region of interest that meets the preset size.

[0076] In the embodiment of the present invention, determining the fourth region of interest having the largest intersection-over-union ratio includes:

[0077] Calculating the aspect ratio of the first region of interest and the preset size respectively;

[0078] If the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size, determining at least one fifth region of interest in the second region of interest whose width is greater than that of the first region of interest;

[0079] A fourth region of interest with the largest intersection-over-union ratio is determined from the fifth regions of interest.

[0080] When determining the fourth region of interest corresponding to the largest intersection-and-joint ratio, the aspect ratio of the first region of interest and the aspect ratio of the preset size can be calculated, and the aspect ratio of the first region of interest and the aspect ratio of the preset size can be compared. If the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size, at least one fifth region of interest with a width greater than that of the first region of interest can be determined in the second region of interest. For example, the width of the first region of interest is 120, and the sizes of the second region of interest are 50*100, 100*150, 150*200, and 200*250, respectively. The fifth region of interest includes 150*200 and 200*250. Then, the fourth region of interest with the largest intersection-and-joint ratio is determined from the fifth region of interest. This can prevent the determined region of interest with the largest intersection-and-joint ratio from being larger than the first region of interest, and the need to enlarge the image.

[0081] In the embodiment of the present invention, the determining of the fourth region of interest with the largest intersection-over-union ratio further includes:

[0082] If the aspect ratio of the first region of interest is not greater than the aspect ratio of the preset size, determining at least one sixth region of interest in the second region of interest whose height is smaller than that of the first region of interest;

[0083] A fourth region of interest with the largest intersection-over-union ratio is determined from the sixth regions of interest.

[0084] If the aspect ratio of the first region of interest is not greater than the aspect ratio of the preset size, at least one sixth region of interest whose height is smaller than that of the first region of interest can be determined in the second region of interest. For example, the height of the first region of interest is 190, and the sizes of the second region of interest are 50*100, 100*150, 150*200, and 200*250 respectively. Then the sixth region of interest includes 50*100 and 100*150, and then the fourth region of interest with the largest intersection-over-union ratio is determined from the sixth region of interest.

[0085] Step 203: performing preset image processing according to the target object in the image corresponding to the target region of interest.

[0086] After obtaining the target region of interest that meets the preset size, the image corresponding to the target region of interest can be input into the classification model. The classification model can perform preset image processing on the image corresponding to the target region of interest according to the target object in the image corresponding to the target region of interest, wherein the preset image processing can be image classification or image screening. The specific preset image processing process can be set according to actual needs, and the embodiments of the present invention are not limited here.

[0087] The original image is input into the chip, and the chip determines the first region of interest including the target object in the original image; the original image is input into the image pyramid module, and the image pyramid module generates at least one second region of interest including the target object, and the size of the at least one second region of interest is different; and according to the first region of interest, a target region of interest that meets the preset size is obtained from the second region of interest; according to the target object in the image corresponding to the target region of interest, a preset image processing is performed. In an embodiment of the present invention, after the first region of interest of the target object is determined by the chip, the original image can be processed by the image pyramid module outside the chip to obtain the image corresponding to the target region of interest, so as to solve the problem in the prior art that the chip needs to calculate the entire process of processing the image, resulting in a large occupation of the processor computing power of the chip.

[0088] In order to enable those skilled in the art to better understand the embodiments of the present invention, the embodiments of the present invention are described below by using an example: Figure 3 , showing a flow chart of a chip-based image processing method provided by an embodiment of the present invention;

[0089] First, the chip obtains the original image and the size information of the original image, and determines whether the format of the original image is a preset format. If not, the format conversion interface is called to convert the format of the original image into the preset format. If yes, there is no need to convert the format. The original image and the size information of the original image can be encapsulated through the chip to determine the first area of ​​interest including the target object in the original image.

[0090] Secondly, the image pyramid module can obtain the original image, generate at least one second region of interest of different sizes including the target object according to the original image, and determine whether there is a third region of interest with the same size as the first region of interest in the second region of interest. If there is a third region of interest with the same size as the first region of interest in the second region of interest, it can be determined whether the third region of interest is consistent with a preset size. If the third region of interest is consistent with the preset size, the third region of interest can be determined as a target region of interest. If the third region of interest is inconsistent with the preset size, an image scaling interface can be called to scale the third region of interest, so that the image in the third region of interest can be scaled to obtain a target region of interest and an image in the target region of interest that meet the preset size.

[0091] Again, if there is no third region of interest with the same size as the first region of interest in the second region of interest, the intersection-and-joint ratio of at least one second region of interest and the preset size is calculated respectively, and the aspect ratio of the first region of interest and the preset size can also be calculated to determine whether the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size. If the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size, at least one fifth region of interest with a width greater than that of the first region of interest is determined in the second region of interest, and a fourth region of interest with the largest intersection-and-joint ratio can be determined from the fifth region of interest. If the aspect ratio of the first region of interest is not greater than the aspect ratio of the preset size, at least one sixth region of interest with a height less than that of the first region of interest can be determined in the second region of interest, and a fourth region of interest with the largest intersection-and-joint ratio can be determined from the sixth region of interest, so that the fourth region of interest can be scaled to obtain a target region of interest that meets the preset size, and preset image processing can be performed according to the target object in the image corresponding to the target region of interest.

[0092] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0093] Reference Figure 4 , shows a structural block diagram of a chip-based image processing system provided by an embodiment of the present invention, wherein the system includes a terminal, and the terminal includes a chip S1 and an image pyramid module S5:

[0094] The chip S1 is used to acquire an original image and determine a first region of interest including a target object in the original image;

[0095] The image pyramid module S5 is used to obtain the original image, generate at least one second region of interest including a target object, and the size of at least one second region of interest is different; and obtain a target region of interest that meets a preset size from the second region of interest according to the first region of interest; and perform preset image processing according to the target object in the image corresponding to the target region of interest.

[0096] Optionally, the chip S1 is further used to obtain size information of the original image; call the image region of interest extraction interface S2 to identify the target object in the original image; and determine a first region of interest that is smaller than the size information of the original image and contains the target object.

[0097] Optionally, the chip S1 is further used to determine whether the format of the original image is a preset format; if the format of the original image is not the preset format, calling the format conversion interface S3 to convert the format of the original image into the preset format.

[0098] Optionally, the image pyramid module S5 is further used to generate at least one second region of interest of different sizes including a target object; if a third region of interest having the same size as the first region of interest exists in the second region of interest, it is determined whether the size of the third region of interest is consistent with the preset size; if the size of the third region of interest is consistent with the preset size, the third region of interest is determined as a target region of interest; if the size of the third region of interest is inconsistent with the preset size, the image scaling interface S4 is called to scale the size of the image of the third region of interest to the preset size, so as to obtain a target region of interest that meets the preset size.

[0099] Optionally, the image pyramid module S5 is further used to, if there is no third region of interest having the same size as the first region of interest in the second regions of interest, determine a corresponding intersection-and-union ratio according to the size of at least one of the second regions of interest and the preset size, and determine a fourth region of interest having the largest corresponding intersection-and-union ratio; and scale the size of the fourth region of interest to the preset size through the image scaling interface S4 to obtain a target region of interest that meets the preset size.

[0100] Optionally, the image pyramid module S5 is further used to calculate the aspect ratio of the first region of interest and the preset size respectively; if the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size, determine at least one fifth region of interest with a width greater than that of the first region of interest in the second region of interest; and determine a fourth region of interest with the largest intersection-over-union ratio from the fifth region of interest.

[0101] Optionally, the image pyramid module S5 is further used to, if the aspect ratio of the first region of interest is not greater than the aspect ratio of the preset size, determine at least one sixth region of interest in the second region of interest whose height is smaller than that of the first region of interest; and determine a fourth region of interest with the largest intersection-over-union ratio from the sixth region of interest.

[0102] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0103] An embodiment of the present invention further provides an electronic device, including:

[0104] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned chip-based image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0105] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned chip-based image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0106] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0107] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0111] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0112] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0113] The above is a detailed introduction to a chip-based image processing method, a chip-based image processing device, an electronic device and a storage medium provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A chip-based image processing method, characterized in that: The method is applied to a terminal, which includes a chip and an image pyramid module: Inputting an original image into the chip, and determining a first region of interest including a target object in the original image through the chip; Inputting the original image into the image pyramid module, generating at least one second region of interest including a target object through the image pyramid module, wherein the size of at least one second region of interest is different; And according to the first region of interest, obtaining a target region of interest that meets a preset size from the second region of interest, including: generating at least one second region of interest of different sizes including the target object by means of the image pyramid module; if a third region of interest having the same size as the first region of interest exists in the second region of interest, determining whether the size of the third region of interest is consistent with the preset size; If the size of the third region of interest is consistent with the preset size, determining the third region of interest as a target region of interest; If the size of the third region of interest is inconsistent with the preset size, calling an image scaling interface to scale the size of the image of the third region of interest to the preset size, so as to obtain a target region of interest that meets the preset size; If there is no third region of interest with the same size as the first region of interest in the second regions of interest, determining a corresponding intersection-over-union ratio according to the size of at least one of the second regions of interest and the preset size, and determining a fourth region of interest with the largest corresponding intersection-over-union ratio; By means of the image scaling interface, the size of the fourth region of interest is scaled to the preset size, so as to obtain a target region of interest that meets the preset size; Preset image processing is performed according to the target object in the image corresponding to the target region of interest.

2. The method according to claim 1, characterized in that The method further comprises: inputting size information of the original image into the chip; The determining, by the chip, a first region of interest including a target object in the original image comprises: The image region of interest extraction interface is called by the chip to identify the target object in the original image; and a first region of interest having a size smaller than that of the original image and containing the target object is determined.

3. The method according to claim 1, characterized in that: After the original image is input into the chip, the method further includes: Determining, by the chip, whether the format of the original image is a preset format; If the format of the original image is not the preset format, the format conversion interface is called by the chip to convert the format of the original image into the preset format.

4. The method according to claim 1, characterized in that The determining of the fourth region of interest having the largest intersection-over-union ratio includes: Calculating the aspect ratio of the first region of interest and the preset size respectively; If the aspect ratio of the first region of interest is greater than the aspect ratio of the preset size, determining at least one fifth region of interest in the second region of interest whose width is greater than that of the first region of interest; A fourth region of interest with the largest intersection-over-union ratio is determined from the fifth regions of interest.

5. The method according to claim 1, characterized in that The determining of the fourth region of interest with the largest intersection-over-union ratio also includes: If the aspect ratio of the first region of interest is not greater than the aspect ratio of the preset size, determining at least one sixth region of interest in the second region of interest whose height is smaller than that of the first region of interest; A fourth region of interest with the largest intersection-over-union ratio is determined from the sixth regions of interest.

6. A chip-based image processing system, characterized in that: The system includes a terminal, and the terminal includes a chip and an image pyramid module: The chip is used to acquire an original image and determine a first region of interest including a target object in the original image; The image pyramid module is used to acquire the original image and generate at least one second region of interest including the target object, wherein the size of at least one second region of interest is different; And according to the first region of interest, obtaining a target region of interest that meets a preset size from the second region of interest, including: generating at least one second region of interest of different sizes including the target object by means of the image pyramid module; if a third region of interest having the same size as the first region of interest exists in the second region of interest, determining whether the size of the third region of interest is consistent with the preset size; If the size of the third region of interest is consistent with the preset size, determining the third region of interest as a target region of interest; If the size of the third region of interest is inconsistent with the preset size, calling an image scaling interface to scale the size of the image of the third region of interest to the preset size, so as to obtain a target region of interest that meets the preset size; If there is no third region of interest with the same size as the first region of interest in the second regions of interest, determining a corresponding intersection-over-union ratio according to the size of at least one of the second regions of interest and the preset size, and determining a fourth region of interest with the largest corresponding intersection-over-union ratio; By means of the image scaling interface, the size of the fourth region of interest is scaled to the preset size, so as to obtain a target region of interest that meets the preset size; Preset image processing is performed according to the target object in the image corresponding to the target region of interest.

7. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the chip-based image processing method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the chip-based image processing method according to any one of claims 1 to 5 are implemented.