Image processing method and device and intelligent driving equipment

Abstract

The embodiment of the invention provides an image processing method and device and intelligent driving equipment, and the method comprises the steps: obtaining a first image, and enabling a first view field angle of the first image to be the field angle of a camera device which collects the first image; the area where the first target object is located in the first image is cut, a second image is obtained, the first view field angle is larger than the second view field angle of the second image, and view depth information of the second image is consistent with view depth information corresponding to the area where the first target object is located in the first image. Therefore, the extraction flexibility of the region of interest in the image can be improved, and the detection flexibility of different dynamic targets is improved, so that the detection precision of the dynamic targets is improved, and the safety of intelligent driving equipment is improved.

Description

Image processing method, device and intelligent driving device Technical Field

[0001] The embodiments of the present application relate to the field of intelligent driving, and more specifically, to an image processing method, apparatus, and intelligent driving device. Background Art

[0002] Dynamic object detection is a key technology for enabling assisted or autonomous driving. As an image acquisition device for dynamic object detection, vehicle-mounted cameras require different field of view (FOV) and detection distance requirements in different application scenarios.

[0003] For example, when there are many dynamic targets close to the vehicle, the onboard camera needs a larger FOV and a shorter detection distance to ensure that more dynamic targets are detected. When there are many dynamic targets far from the vehicle, the onboard camera needs a smaller FOV and a longer detection distance to ensure that distant dynamic targets are detected.

[0004] Therefore, how to improve the flexibility of extracting regions of interest in images for different dynamic targets and thus improve the flexibility of dynamic target detection has become an urgent problem to be solved.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide an image processing method, apparatus, and intelligent driving device, which can improve the flexibility of extracting regions of interest in images, thereby improving the flexibility of detecting different dynamic targets, thereby improving the accuracy of dynamic target detection, and further improving the safety of intelligent driving devices.

[0007] The intelligent driving devices in this application may include road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc. For example, the intelligent driving device can be a vehicle, which is a vehicle in a broad sense, and can be a vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a mower, a harvester, etc.), amusement equipment, a toy vehicle, etc. The embodiment of this application does not specifically limit the type of vehicle. For another example, the intelligent driving device can be a vehicle such as an airplane or a ship.

[0008] In a first aspect, an image processing method is provided, comprising: acquiring a first image, wherein a first view field of view angle of the first image is the field of view angle of a camera device used to capture the first image; cropping a region in the first image where a first target object is located to obtain a second image, wherein the first view field of view angle is greater than the second view field of view angle of the second image, and view depth information of the second image is consistent with view depth information corresponding to the region where the first target object is located in the first image.

[0009] It should be understood that the field of view of a camera is an inherent property of the camera. The field of view represents the angular range in the physical world corresponding to different regions in an image. Different regions in the same image can have different field of view angles.

[0010] It should also be understood that the view depth information of the second image is not lost, that is, the view depth information in the second image cropped from the first image is consistent with the view depth information of the corresponding area in the first image.

[0011] In the above technical solution, a second image of the area where the first target object is located is obtained by cropping the first image captured by a camera device with a fixed field of view, wherein different second images obtained by cropping different first target objects are different, and the view depth information of different second images also differs to varying degrees. In this way, the flexibility of extracting the region of interest in the image can be improved. Compared with target detection based on the region of interest, the input image of the target detection model based on the region of interest is the original image captured by the camera device, and then the original image after size scaling is subjected to target detection based on the perceptual interest region. Since the original image after size scaling will lose part of the image information, such as the view information of the original image. Therefore, the image processing method before target detection in the present application can improve the detection flexibility of different target objects, thereby improving the detection accuracy of different target objects.

[0012] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: obtaining view depth information of the first image and size information of one or more target objects in the first image, where the first target object is a target among the one or more target objects. Based on the view depth information and the size information, the first image is divided into multiple regions, where the multiple regions include a first region, and the first region includes the first target object. Cropping the region in the first image where the first target object is located to obtain the second image includes: cropping the first region to obtain the second image.

[0013] In the above technical solution, the first image is divided into regions according to the view depth information of the first image and the size information of the target object, so that a more accurate region division result can be obtained, which helps to obtain a more accurate second image including the first target object, thereby improving the accuracy of target detection.

[0014] As a possible implementation manner, the area where the first target object is located in the first area is used as the clipping range, and the first image is clipped to obtain the second image.

[0015] In the above technical solution, the cropping range of the first image is reduced, which can further increase the range occupied by the first target object in the second image, thereby further improving the accuracy of target detection.

[0016] In conjunction with the first aspect, in certain implementations of the first aspect, the first image is divided into multiple regions based on a scene category of the first image, the multiple regions including the first region, and the first region including the first target object. Cropping the region in the first image where the first target object is located to obtain the second image includes: cropping the first region to obtain the second image.

[0017] In combination with the first aspect, in some implementations of the first aspect, the method further includes: performing detail enhancement on the second image when the number of pixels in the first area in the first image is smaller than the number of pixels in other areas in the first image.

[0018] In the above technical solution, if the area type of the first area is a distant area, the pixels of the first target object in the first area are few, and the second image after detail enhancement can further improve the detection accuracy of the target detection.

[0019] In conjunction with the first aspect, in certain implementations of the first aspect, cropping the area in the first image where the first target object is located to obtain the second image includes: determining a first image cropping strategy from multiple image cropping strategies based on a scene category of the first image; and cropping the area in the first image where the first target object is located based on the first image cropping strategy to obtain the second image.

[0020] In the above technical solution, the first image is divided into regions according to the scene category of the first image, which not only simplifies the region division process, but also obtains more accurate region division results, thereby helping to obtain a more accurate second image including the first target object, thereby improving the accuracy of target detection.

[0021] In combination with the first aspect, in some implementations of the first aspect, the method further includes: determining a detection result of the first target object based on the second image.

[0022] In the above technical solution, target detection is performed based on the second image obtained by cropping the first image, which can improve the detection distance of the first target object. For example, currently, the first image is directly input into the target detection model, and the detection distance range of the output detection result of the first target object is 0m to 244m. In the above technical solution, the cropped second image retains the view depth information of the area where the first target object is located in the first image. Therefore, inputting the cropped second image into the target detection model can expand the range of detection distance. For example, the first target object can be located at a detection distance of 244m to 696m. In other words, the detection result currently obtained by directly inputting the original image into the target detection model is difficult to detect distant target objects due to the loss of view depth information of distant target objects in the original image caused by the scaling process of the original image. However, in the above technical solution, distant target objects included in the original image can be detected.

[0023] In conjunction with the first aspect, in certain implementations of the first aspect, determining a detection result for the first target object based on the second image includes: adjusting pixel values ​​of the second image to preset pixel values; or adjusting pixel values ​​of the second image after detail enhancement to preset pixel values. Inputting the second image with adjusted pixel values ​​into the object detection model to obtain a detection result.

[0024] In a second aspect, a method for image processing is provided, the method comprising: acquiring a first image, wherein the first view field angle of the first image is the field angle of the camera device that captures the first image. Based on the view depth information of the first image and the size information of one or more target objects in the first image, the area where the first target object is located in the first image is cropped to obtain a second image. Alternatively, based on the scene category of the first image, the area where the first target object is located is cropped to obtain the second image. The first view field angle is greater than or equal to the second view field angle of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

[0025] In the above technical solution, the first image is adaptively cropped through the view depth information of the first image and the size information of the target object, or through the scene category of the first image, to obtain the second image. This can improve the flexibility of extracting the area of ​​interest in the image, and then the target detection based on the second image can improve the detection flexibility of different target objects, thereby improving the detection accuracy of different target objects.

[0026] In conjunction with the second aspect, in certain implementations of the second aspect, cropping a region in the first image where the first target object is located based on view depth information of the first image and size information of one or more target objects in the first image to obtain the second image includes: dividing the first image into multiple regions based on the view depth information and the size information, the multiple regions including the first region, the first region including the first target object, and cropping the first region to obtain the second image.

[0027] In conjunction with the second aspect, in certain implementations of the second aspect, cropping the region where the first target object is located based on the scene category of the first image to obtain the second image includes: dividing the first image into multiple regions based on the scene category, the multiple regions including the first region, the first region including the first target object; and cropping the first region to obtain the second image.

[0028] In combination with the second aspect, in some implementations of the second aspect, the method further includes: performing detail enhancement on the second image when the number of pixels in the first area in the first image is smaller than the number of pixels in other areas in the first image.

[0029] In conjunction with the second aspect, in certain implementations of the second aspect, cropping the area of ​​the first image where the first target object is located based on the scene category of the first image to obtain the second image includes: determining a first image cropping strategy from multiple image cropping strategies based on the scene category of the first image; and cropping the area of ​​the first image where the first target object is located based on the first image cropping strategy to obtain the second image.

[0030] In combination with the second aspect, in certain implementations of the second aspect, the method further includes: when the scene category is a high-speed scene, performing detail enhancement on the second image.

[0031] In combination with the second aspect, in some implementations of the second aspect, the method further includes: determining a detection result of the first target object based on the second image.

[0032] In conjunction with the second aspect, in certain implementations of the second aspect, determining a detection result for the first target object based on the second image includes: adjusting pixel values ​​of the second image to preset pixel values; or adjusting pixel values ​​of the second image after detail enhancement to preset pixel values. Inputting the second image with adjusted pixel values ​​into the object detection model to obtain a detection result.

[0033] In a third aspect, an image processing device is provided, the device comprising an acquisition unit and a processing unit. The acquisition unit is configured to acquire a first image, wherein a first view field of view angle of the first image is the field of view angle of a camera device that captured the first image. The processing unit is configured to crop an area in the first image where a first target object is located to obtain a second image, wherein the first view field of view angle is greater than the second view field of view angle of the second image, and view depth information of the second image is consistent with view depth information corresponding to the area where the first target object is located in the first image.

[0034] For the technical effects corresponding to the technical solution of the third aspect, please refer to the corresponding description of the first aspect.

[0035] In conjunction with the third aspect, in certain implementations of the third aspect, the acquisition unit is further configured to acquire view depth information of the first image and size information of one or more target objects in the first image, where the first target object is a target among the one or more target objects. The processing unit is further configured to divide the first image into multiple regions based on the view depth information and the size information, where the multiple regions include the first region, and the first region includes the first target object. The processing unit is specifically configured to crop the first region to obtain the second image.

[0036] In conjunction with the third aspect, in certain implementations of the third aspect, the processing unit is further configured to, based on a scene category of the first image, divide the first image into multiple regions, where the multiple regions include a first region, and the first region includes the first target object. The processing unit is specifically configured to crop the first region to obtain the second image.

[0037] In combination with the third aspect, in certain implementations of the third aspect, the processing unit is further used to: perform detail enhancement on the second image when the number of pixels in the first area in the first image is smaller than the number of pixels in other areas in the first image.

[0038] In conjunction with the third aspect, in certain implementations of the third aspect, the processing unit is specifically configured to: determine, based on the scene category of the first image, a first image cropping strategy from multiple image cropping strategies; and crop, based on the first image cropping strategy, an area in the first image where the first target object is located to obtain a second image.

[0039] In combination with the third aspect, in certain implementations of the third aspect, the processing unit is further configured to: when the scene category is a high-speed scene, perform detail enhancement on the second image.

[0040] In combination with the third aspect, in certain implementations of the third aspect, the processing unit is further used to: determine a detection result of the first target object based on the second image.

[0041] In conjunction with the third aspect, in certain implementations of the third aspect, the processing unit is specifically configured to: adjust pixel values ​​of the second image to preset pixel values; or adjust pixel values ​​of the detail-enhanced second image to preset pixel values; and input the second image with adjusted pixel values ​​into the object detection model to obtain a detection result.

[0042] In a fourth aspect, an image processing device is provided, which includes an acquisition unit and a processing unit. The acquisition unit is used to acquire a first image, wherein the first view field angle of the first image is the field angle of the camera device that captures the first image. The processing unit is used to crop the area where the first target object is located in the first image based on the view depth information of the first image and the size information of one or more target objects in the first image to obtain a second image. Alternatively, the processing unit is used to crop the area where the first target object is located based on the scene category of the first image to obtain a second image; wherein the first view field angle is greater than or equal to the second view field angle of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

[0043] For the technical effects corresponding to the technical solution of the fourth aspect, reference can be made to the corresponding description of the first or second aspect.

[0044] In conjunction with the fourth aspect, in certain implementations of the fourth aspect, the processing unit is specifically configured to: divide the first image into multiple regions based on the view depth information and the size information, the multiple regions including the first region, the first region including the first target object, and crop the first region to obtain the second image.

[0045] In conjunction with the fourth aspect, in certain implementations of the fourth aspect, the processing unit is specifically configured to: divide the first image into multiple regions based on the scene category, the multiple regions including the first region, the first region including the first target object, and crop the first region to obtain the second image.

[0046] In combination with the fourth aspect, in certain implementations of the fourth aspect, the processing unit is further used to: enhance the details of the second image when the number of pixels in the first area in the first image is smaller than the number of pixels in other areas in the first image.

[0047] In conjunction with the fourth aspect, in certain implementations of the fourth aspect, the processing unit is specifically configured to: determine, based on the scene category of the first image, a first image cropping strategy from multiple image cropping strategies; and crop, based on the first image cropping strategy, an area in the first image where the first target object is located to obtain a second image.

[0048] In combination with the fourth aspect, in certain implementations of the fourth aspect, the processing unit is further configured to: when the scene category is a high-speed scene, perform detail enhancement on the second image.

[0049] In combination with the fourth aspect, in certain implementations of the fourth aspect, the processing unit is further used to: determine a detection result of the first target object based on the second image.

[0050] In conjunction with the fourth aspect, in certain implementations of the fourth aspect, the processing unit is specifically configured to: adjust pixel values ​​of the second image to preset pixel values; or adjust pixel values ​​of the detail-enhanced second image to preset pixel values; and input the second image with adjusted pixel values ​​into the object detection model to obtain a detection result.

[0051] In a fifth aspect, a device for image processing is provided, which includes a processor and a memory, wherein the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the device to perform any possible method in the first aspect, or to enable the device to perform any possible method in the second aspect.

[0052] In a sixth aspect, an image processing system is provided, which includes a camera device and a computing platform, wherein the computing platform includes any possible device in the third aspect or the fourth aspect.

[0053] In the seventh aspect, an intelligent driving device is provided, which includes any possible device in the third aspect, the fourth aspect, or the fifth aspect, or includes the system described in the sixth aspect.

[0054] In some possible implementations, the intelligent driving device is a vehicle.

[0055] In an eighth aspect, a server is provided, which includes the device described in any one of the third or fourth aspects above, and the server is further used to send a second image or a detection result of the first target object to the intelligent driving device.

[0056] In the ninth aspect, a computer program product is provided, which includes: computer program code, which, when the computer program code runs on a computer, enables the computer to execute any possible method in the first aspect above, or enables the device to execute any possible method in the second aspect.

[0057] It should be noted that the above-mentioned computer program code can be stored in whole or in part on the first storage medium, wherein the first storage medium can be packaged together with the processor or separately packaged with the processor, and the embodiments of the present application do not specifically limit this.

[0058] In the tenth aspect, a computer-readable medium is provided, which stores a program code. When the computer program code is run on a computer, the computer is caused to execute any possible method in the first aspect, or the device is caused to execute any possible method in the second aspect.

[0059] In the eleventh aspect, an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes any possible method in the above-mentioned first aspect, or so that the device executes any possible method in the second aspect.

[0060] In combination with the eleventh aspect, in a possible implementation, the processor is coupled to the memory through an interface.

[0061] In combination with the eleventh aspect, in a possible implementation, the chip system also includes a memory, in which a computer program or computer instructions are stored. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] FIG1 is a schematic diagram of a field of view of a camera provided in an embodiment of the present application;

[0063] FIG2 is a functional schematic block diagram of an intelligent driving device provided in an embodiment of the present application;

[0064] FIG3 is a schematic diagram of a dynamic target detection scenario provided in an embodiment of the present application;

[0065] FIG4 is a flow chart of an image processing method provided in an embodiment of the present application;

[0066] FIG5 is a schematic diagram of a target detection method provided in an embodiment of the present application;

[0067] FIG6 is a schematic diagram of an image processing method provided in an embodiment of the present application;

[0068] FIG7 is a schematic diagram of another image processing method provided in an embodiment of the present application;

[0069] FIG8 is a schematic diagram of another image processing method provided in an embodiment of the present application;

[0070] FIG9 is a schematic diagram of another image processing method provided in an embodiment of the present application;

[0071] FIG10 is a schematic block diagram of an image processing apparatus 1000 provided in an embodiment of the present application;

[0072] FIG11 is a schematic block diagram of an image processing system 1100 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0074] First, in this application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments based on their internal logical relationships.

[0075] Second, in this application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In the text description of this application, the character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c. Wherein a, b and c can be single or multiple, respectively.

[0076] Third, throughout this application, the terms "first," "second," and various numerical references are used for descriptive purposes only and are not intended to limit the scope of the embodiments of this application. For example, they are used to distinguish between different images, rather than to describe a specific order or precedence. It should be understood that these terms are interchangeable, where appropriate, to describe aspects beyond the embodiments of this application.

[0077] Fourth, in this application, descriptions such as "when...", "in the case of...", and "if" all mean that the device will take corresponding actions under certain objective circumstances. They do not limit the time, nor do they require the device to perform judgment actions when implementing them, nor do they mean that there are other limitations.

[0078] Fifth, in this application, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products or apparatuses.

[0079] Sixth, in this application, "storage" may refer to storage in one or more memories. The one or more memories may be provided separately or integrated into an encoder or decoder, a processor, or a communication device. The one or more memories may also be partially provided separately and partially integrated into a decoder, a processor, or a communication device. The memory may be any type of storage medium and is not limited in this application.

[0080] First, the professional terms involved in the embodiments of this application are explained in detail.

[0081] 1. Region of interest (ROI)

[0082] The region of interest is the area in the image that may attract the visual attention of the human eye.

[0083] 2. Field of view (FOV)

[0084] In optical instruments, the angle between the two edges of the lens, with the lens at its apex, and the maximum range through which the image of the object being measured can pass through the lens, is called the field of view. The field of view angle determines the field of view of the optical instrument.

[0085] For example, Figure 1 is a schematic diagram of the field of view of a camera provided in an embodiment of the present application. The camera device 100 shown in Figure 1 is a top-down perspective. The field of view (FOV) is an inherent physical property of the camera device 100. The field of view (FOV) can be a horizontal or vertical field of view. Figure 1 shows the horizontal field of view.

[0086] The technical solutions in the embodiments of the present application will be described in detail below with reference to FIG. 2 to FIG. 11 .

[0087] FIG2 is a functional schematic block diagram of an intelligent driving device 200 provided in an embodiment of the present application.

[0088] The intelligent driving device 200 may include a perception system 210 and a computing platform 220. The perception system 210 may include one or more sensors for sensing information about the environment surrounding the intelligent driving device 200. For example, the perception system 210 may include a positioning system, which may be a global positioning system (GPS), a Beidou system, or other positioning systems. The perception system 210 may also include one or more of an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and the camera device 100 shown in FIG1 .

[0089] Some or all functions of the intelligent driving device 200 can be controlled by the computing platform 220. The computing platform 220 may include one or more processors, such as processors 221 to 22n (n is a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration file to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, the processor may also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc. In addition, the computing platform 220 may also include a memory for storing instructions, and some or all of the processors 221 to 22n may call the instructions in the memory to implement corresponding functions.

[0090] The intelligent driving device 200 in this application may include road vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc. For example, the intelligent driving device 200 can be a vehicle, which is a vehicle in a broad sense, and can be a vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a mower, a harvester, etc.), amusement equipment, a toy vehicle, etc. The embodiment of this application does not specifically limit the type of vehicle. For another example, the intelligent driving device 200 can be a vehicle such as an airplane or a ship.

[0091] As mentioned earlier, dynamic target recognition is becoming a key technology for intelligent driving. For dynamic targets in different scenarios, if image acquisition devices can adapt to the different requirements of different scenarios, the accuracy of dynamic target detection in different scenarios will be improved.

[0092] FIG3 is a schematic diagram of a dynamic target detection scenario 300 provided in an embodiment of the present application.

[0093] As shown in Figure 3(a), in an urban setting, dynamic targets around the ego vehicle move slowly, and there are many dynamic targets close to the ego vehicle, such as pedestrians, cyclists, and vehicles. In this case, one goal of dynamic target detection is to detect more dynamic targets, thereby helping the ego vehicle make more accurate driving strategies and improving the driving safety of intelligent driving devices. In this case, the image acquisition device is required to have a large field of view, so that it can capture a sufficiently large image range around the ego vehicle, thereby facilitating the detection of a sufficient number of dynamic targets around the ego vehicle.

[0094] However, in high-speed scenarios, the goal of dynamic target detection for the ego vehicle is completely different from that in urban scenarios. As shown in Figure 3(b), in high-speed scenarios, the ego vehicle and the dynamic targets around it generally maintain a relatively fast movement speed, and the ego vehicle and the dynamic targets around it generally maintain a relatively long distance. Therefore, in high-speed scenarios, the goal of dynamic target detection is to detect dynamic targets in a short period of time, thereby helping the ego vehicle to make more accurate driving strategies and thus improving the driving safety of intelligent driving devices. In this case, the image acquisition device is required to have a small field of view, so that it can capture images that are farther away from the ego vehicle, thereby helping to improve the speed of dynamic target detection.

[0095] Currently, there are two image acquisition devices and methods that are suitable for different scenarios. The first method uses a novel beam steering method based on a set of laser radars. The field of view of the laser radar is adjusted in different scenarios to collect point cloud data suitable for dynamic target recognition in different scenarios. Although this method can obtain point cloud data suitable for dynamic target recognition in different scenarios, the cost of the laser radar that automatically adjusts the field of view is high and the system is complex. In addition, the information in the point cloud data is less than that in the image, and the accuracy of dynamic target detection needs to be improved.

[0096] The second method is to use three cameras to meet the demand for collecting images with different field of view angles in different scenarios. The three-camera system mainly consists of a forward-looking wide-field camera (for example, with a field of view of 150°), a main field of view camera (for example, with a field of view of 52°), and a narrow field of view camera (for example, with a field of view of 28°). Among them, the wide-field camera can capture objects that are closer to the vehicle and have a lower light range, which is suitable for urban scenes. The main field of view camera can cover most traffic scenes. The narrow field of view camera can capture distant objects and is suitable for high-speed scenes. Although this method can adapt to different scenes through three cameras with different field of view angles, the field of view angle of each camera is fixed, and the flexibility of image acquisition needs to be improved. In addition, this method has high equipment cost and high system complexity.

[0097] To address the above issues, the present invention proposes an image processing method, apparatus, and intelligent driving device. It should be understood that the image processing method proposed in the present invention can be applied not only to target detection in intelligent driving devices, but also to image processing systems including camera devices and computing platforms.

[0098] An image processing method of the present application will be described in detail below with reference to FIG. 4 to FIG. 9 .

[0099] FIG4 is a flow chart of an image processing method provided by an embodiment of the present application. As shown in FIG4 , the method can be executed by the above-mentioned intelligent driving device 200, or the method can be executed by the above-mentioned computing platform 220, or the method can be executed by the system-on-a-chip (SoC) in the above-mentioned computing platform 220, or the method can be executed by the processor in the computing platform 220. The method can also be executed by an image processing system including a camera device and a computing platform. The method can also be executed by a server. As shown in FIG4 , the method includes:

[0100] S410: Acquire a first image, where a first view angle of the first image is the field angle of a camera device that captures the first image.

[0101] It should be understood that the camera device that captures the first image can be a monocular camera, a multi-camera camera, or a depth camera. The embodiment of the present application does not limit the type of camera device.

[0102] It should also be understood that the field of view of a camera device is an inherent property of the camera device, and the field of view of a vehicle-mounted camera device is generally a fixed angle.

[0103] In this application, the field of view (FOV) represents the angular range of the physical world corresponding to different regions in an image. Different regions in the same image can have different FOVs. For example, a region farther from the camera in the physical world represented by the first image may have a smaller FOV.

[0104] Among them, the field of view of the first view corresponding to the first image is the field of view of the view corresponding to the entire area of ​​the first image before cropping. Since the first image is the original image captured by the camera device, the field of view of the first view and the field of view of the camera device are the same, that is, the field of view of the first view is equal to the field of view corresponding to the camera device that captured the first image.

[0105] S420: Cropping the area where the first target object is located in the first image to obtain a second image, wherein the first view field angle is greater than the second view field angle of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

[0106] It should be understood that the second image is the area where the first target object is located in the first image. For example, the second image is the area where one first target object is located, or it can be the area where multiple first target objects are gathered. The embodiment of the present application does not limit the number of first target objects.

[0107] It should also be understood that the second view field angle of the second image represents the angular range of the area corresponding to the second image relative to the camera device in the physical world, and the second view field angle of the second image is less than or equal to the first view field angle.

[0108] In the case where the field of view angle of the second view is equal to the field of view angle of the first view, since the second image is obtained by cropping the first image, the area of ​​the second image is smaller than that of the first image.

[0109] The view depth information of the second image is not lost, that is, the view depth information in the second image cropped from the first image is consistent with the view depth information of the corresponding area in the first image.

[0110] Optionally, S430 , determine a detection result of the first target object according to the second image.

[0111] It should be understood that the detection result of the first target object may include relative position information of the first target object in the first image, real position information of the first target object in the physical world, and category information of the first target object.

[0112] For example, the relative position information may be the position information of the detection frame of the first target object, including the horizontal and vertical positions of the first target object in the first image, as well as the depth information of the first target object in the first image. The view depth information of the first target object can be understood as the distance between the first target object and the camera acquisition device.

[0113] For another example, the real location information may be the location information of the first target object in the physical world relative to the coordinate system corresponding to the camera device (for example, when the camera device is on a vehicle, the coordinate system may be the vehicle coordinate system). Alternatively, the real location information may be the absolute location information of the first target object in the physical world, for example, the absolute location information in the Earth coordinate system, i.e., the longitude and latitude of the first target object.

[0114] In the above scheme, a second image of the area where the first target object is located is obtained by cropping the first image captured by a camera device with a fixed field of view, wherein different second images obtained by cropping different first target objects are different, and different second images have different view depth information. In this way, the flexibility of extracting the area of ​​interest in the image can be improved. Compared with ROI-based target detection, the input image of the target detection model is the original image captured by the camera device, and then ROI-based target detection is performed on the original image after size scaling. Since the original image after size scaling will lose part of the image information, such as the view information of the original image. Therefore, the image processing method before target detection in the present application can improve the detection flexibility of different target objects, thereby improving the detection accuracy of different target objects.

[0115] FIG5 is a schematic diagram of a target detection method provided in an embodiment of the present application.

[0116] The first step is to obtain a first image, where the first image includes a first target object.

[0117] As a possible implementation method, the first image is acquired by capturing the camera device 100. The embodiment of the present application does not limit the specific type of the camera device 100. For example, it can be a monocular camera, a multi-camera camera, or a depth camera.

[0118] For example, a first image is captured by an 8M uniform camera, where the field of view of the captured image of the 8M uniform camera is 30° and the resolution is 3648*2432. In this case, the field of view of the first view of the first image is equal to the field of view of the camera device, for example, 30°.

[0119] In the second step, the first image is input into the adaptive cropping model 510, and the first image is cropped according to the target cropping strategy determined by the adaptive cropping model 510 to output a second image, wherein the second view field angle of the second image is less than or equal to the first view field angle, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

[0120] As a possible implementation method, the target cropping strategy determined by the adaptive cropping model 510 is related to the depth information of the first image and the relative size of the first target object in the first image. The specific method will be described in detail in conjunction with Figures 6 and 7.

[0121] As a possible implementation manner, the target cropping strategy determined by the adaptive cropping model 510 is related to the scene category of the first image. The specific implementation manner will be described in detail with reference to FIG. 8 and FIG. 9 .

[0122] Optionally, after obtaining the second image, pixel adjustment may be performed on the second image to obtain an adjusted second image.

[0123] For example, the pixels of the second image before adjustment are 3648*2432, and the pixels of the second image after adjustment are 1280*720.

[0124] In the third step, the second image is input into the target detection model 520, and the detection result of the first target object is output.

[0125] The detailed explanation of the detection result of the first target object can be referred to S430 and will not be repeated here.

[0126] For example, the object detection model 520 may be an object detection sub-model for different types of regions, or may be a general object detection model that does not distinguish between region types.

[0127] In the above technical solution, the second image obtained by cropping the first image can improve the detection distance of the first target object. For example, at present, the first image is directly input into the target detection model, and the detection distance range of the output detection result of the first target object is 0m to 244m. In the above technical solution, the cropped second image retains the view depth information of the area where the first target object is located in the first image. Therefore, inputting the cropped second image into the target detection model can expand the range of detection distance. For example, the first target object can be located at a detection distance of 244m to 696m. In other words, the detection result currently obtained by directly inputting the original image into the target detection model is difficult to detect distant target objects due to the loss of view depth information of distant target objects in the original image caused by the scaling process of the original image. However, in the above technical solution, distant target objects included in the original image can be detected.

[0128] Furthermore, compared to performing target detection on images captured by a trinocular camera with different field-of-view angles, the processing device must process multiple original images captured by the trinocular camera to obtain target detection results. However, in the above technical solution, the target object detection results can be obtained by simply cropping a single original image and inputting the resulting cropped second image into the target detection model, thereby improving target detection efficiency.

[0129] The following will describe in detail the pre-processing process of the original image captured by the camera device with reference to FIG. 6 to FIG. 9 .

[0130] FIG6 is a schematic diagram of an image processing method provided in an embodiment of the present application. The method includes the following steps:

[0131] S610: Acquire view depth information of a first image.

[0132] As a possible implementation manner, the acquired first image is input into the depth estimation model 511 to obtain the view depth information of the first image.

[0133] The view depth information can be understood as the depth information of each pixel in the first image. Therefore, the view depth information also includes the depth information of the first target object.

[0134] It should be understood that the method of obtaining the view depth information of the first image through S610 is applicable to a monocular camera or a multi-camera.

[0135] Therefore, for the first image captured by the depth camera, the view depth information of each pixel in the first image can be acquired.

[0136] S620: Acquire size information of one or more target objects in the first image, where the first target object is a target among the one or more target objects.

[0137] As a possible implementation manner, the acquired first image is input into the target size estimation model 512 to obtain size information of one or more target objects.

[0138] For example, the target size estimation model 512 may be a lightweight detection network model. The embodiment of the present application does not limit the type of the target size estimation model 512 .

[0139] It should be understood that the size information of the target object refers to the relative size information of the target object in the first image, for example, the area occupied by the target object in the first image. If the area occupied by the target object in the first image is relatively small, the target object can be determined to be a small target object or a distant target object. If the area occupied by the target object in the first image is relatively large, the target object can be determined to be a large target object or a nearby target object.

[0140] It should be understood that there is no fixed order for the execution of S610 and S620, and the embodiment of the present application does not limit the execution order of S610 and S620.

[0141] S630 : Divide the first image into a plurality of regions according to the view depth information and the size information of the one or more target objects, wherein the plurality of regions include a first region, and the first region includes the first target object.

[0142] As a possible implementation manner, view depth information and size information of one or more target objects are input into a first region division model to divide the first image into one or more regions.

[0143] The first region division model may be a neural network model or a preset region division strategy.

[0144] For example, the types of the multiple regions may be a distant region and a near region, or may be a left region and a right region of the first image. The embodiment of the present application does not limit the types of the multiple regions.

[0145] It should be understood that the far and near regions are determined based on the depth information of one or more target objects. For example, if the depth information of the pixels in the first region is greater than or equal to a depth threshold, then the first region is a far region; if the depth information of the pixels in the first region is less than the depth threshold, then the first region is a near region. The left and right regions are determined based on the relative positions of one or more target objects in the first image.

[0146] S640: Crop a first area among the multiple areas of the first image to obtain a second image.

[0147] As a possible implementation manner, target cropping strategies corresponding to the multiple regions are determined according to the multiple regions; based on the target cropping strategy, a first region among the multiple regions of the first image is cropped to obtain a second image.

[0148] As a possible implementation manner, the area where the first target object is located in the first area is used as the clipping range, and the first image is clipped to obtain the second image.

[0149] In this way, the cropping range of the first image is reduced, and the range occupied by the first target object in the second image can be further increased, thereby further improving the accuracy of target detection.

[0150] It should be noted that the detailed clipping method based on the region type of the near region and the far region will be described in detail in conjunction with FIG. 7 .

[0151] Optionally, when the number of pixels of the first area in the first image is smaller than the number of pixels of other areas in the first image, detail enhancement is performed on the second image; and the detail-enhanced second image is used to determine the detection result of the first target object.

[0152] In this way, if the area type of the first area is a distant area, the pixels of the first target object in the first area are few, and the second image after detail enhancement can further improve the detection accuracy of the target detection.

[0153] Optionally, the pixel values ​​of the second image are adjusted to preset pixel values, or the pixel values ​​of the second image after detail enhancement are adjusted to preset pixel values; and the pixel-adjusted second image is used to determine the detection result of the first target object.

[0154] It should be understood that the preset pixel value is the pixel value requirement of the trained target detection model for the input image, and the pixel value of the training image during the target detection model training process is also the preset pixel value.

[0155] In the above technical solution, the first image is divided into regions according to the view depth information of the first image and the size information of the target object, so that a more accurate region division result can be obtained, which helps to obtain a more accurate second image including the first target object, thereby improving the accuracy of target detection.

[0156] Fig. 7 is a schematic diagram of another image processing method provided by an embodiment of the present application. Fig. 7 is illustrated by taking the case where the multiple regions are divided into distant regions and near regions as an example.

[0157] As shown in FIG7 , the region types of the multiple regions may be distant regions and near regions. Different image processing methods may be used for different types of regions.

[0158] The first one, as shown in S710 and S720 , is that if the number of target objects in the near area among the multiple areas is greater than the number of target objects in the far area, the first area with the area type of the near area is used as the clipping range.

[0159] Specifically, in S710 , when the area type of the first area is a near area, the first image is cropped with the first area as a cropping range to obtain a first cropped image, where the first cropped image has a first target view field angle.

[0160] It should be understood that the first target view field angle is the view field angle corresponding to the first target object concentration range in the first area, and the first target view field angle can be smaller than the first view field angle and larger than the field angle threshold. The field angle threshold can be used to distinguish between distant areas and near areas.

[0161] In other words, the field of view of the first target view is also the field of view of the second view of the second image.

[0162] Alternatively, the field of view angle of the first target view may be equal to the field of view angle of the first view, that is, the field of view angle of the camera device that captured the first image is retained. In this case, the first cropped image may include only the area where the first target object is located in the first area, or the first cropped image may include the first area.

[0163] Optionally, when the region type of the first region is a near region, the first image is retained, that is, the original image is retained.

[0164] Optionally, at S720 , pixel values ​​of the first cropped image are adjusted to preset pixel values ​​to obtain a second image.

[0165] At this time, the second image is an image after pixel adjustment.

[0166] Second, as shown in S710 , S730 and S740 , if the number of target objects in the distant area among the multiple areas is greater than the number of target objects in the near area, the first area with the area type of the distant area is used as the clipping range.

[0167] Specifically, in S710 , when the area type of the first area is a distant area, the first image is cropped with the first area as a cropping range to obtain a second cropped image, where the second cropped image has a second target view field angle.

[0168] It should be understood that the field of view of the second target view is the field of view angle corresponding to the first target object concentration range in the first area, and the field of view angle of the second target view can be smaller than the field of view angle threshold. The field of view angle threshold can be used to distinguish between distant areas and near areas. The field of view angle of the second target view is smaller than the field of view angle of the first view.

[0169] Optionally, at S730 , detail enhancement is performed on the second cropped image to obtain a detail-enhanced second cropped image.

[0170] Optionally, at S740 , pixel values ​​of the detail-enhanced second cropped image are adjusted to preset pixel values, and the pixel-adjusted second cropped image is used as the second image.

[0171] The third scenario, not shown in Figure 7, involves the case where there is no significant difference between the number of target objects in the distant region and the number of target objects in the near region. In this case, the first and second specific cropping strategies can be used to crop both the distant region and the near region, respectively, to obtain multiple cropped images. These cropped images can then be used sequentially as input to the target detection model to obtain detection results. Alternatively, cropped images obtained from cropping different regions can be input into target detection models of different region types to obtain detection results.

[0172] FIG8 is a schematic diagram of another image processing method provided in an embodiment of the present application. The method includes the following steps:

[0173] S810: Determine a scene category of the first image according to the first image.

[0174] As a possible implementation manner, the first image is input into the scene classification model 513, and the scene category of the first scene is output.

[0175] The scene classification model 513 is a pre-trained model. The scene category of the first scene can include traffic scenes such as urban scenes or highway scenes. The embodiments of the present application use urban scenes and highway scenes as examples for detailed description, but the embodiments of the present application do not limit the scene category of traffic scenes. For example, the scene category can also include intersection scenes or suburban road scenes.

[0176] There are two specific implementation methods for cropping the first image according to the scene category to obtain the second image. One method includes S820 and S830, and the other method includes S840 and S850.

[0177] S820: Divide the first image into a plurality of regions according to the scene category, where the plurality of regions include a first region, and the first region includes a first target object.

[0178] As a possible implementation manner, the scene category is input into the second region division model to divide the first image into one or more regions.

[0179] The second region division model may be a neural network model or a preset division strategy model.

[0180] It should be understood that a detailed explanation of the multiple areas can be found in S630 and will not be repeated here.

[0181] S830: Crop a first area among the multiple areas of the first image to obtain a second image.

[0182] It should be understood that S830 is similar to S640 and is not described in detail here for the sake of brevity. Please refer to the detailed explanation of S640 and the detailed description shown in Figure 7.

[0183] Optionally, S840 , determine a first image cropping strategy from a plurality of image cropping strategies according to the scene category.

[0184] The image cropping strategies in different scenarios may be obtained by performing regional statistical division on the training data in different scenarios.

[0185] For example, when the scene category of the first image is an urban scene, the first image cropping strategy may indicate that the first target object is concentrated in the lateral distribution area of ​​the first image as the cropping range; when the first scene category is a high-speed scene, the first image cropping strategy may indicate that the first target object is concentrated in the central area of ​​the first image as the cropping range.

[0186] S850: Crop the area where the first target object is located in the first image according to the image cropping strategy to obtain a second image.

[0187] 9 , S850 will be described in detail by taking the urban scene and the highway scene as examples.

[0188] FIG9 is a schematic diagram of another image processing method provided in an embodiment of the present application.

[0189] As shown in Figure 9, the scene categories of the first image can include urban scenes and highway scenes. Different image cropping strategies can be used for different scene types. S910 to S920 are image processing methods for urban scenes. S930 to S950 are image processing methods for highway scenes.

[0190] The image processing method for urban scenes includes S910 and S920.

[0191] S910 : When the scene category of the first image is an urban scene, the first image is cropped according to a cropping range in an image cropping strategy for urban scenes to obtain a first cropped image.

[0192] For example, the first image is cropped with the horizontal distribution area of ​​the first image as the cropping range to obtain a first cropped image.

[0193] Alternatively, optionally, when the scene category of the first image is a city scene, the first image is retained, that is, the original image is retained.

[0194] Optionally, S920 , pixel values ​​of the first cropped image are adjusted to preset pixel values ​​to obtain a second image.

[0195] At this time, the second image is an image after pixel adjustment.

[0196] The image processing method for high-speed scenes includes S930 to S950.

[0197] S930 : When the scene category of the first image is a high-speed scene, the first image is cropped according to a cropping range in an image cropping strategy for high-speed scenes to obtain a second cropped image.

[0198] For example, the first image is cropped with the central area of ​​the first image as the cropping range to obtain a second cropped image.

[0199] Optionally, at S940 , detail enhancement is performed on the second cropped image to obtain a detail-enhanced second cropped image.

[0200] Optionally, at S950 , pixel values ​​of the detail-enhanced second cropped image are adjusted to preset pixel values, and the pixel-adjusted second cropped image is used as the second image.

[0201] The above content is the image processing method of the embodiment of the present application. The image processing device will be described in detail below with reference to Figures 10 and 11. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment. Therefore, for matters not described in detail, please refer to the method embodiment above. For the sake of brevity, it will not be repeated here.

[0202] An embodiment of the present application also provides an apparatus for implementing any of the above methods. For example, an apparatus is provided that includes units (or means) for implementing each step performed by an intelligent driving device or computing platform in any of the above methods.

[0203] FIG10 is a schematic block diagram of an image processing apparatus 1000 provided in an embodiment of the present application. As shown in FIG10 , the apparatus 1000 includes: an acquisition unit 1010 and a processing unit 1020 .

[0204] The acquisition unit 1010 is configured to acquire a first image, where the field of view of a first view of the first image is the field of view of a camera device that captures the first image.

[0205] The processing unit 1020 is used to crop the area where the first target object is located in the first image to obtain a second image, where the first view field angle is greater than the second view field angle of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

[0206] Alternatively, the processing unit 1020 is configured to crop the area in the first image where the first target object is located based on the view depth information of the first image and size information of one or more target objects in the first image to obtain a second image. Alternatively, the processing unit 1020 is configured to crop the area where the first target object is located based on the scene category of the first image to obtain a second image. The first view field of view angle is greater than or equal to the second view field of view angle of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

[0207] For example, the acquisition unit 1010 may be the computing platform or a processing circuit, processor, or controller in the computing platform in Figure 2. For example, if the acquisition unit 1010 is the processor 221 in the computing platform, the processor 221 may acquire the first image captured by the camera device.

[0208] For another example, processing unit 1020 may be the computing platform in FIG2 or a processing circuit, processor, or controller in the computing platform. For example, taking processing unit 1020 as processor 222 in the computing platform, processor 222 may crop the area where the first target object is located in the first image acquired by processor 221 to obtain a second image.

[0209] The functions implemented by the above acquisition unit 1010 and the functions implemented by the processing unit 1020 can be implemented by different processors respectively, or some functions can be implemented by the same processor, or all functions can be implemented by the same processor. The embodiments of the present application do not limit this.

[0210] It should be understood that the division of the various units in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or they may be physically separated. Furthermore, the units in the device may be implemented in the form of a processor calling software; for example, the device includes a processor connected to a memory storing instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or the functions of the various units in the device, where the processor is, for example, a general-purpose processor such as a CPU or a microprocessor, and the memory is a memory within the device or a memory external to the device. Alternatively, the units in the device may be implemented in the form of hardware circuits, and the functions of some or all of the units may be implemented through the design of the hardware circuits. The hardware circuits may be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units may be implemented through the design of the logical relationships between the components within the circuits. In another implementation, the hardware circuit may be implemented using a PLD, such as an FPGA, which may include a large number of logic gate circuits, and the connections between the logic gate circuits may be configured using a configuration file to implement the functions of some or all of the above units. All units of the above apparatus may be implemented entirely in the form of software called by a processor, or entirely in the form of hardware circuits, or partially in the form of software called by a processor and the rest in the form of hardware circuits.

[0211] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and execute instructions, such as a CPU, a microprocessor, a GPU, or a DSP. In another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0212] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0213] In addition, the various units in the above apparatus may be fully or partially integrated together, or may be implemented independently. In one implementation, these units are integrated together and implemented in the form of a system-on-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or implementing the functions of the various units of the apparatus. The at least one processor may be of different types, such as a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0214] An embodiment of the present application also provides an image processing device, which includes a processor and a memory, wherein the memory is used to store instructions, and the processor executes the instructions stored in the memory to enable the device to perform the method or steps performed in the above embodiment.

[0215] Alternatively, if the device is located in a vehicle, the processor may be the processors 221 - 22n shown in FIG. 2 .

[0216] Figure 11 is a schematic block diagram of an image processing system 1100 provided in an embodiment of the present application. As shown in Figure 11 , the prediction system 1100 includes a camera device and a computing platform, wherein the computing platform may include the image processing device 1000 described above.

[0217] An embodiment of the present application further provides an intelligent driving device, which may include the above-mentioned image processing apparatus 1000 or image processing system 1100.

[0218] Optionally, the intelligent driving device may be a vehicle.

[0219] The embodiment of the present application further provides a server, which includes the above-mentioned image processing device and is further configured to send a second image or a detection result of the first target object to the intelligent driving device.

[0220] An embodiment of the present application further provides a computer program product, which includes: computer program code, which enables the computer to execute the above method when the computer program code is run on a computer.

[0221] An embodiment of the present application further provides a computer-readable medium, wherein the computer-readable medium stores a program code. When the computer program code is run on a computer, the computer executes the above method.

[0222] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or a power-on erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0223] It should be understood that in the embodiment of the present application, the memory may include a read-only memory and a random access memory, and provide instructions and data to the processor.

[0224] It should also be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0225] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0226] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0227] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0228] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0229] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0230] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0231] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for image processing, characterized in that: include: Acquire a first image, where a first view angle of the first image is a field angle of a camera device that acquires the first image; The area where the first target object is located in the first image is cropped to obtain a second image, the field of view of the first view is greater than the second field of view of the second view of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

2. The method according to claim 1, wherein The method further comprises: Acquire view depth information of the first image and size information of one or more target objects in the first image, where the first target object is a target among the one or more target objects; Divide the first image into a plurality of regions according to the view depth information and the size information, wherein the plurality of regions include a first region, and the first region includes the first target object; The step of cropping the area where the first target object is located in the first image to obtain the second image includes: The first area is cropped to obtain the second image.

3. The method according to claim 1, wherein The method further comprises: Dividing the first image into a plurality of regions according to a scene category of the first image, wherein the plurality of regions include a first region, and the first region includes the first target object; The step of cropping the area where the first target object is located in the first image to obtain the second image includes: The first area is cropped to obtain the second image.

4. The method according to claim 2 or 3, wherein: The method further comprises: When the number of pixels of the first area in the first image is smaller than the number of pixels of other areas in the first image, detail enhancement is performed on the second image.

5. The method according to claim 1, wherein The cropping of the area where the first target object is located in the first image to obtain the second image includes: determining a first image cropping strategy from a plurality of image cropping strategies according to the scene category of the first image; According to the first image cropping strategy, the area where the first target object is located in the first image is cropped to obtain the second image.

6. The method according to claim 5, wherein The method further comprises: When the scene category is a high-speed scene, detail enhancement is performed on the second image.

7. The method according to any one of claims 1 to 6, wherein: The method further comprises: A detection result of the first target object is determined based on the second image.

8. The method according to claim 7, wherein Determining a detection result of the first target object according to the second image includes: Adjusting the pixel values ​​of the second image to preset pixel values; or adjusting the pixel values ​​of the second image after detail enhancement to preset pixel values; The second image after adjusting the pixel values ​​is input into the target detection model to obtain the detection result.

9. A method for image processing, characterized in that: The method comprises: Acquire a first image, where a first view angle of the first image is a field angle of a camera device that acquires the first image; cropping the area where the first target object is located in the first image according to the view depth information of the first image and the size information of one or more target objects in the first image to obtain a second image; or cropping the area where the first target object is located according to the scene category of the first image to obtain the second image; The field of view angle of the first view is greater than or equal to the field of view angle of the second view of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

10. The method according to claim 9, wherein The step of clipping the area where the first target object is located in the first image according to the view depth information of the first image and the size information of one or more target objects in the first image to obtain the second image includes: Divide the first image into a plurality of regions according to the view depth information and the size information, wherein the plurality of regions include a first region, and the first region includes the first target object; The first area is cropped to obtain the second image.

11. The method according to claim 9, wherein The step of clipping the area where the first target object is located according to the scene category of the first image to obtain the second image includes: Dividing the first image into a plurality of regions according to the scene category, wherein the plurality of regions include a first region, and the first region includes the first target object; The first area is cropped to obtain the second image.

12. The method according to claim 10 or 11, wherein: The method further comprises: When the number of pixels of the first area in the first image is smaller than the number of pixels of other areas in the first image, detail enhancement is performed on the second image.

13. The method according to claim 9, wherein The step of cropping the area of ​​the first image where the first target object is located according to the scene category of the first image to obtain the second image includes: determining a first image cropping strategy from a plurality of image cropping strategies according to the scene category of the first image; According to the first image cropping strategy, the area where the first target object is located in the first image is cropped to obtain the second image.

14. The method according to claim 13, wherein The method further comprises: When the scene category is a high-speed scene, detail enhancement is performed on the second image.

15. The method according to any one of claims 9 to 14, characterized in that The method further comprises: A detection result of the first target object is determined based on the second image.

16. The method according to claim 15, wherein Determining a detection result of the first target object according to the second image includes: Adjusting the pixel values ​​of the second image to preset pixel values; or adjusting the pixel values ​​of the second image after detail enhancement to preset pixel values; The second image after adjusting the pixel values ​​is input into the target detection model to obtain the detection result.

17. An image processing device, characterized in that: The device comprises an acquisition unit and a processing unit: The acquisition unit is configured to acquire a first image, wherein a first view angle of the first image is a field angle of a camera device that acquires the first image; The processing unit is used to crop the area where the first target object is located in the first image to obtain a second image, the first view field angle is greater than the second view field angle of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

18. The device according to claim 17, wherein The acquisition unit is further configured to acquire view depth information of the first image and size information of one or more target objects in the first image, where the first target object is a target among the one or more target objects; The processing unit is further configured to divide the first image into a plurality of regions according to the view depth information and the size information, wherein the plurality of regions include a first region, and the first region includes the first target object; The processing unit is specifically configured to: The first area is cropped to obtain the second image.

19. The device according to claim 17, wherein The processing unit is further configured to divide the first image into a plurality of regions according to a scene category of the first image, wherein the plurality of regions include a first region, and the first region includes the first target object; The processing unit is specifically configured to: The first area is cropped to obtain the second image.

20. The device according to claim 18 or 19, characterized in that The processing unit is further configured to: When the number of pixels of the first area in the first image is smaller than the number of pixels of other areas in the first image, detail enhancement is performed on the second image.

21. The device according to claim 17, wherein The processing unit is specifically configured to: determining a first image cropping strategy from a plurality of image cropping strategies according to the scene category of the first image; According to the first image cropping strategy, the area where the first target object is located in the first image is cropped to obtain the second image.

22. The device according to claim 21, wherein The processing unit is further configured to: When the scene category is a high-speed scene, detail enhancement is performed on the second image.

23. The device according to any one of claims 17 to 22, characterized in that The processing unit is further configured to: A detection result of the first target object is determined based on the second image.

24. The device according to claim 23, wherein The processing unit is specifically configured to: Adjusting the pixel values ​​of the second image to preset pixel values; or adjusting the pixel values ​​of the second image after detail enhancement to preset pixel values; The second image after adjusting the pixel values ​​is input into the target detection model to obtain the detection result.

25. An image processing device, characterized in that: The device comprises an acquisition unit and a processing unit: The acquisition unit is configured to acquire a first image, wherein a first view angle of the first image is a field angle of a camera device that acquires the first image; The processing unit is configured to crop the area where the first target object is located in the first image according to the view depth information of the first image and the size information of one or more target objects in the first image to obtain a second image; or The processing unit is configured to crop the area where the first target object is located according to the scene category of the first image to obtain the second image; The field of view angle of the first view is greater than or equal to the field of view angle of the second view of the second image, and the view depth information of the second image is consistent with the view depth information corresponding to the area where the first target object is located in the first image.

26. The device according to claim 25, characterized in that The processing unit is specifically configured to: Divide the first image into a plurality of regions according to the view depth information and the size information, wherein the plurality of regions include a first region, and the first region includes the first target object; The first area is cropped to obtain the second image.

27. The device according to claim 25, wherein The processing unit is specifically configured to: Dividing the first image into a plurality of regions according to the scene category, wherein the plurality of regions include a first region, and the first region includes the first target object; The first area is cropped to obtain the second image.

28. The device according to claim 26 or 27, characterized in that The processing unit is further configured to: When the number of pixels of the first area in the first image is smaller than the number of pixels of other areas in the first image, detail enhancement is performed on the second image.

29. The device according to claim 25, wherein The processing unit is specifically configured to: determining a first image cropping strategy from a plurality of image cropping strategies according to the scene category of the first image; According to the first image cropping strategy, the area where the first target object is located in the first image is cropped to obtain the second image.

30. The device according to claim 29, wherein The processing unit is further configured to: When the scene category is a high-speed scene, detail enhancement is performed on the second image.

31. The device according to any one of claims 17 to 22, characterized in that The processing unit is further configured to: A detection result of the first target object is determined based on the second image.

32. The device according to claim 31, wherein The processing unit is specifically configured to: Adjusting the pixel values ​​of the second image to preset pixel values; or adjusting the pixel values ​​of the second image after detail enhancement to preset pixel values; The second image after adjusting the pixel values ​​is input into the target detection model to obtain the detection result.

33. An image processing device, characterized in that: include: a memory for storing instructions; A processor, configured to execute instructions stored in the memory, so that the apparatus performs the method according to any one of claims 1 to 8, or so that the apparatus performs the method according to any one of claims 9 to 16.

34. An image processing system, characterized in that: The device comprises a camera device and a computing platform, wherein the computing platform comprises the device according to any one of claims 17 to 24, or the computing platform comprises the device according to any one of claims 25 to 33.

35. An intelligent driving device, characterized in that: comprising an apparatus as claimed in any one of claims 17 to 24, or comprising an apparatus as claimed in any one of claims 25 to 33, or comprising a system as claimed in claim 34.

36. The intelligent driving device according to claim 35, characterized in that: The intelligent driving device is a vehicle.

37. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 8 is implemented, or the device is caused to perform the method according to any one of claims 9 to 16.

38. A computer program product, characterized in that The computer program product comprises: a computer program code, and when the computer program code is run on a computer, the method according to any one of claims 1 to 8 is implemented, or the device is caused to perform the method according to any one of claims 9 to 16.

39. A chip, characterized in that: The method comprises a circuit for performing the method according to any one of claims 1 to 8 or a circuit for performing the method according to claims 9 to 16.

Images