Methods, apparatus, media, and devices for determining sensing results

Abstract

Disclosed are a method and apparatus for determining a sensing result, a medium and a device.SOLUTION: The method includes determining a first image collected by a wide-angle camera and a second image collected by a narrow-angle camera, a field of view of the narrow-angle camera being smaller than a field of view of the wide-angle camera, determining a first perception result corresponding to each distance range based on the first image, the second image, and a perception task model corresponding to at least one distance range, and determining a target perception result based on the first perception result corresponding to each distance range.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] This disclosure relates to computer vision technology, and particularly to a method, apparatus, medium, and device for determining perception results.

Background Art

[0002] In advanced driver assistance systems, it is generally necessary to recognize objects at various distances in front of the vehicle. In related technologies, multi-scale feature extraction is generally performed based on high-resolution images, and objects in each distance range are recognized based on the multi-scale features. However, multi-scale feature extraction and object recognition tend to cause a relatively large computational amount of the perception task model, which is disadvantageous for deploying the model on an in-vehicle terminal, and the recall rate for distant objects is relatively low.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Embodiments of this disclosure provide a method, apparatus, medium, and device for determining perception results, which can reduce the computational amount of the perception task model and improve the recall rate of distant objects.

Means for Solving the Problems

[0004] A method for determining a perception result according to a first aspect of this disclosure includes: determining a first image collected by a wide-angle camera and a second image collected by a narrow-angle camera, where the field of view of the narrow-angle camera is smaller than the field of view of the wide-angle camera; determining a first perception result corresponding to each of the distance ranges based on the first image, the second image, and a perception task model corresponding to at least one distance range; and determining a target perception result based on the first perception results corresponding to each of the distance ranges.

[0005] An apparatus for determining sensing results according to a second aspect of the present disclosure includes a first processing module for determining a first image collected by a wide-angle camera and a second image collected by a narrow-angle camera, wherein the field of view of the narrow-angle camera is smaller than the field of view of the wide-angle camera; a second processing module for determining a first sensing result corresponding to each of the distance ranges based on the first image, the second image and a sensing task model corresponding to at least one distance range; and a third processing module for determining a target sensing result based on the first sensing result corresponding to each of the distance ranges.

[0006] A computer-readable storage medium according to a third aspect of the present disclosure stores a computer program for performing a method for determining a sensing result as described in any of the above embodiments of the present disclosure.

[0007] A fourth aspect of the present disclosure includes an electronic device, a processor, and a memory for storing instructions that the processor can execute, the processor being used to read and execute the instructions from the memory to implement a method for determining a sensing result as described in any of the above embodiments of the present disclosure.

[0008] A fifth aspect of the present disclosure provides a computer program product in which, when an instruction in the computer program product is executed by a processor, performs a method for determining a sensing result as described in any of the above embodiments of the present disclosure. [Effects of the Invention]

[0009] The method, apparatus, medium, and equipment for determining sensing results according to the above embodiment of this disclosure can achieve object detection in each distance range by combining a wide-angle image (first image) collected by a wide-angle camera and a narrow-angle image (second image) collected by a narrow-angle camera with sensing task models of different distance ranges, obtain a first sensing result corresponding to each distance range, and further determine a target sensing result by combining the first sensing results of each distance range. Since the wide-angle camera has high sensing accuracy for nearby objects, while the narrow-angle camera has high sensing accuracy for distant objects, the wide-angle and narrow-angle images can cover objects in the entire distance range from near to far. Furthermore, by combining them with sensing task models of different distance ranges, accurate reproduction of objects within the entire distance range can be achieved, and the reproduction rate of distant objects can be significantly improved while guaranteeing the reproduction rate of nearby objects. Furthermore, by using sensing task models for different distance ranges to achieve object detection within different distance ranges, it becomes unnecessary to rely on multiscale feature extraction from high-resolution images. This effectively reduces the computational complexity of the sensing task model and facilitates its deployment to terminals. [Brief explanation of the drawing]

[0010] [Figure 1] This is one exemplary application scenario of the method for determining the sensing results related to this disclosure. [Figure 2] This is a flowchart of a method for determining a sensing result according to one exemplary embodiment of the present disclosure. [Figure 3] This is a flowchart of a method for determining a sensing result according to another exemplary embodiment of the present disclosure. [Figure 4] This is a flowchart of a method for determining a sensing result according to yet another exemplary embodiment of the present disclosure. [Figure 5] This is a flowchart of a method for determining a sensing result according to yet another exemplary embodiment of the present disclosure. [Figure 6] This is a flowchart of a method for determining a sensing result according to yet another exemplary embodiment of the present disclosure. [Figure 7] This is a flowchart for determining the sensing result according to one exemplary embodiment of the present disclosure. [Figure 8] This is a schematic diagram of the trimming principle of an input image corresponding to a sensing task model D according to one exemplary embodiment of the present disclosure. [Figure 9] This is a schematic diagram of the trimming principle of an input image corresponding to a sensing task model E according to one exemplary embodiment of the present disclosure. [Figure 10] This is a schematic diagram of the trimming principle of an input image corresponding to a sensing task model F according to one exemplary embodiment of the present disclosure. [Figure 11] This is a schematic diagram of the structure of an apparatus for determining sensing results according to one exemplary embodiment of the present disclosure. [Figure 12] This is a schematic diagram of the structure of a device for determining sensing results according to another exemplary embodiment of the present disclosure. [Figure 13] This is a schematic diagram of the structure of an apparatus for determining sensing results according to yet another exemplary embodiment of the present disclosure. [Figure 14] This is a structural diagram of an electronic device according to an embodiment of the present disclosure. [Modes for carrying out the invention]

[0011] Hereinafter, exemplary embodiments of the Disclosure will be described in detail with reference to the drawings in order to explain the Disclosure. Clearly, the embodiments described are only a selection of the embodiments of the Disclosure, not all embodiments, and the Disclosure is not limited to exemplary embodiments.

[0012] Unless otherwise specifically stated, the relative arrangements of the components and steps, the formulas, and the numerical values ​​described in these embodiments do not limit the scope of this disclosure.

[0013] Summary of this disclosure In the process of realizing this disclosure, the inventors discovered the following: Advanced driver assistance systems generally need to recognize objects at various distances in front of the vehicle, and related technologies commonly perform multiscale feature extraction based on high-resolution images and recognize objects at each distance range based on multiscale features. However, performing multiscale feature extraction and object recognition based on high-resolution images tends to result in a complex network structure for the sensing task model, which increases the computational load of the sensing task model and is disadvantageous for deploying the model to an in-vehicle terminal. Furthermore, related technologies commonly rely on wide-angle images collected by cameras with a relatively large field of view, and wide-angle images have a relatively high recall rate for nearby objects but a relatively low recall rate for distant objects.

[0014] Example Overview Figure 1 shows one exemplary application scene of the method for determining the sensing result according to this disclosure. As shown in Figure 1, as a vehicle 11 travels on a road, a wide-angle camera 12 on the vehicle 11 can collect a wide-angle image (referred to as the first image) of the area in front of the vehicle 11, and a narrow-angle camera 13 can collect a narrow-angle image (referred to as the second image) of the area in front of the vehicle 11. The field of view (FOV) of the wide-angle camera 12 is larger than that of the narrow-angle camera 13, that is, FOV1 in the figure is larger than FOV2. For example, FOV1 may be 120 degrees and FOV2 may be 30 degrees. By having a coverage area where the wide-angle camera 12 and the narrow-angle camera 13 overlap, the narrow-angle camera 13 can be used as a supplement to the wide-angle camera 12 to improve the accuracy of reproducibility of distant objects. The objects to be sensed may include, but are not limited to, a curb 14, lane markings 15, other vehicles 16, other objects 17, etc., in front of the vehicle 11. Other objects 17 may include, for example, pedestrians, cyclists, traffic lights, signs, traffic cones, road arrows, crosswalks, stop lines, etc. When the first image collected by the wide-angle camera 12 and the second image collected by the narrow-angle camera 13 are determined by the method for determining the sensing result of this disclosure, a first sensing result corresponding to each distance range can be determined based on the first image, the second image and a sensing task model corresponding to at least one distance range, and a target sensing result can be determined based on the first sensing result corresponding to each distance range. Since the wide-angle camera 12 has high sensing accuracy for nearby objects, while the narrow-angle camera has high sensing accuracy for distant objects, the wide-angle and narrow-angle images can cover objects in the entire distance range from nearby to far away. Furthermore, by combining them with sensing task models for different distance ranges, accurate reproduction of objects within the entire distance range can be achieved, and the reproduction rate of distant objects can be significantly improved while guaranteeing the reproduction rate of nearby objects. Furthermore, by using sensing task models for different distance ranges to achieve object detection within different distance ranges, it becomes unnecessary to rely on multiscale feature extraction from high-resolution images. This effectively reduces the computational complexity of the sensing task model and facilitates its deployment to terminals.The method for determining the perception result of the present disclosure is not limited to being used in the intelligent driving field or scene, and may also be applied to other fields or scenes such as the security monitoring field.

[0015] Exemplary method FIG. 2 is a flowchart of a method for determining a perception result according to one exemplary embodiment of the present disclosure. This embodiment can be specifically applied to an electronic device such as an in-vehicle computing platform. As shown in FIG. 2, the method includes steps 201 to 203.

[0016] In step 201, determine a first image collected by a wide-angle camera and a second image collected by a narrow-angle camera.

[0017] Here, the viewing field of the narrow-angle camera is smaller than that of the wide-angle camera, as shown in FIG. 1. The viewing field ranges of the wide-angle camera and the narrow-angle camera have an overlapping area. For example, both the wide-angle camera and the narrow-angle camera are front-viewpoint cameras and are used to sense objects in front of the vehicle. The objects may include, for example, curbs, lane markings, other vehicles, pedestrians, cyclists, traffic lights, signboards, road markings, etc. The first image collected by the wide-angle camera is an image corresponding to the shooting range of the wide-angle camera, that is, a wide-angle image. The second image collected by the narrow-angle camera is an image corresponding to the shooting range of the narrow-angle camera, that is, a narrow-angle image.

[0018] In step 202, based on the first image, the second image, and a perception task model corresponding to at least one distance range, determine a first perception result corresponding to each distance range respectively.

[0019] Here, at least one distance range may include one or more distance ranges. Each distance range may correspond to one or more sensing task models. That is, the number of sensing task models corresponding to each distance range may be one or more. By performing object sensing processing with sensing task models corresponding to different distance ranges, a first sensing result corresponding to each distance range can be obtained. The first sensing result may include sensing results corresponding to one or more types of objects. The number of objects of each type may be one or more. The sensing results corresponding to each type of object may include sensing results for each object of that type. The sensing results for each object may include target detection results, semantic segmentation results, etc., and the task types included in the specific sensing results may be set according to actual needs, and the embodiments of this disclosure are not limited thereto.

[0020] In some selectable embodiments, at least one distance range generally includes multiple distance ranges in order to cover the entire distance range in front of the vehicle. For example, a sensing task model may be set up for at least one distance range (referred to as the first distance range) for a wide-angle first image. A sensing task model may be set up for at least one distance range (referred to as the second distance range) for a narrow-angle second image.

[0021] In some selectable embodiments, any one of the distance ranges may include distance ranges corresponding to one or more types of objects. For example, taking a wide-angle camera as an example, distance range a corresponds to sensing task model A, and if sensing task model A is a multi-task sensing model and can sense multiple types of objects simultaneously, then sensing task model A has different effective sensing distance ranges for different objects, and distance range a includes the distance ranges for different objects. For example, sensing task model A can effectively sense other vehicles (other vehicles present around its own vehicle) within a range of 0 to 55 meters, sensing task model A can effectively sense pedestrians or cyclists within a range of 0 to 21 meters, and sensing task model A can effectively sense traffic lights within a range of 0 to 28 meters. In other words, distance range a includes the range of 0 to 55 meters for other vehicles, the range of 0 to 21 meters for pedestrians or cyclists, and the range of 0 to 28 meters for traffic lights. For example, sensing task model B can effectively detect other vehicles within a range of 55 to 110 meters, pedestrians or cyclists within a range of 21 to 43 meters, and traffic lights within a range of 28 to 57 meters. Sensing task model C can effectively detect other vehicles within a range of 110 to 220 meters, pedestrians or cyclists within a range of 43 to 87 meters, and traffic lights within a range of 57 to 114 meters. In short, a single sensing task model may have different effective sensing distance ranges for different types of objects, and multiple sensing task models can cover effective sensing across multiple distance ranges for a single type of object. By combining sensing task models with different distance ranges corresponding to wide-angle and narrow-angle images, it is possible to achieve sensing reproduction of objects across all distance ranges, guaranteeing the sensing reproduction rate and accuracy of near-range objects while significantly improving the sensing reproduction rate and accuracy of far-range objects.

[0022] In step 203, the target detection result is determined based on the first detection result corresponding to each distance range.

[0023] Here, after obtaining a first sensing result corresponding to each distance range, the target sensing result can be determined by combining the first sensing results corresponding to each distance range. For example, by combining the wide-angle sensing result and the narrow-angle sensing result of the vehicle's forward view to obtain a vehicle forward view sensing result, the target sensing result can include the sensing results of all objects within the entire distance range. The entire distance range means the overall distance range covered by the wide-angle image and the narrow-angle image. Distance may also mean the vertical distance from the camera (wide-angle camera or narrow-angle camera). In a vehicle forward view sensing scene, distance may also mean the vertical distance from the vehicle itself.

[0024] The method for determining the sensing result according to this embodiment combines wide-angle images collected by a wide-angle camera and narrow-angle images collected by a narrow-angle camera with sensing task models for different distance ranges to realize object detection for each distance range, obtain a first sensing result corresponding to each distance range, and further determine the target sensing result by combining the first sensing results for each distance range. Since the wide-angle camera has high sensing accuracy for nearby objects, while the narrow-angle camera has high sensing accuracy for distant objects, the wide-angle and narrow-angle images can cover objects in the entire distance range from near to far. Furthermore, by combining them with sensing task models for different distance ranges, accurate reproduction of objects within the entire distance range can be achieved, and the reproduction rate of distant objects can be significantly improved while guaranteeing the reproduction rate of nearby objects. In addition, by realizing object detection in different distance ranges using sensing task models for different distance ranges, it is not necessary to rely on multiscale feature extraction of high-resolution images, thereby effectively reducing the computational load of the sensing task model and facilitating the deployment of the model to the terminal.

[0025] Figure 3 is a flowchart of a method for determining a sensing result according to another exemplary embodiment of the present disclosure.

[0026] In some optional embodiments, step 202, which determines a first sensing result corresponding to each distance range based on a sensing task model corresponding to a first image, a second image, and at least one distance range, as shown in Figure 3 above, may specifically include steps 2021 to 2023.

[0027] In step 2021, wide-angle sensing results corresponding to each first distance range are determined based on the first image and a sensing task model for at least one first distance range corresponding to the first image.

[0028] Here, each first distance range may include distance ranges corresponding to one or more types of objects. Each first distance range and its corresponding sensing task model may be set based on the effective sensing distance range of the wide-angle camera. For objects of the same type, multiple first distance ranges correspond to different longitudinal distance ranges of that type of object relative to the vehicle. For example, the above ranges for other vehicles could be 0 to 55 meters, 55 to 110 meters, and 110 to 220 meters. The sensing task models for the three first distance ranges can each achieve effective sensing and reproduction of other vehicles within these three distance ranges, thereby covering effective reproduction of other vehicles in the 0 to 220 meter range. The wide-angle effective sensing ranges for different types of objects may differ. For example, for pedestrians or cyclists, the ranges 0 to 21 meters, 21 to 43 meters, and 43 to 87 meters can be used to achieve effective sensing and reproduction of pedestrians or cyclists within the 0 to 87 meter range. This relates primarily to the relationship between the size, distance, and minimum pixel required for detection in an image of different types of objects. For example, with the same camera, if objects are at the same distance, larger objects are easier to detect and reproduce, while smaller objects are more difficult to detect and reproduce. In practical applications, multiple first distance ranges and corresponding detection task models may be set up depending on the camera's performance parameters, distance, and the actual physical size of the objects. The wide-angle detection results corresponding to each first distance range include the detection results of each object detected from the input image by the detection task model for that first distance range.

[0029] In some selectable embodiments, the first image may be used as the input image for a sensing task model corresponding to any one of the first distance ranges, that is, the sensing task model performs sensing processing on the first image and obtains wide-angle sensing results corresponding to the first distance range.

[0030] In some selectable embodiments, a first image may be preprocessed for a sensing task model corresponding to any one of the first distance ranges, and the preprocessed image may be used as the input image for the sensing task model. The preprocessing may include at least one of the following: scaling, cropping, etc. Scaling may be implemented by downsampling (subsampling), that is, downsampling the first image to obtain an image at a relatively low scale, such as 1 / 2 scale, 1 / 4 scale, or 1 / 8 scale, thereby reducing the size of the input image for the sensing task model and improving the sensing processing efficiency. Cropping may be implemented by setting cropping parameters based on the effective distance ranges covered by different sensing task models. For example, if the sensing task model needs to cover a distance range of 0 to 55, the corresponding region may be cropped from the first image based on the region occupied by the distance range of 0 to 55 in the first image, and this may be used as the input image for the sensing task model. On the one hand, this improves the effectiveness of the input image, which helps to improve the accuracy and precision of the model sensing results, and on the other hand, it further reduces the size of the input image, reduces the computational complexity of the sensing task model, and further improves the sensing processing efficiency.

[0031] In step 2022, narrow-angle sensing results corresponding to each second distance range are determined based on the second image and the sensing task model of at least one second distance range corresponding to the second image.

[0032] Here, each second distance range may include distance ranges corresponding to one or more types of objects. For objects of the same type, the distance covered by at least some sub-ranges of the second distance range is greater than the distance covered by the first distance range. That is, the total distance of the second distance range may be greater than the distance within the first distance range, and it may have ranges that partially overlap with the first distance range. For example, if the object is another vehicle and the furthest first distance range in the wide-angle is in the range of 110 meters to 220 meters, then at least one second distance range may include the range of 134 meters to 551 meters or the range of 220 meters to 551 meters.

[0033] In some optional embodiments, at least one second distance range may include one or more second distance ranges, for example, taking another vehicle as an example, one second distance range may include the range of 134 meters to 300 meters of the other vehicle, and another second distance range may include the range of 300 meters to 600 meters of the other vehicle. The number of second distance ranges may be set according to the actual sensing requirements, and embodiments of the present disclosure are not limited thereto.

[0034] In step 2023, the first sensing result corresponding to each distance range is determined based on the wide-angle sensing result corresponding to each first distance range and the narrow-angle sensing result corresponding to each second distance range.

[0035] Here, after obtaining wide-angle sensing results corresponding to each first distance range and narrow-angle sensing results corresponding to each second distance range, the sensing results for each distance range may be designated as the first sensing results corresponding to that distance range. For example, the wide-angle sensing results corresponding to each first distance range may be designated as the first sensing results corresponding to that first distance range, and the narrow-angle sensing results corresponding to each second distance range may be designated as the first sensing results corresponding to that second distance range.

[0036] In this embodiment, for a wide-angle camera, at least one first distance range sensing task model covers the detection and reproduction of objects within the effective sensing range of the wide-angle, and for a narrow-angle camera, at least one second distance range sensing task model effectively senses distant objects to supplement the wide-angle sensing results and improve the detection and reproduction rate of distant objects. Furthermore, it does not rely on acquiring object sensing for different distance ranges using multiple sensing task models and performing multiscale feature extraction on high-resolution images, thereby significantly reducing the complexity of the network structure of the sensing task models, lowering the computational requirements of the models, and facilitating the deployment of the models to terminals.

[0037] Figure 4 is a flowchart of a method for determining a sensing result according to yet another exemplary embodiment of the present disclosure.

[0038] In some optional embodiments, as shown in Figure 4 above, step 2021, which determines wide-angle sensing results corresponding to each first distance range based on a sensing task model of a first image and at least one first distance range corresponding to the first image, may include steps 20211 and 20212.

[0039] In step 20211, a third image is determined based on the first image, with multiple scales corresponding to the first image.

[0040] Here, multiple scales may mean multiple different resolutions; that is, different scales correspond to different resolutions. The third image of multiple scales may include at least two of the following: the first image at the original scale (which may be expressed as 1 / 1 scale), an image at 1 / 2 scale of the first image, an image at 1 / 4 scale of the first image, an image at 1 / 8 scale of the first image, and so on. An image at 1 / 2 scale of the first image means an image whose height and width are both 1 / 2 of the first image, an image at 1 / 4 scale of the first image means an image whose height and width are both 1 / 4 of the first image, and an image at 1 / 8 scale of the first image means an image whose height and width are both 1 / 8 of the first image. Taking the 1 / 4 scale image as an example, if the resolution of the first image is expressed as H*W, then the 1 / 4 scale image of the first image is (H / 4)*(W / 4). For example, if the resolution of the first image is 2160*3840, then the resolution of a quarter-scale image of the first image is 540*960.

[0041] In some selectable embodiments, among a plurality of third images at different scales, the third images at scales other than the original scale image may be obtained by downsampling or any other form of processing on the first image, and are not specifically limited.

[0042] In step 20212, wide-angle sensing results corresponding to each first distance range are determined based on the sensing task models corresponding to the third image of each scale and each first distance range.

[0043] Here, the sensing task models corresponding to different first distance ranges may utilize third images of different scales. Images of the same image at different scales may contain different levels of features of the object, and images of the same scale will have different sensing effects on objects of different sizes. For example, a larger scale results in higher resolution, making objects in the image sharper, while conversely, objects become blurred. Based on this, the scale corresponding to each sensing task model can be set by combining the computational requirements of the model and the sensing conditions of objects within the distance range that the model needs to cover. For example, for a sensing task model in the near-range, objects within the near-range generally occupy a relatively large area in the image, and the corresponding objects can be effectively reproduced even using images of a relatively small scale. Therefore, considering the computational requirements of the sensing task model, for a sensing task model in the near-range, images of a relatively small scale may be sampled as input images to reduce the computational complexity of the model.

[0044] This embodiment achieves object detection for each first distance range using multiple images of different scales corresponding to a first image and sensing task models for different first distance ranges. The sensing task models for different first distance ranges may use images of different scales as input images. By reducing the scale of the input images for the sensing task models, the computational complexity of the sensing task models can be reduced, thereby further improving the sensing efficiency of the models.

[0045] In some selectable embodiments, step 20212 determines the wide-angle sensing result corresponding to each first distance range, based on a sensing task model corresponding to each third image of each scale and each first distance range, respectively. The steps include determining a target scale and trimming parameters corresponding to a target distance range, where different trimming parameters correspond to different distance ranges; determining a first target image from each third image based on the target scale; determining a fourth image based on the trimming parameters and the first target image; and performing sensing processing on the fourth image based on a sensing task model corresponding to the target distance range to obtain a wide-angle sensing result corresponding to the target distance range.

[0046] Here, the target scale (which may also be called the first target scale to distinguish it from the target scale in the narrow-angle case) corresponding to the target distance range (which may also be called the first target distance range to distinguish it from the target distance range in the narrow-angle case) means the scale of the third image required for the input image of the sensing task model corresponding to the target distance range. In other words, the third image of the target scale among multiple scales is used to determine the input image of the sensing task model for the target distance range. For example, the third image of the target scale is used as the input image of the sensing task model for the target distance range, or the third image of the target scale is trimmed and the trimmed area is used as the input image. The trimming parameters corresponding to the target distance range are the parameters required to trim the third image of the target scale. The trimming parameters corresponding to the target distance range may include parameters for determining the trimming area, for example, parameters for determining the boundary of the trimming area. Different trimming parameters correspond to different distance ranges. The specific correspondence may be pre-set and stored according to the size of the area occupied by the actual distance range in the image. Wide-angle sensing results corresponding to the target distance range are called wide-angle sensing results because they are sensing results obtained based on wide-angle images.

[0047] In some selectable embodiments, a target scale corresponding to each first distance range may be pre-set, and the correspondence between each first distance range and the target scale may be stored, allowing the target scale corresponding to the target distance range to be determined based on the correspondence when in use. For example, the target scale corresponding to the first distance range a may be a 1 / 4 scale, the target scale corresponding to the first distance range b may be a 1 / 2 scale, and the target scale corresponding to the first distance range c may be a 1 / 1 scale.

[0048] In some selectable embodiments, after determining the target scale and cropping parameters corresponding to the target distance range, the first target image can be determined from each third image based on the target scale; that is, the third image with the same scale as the target scale among the third images of multiple scales is designated as the first target image. For example, if the target scale is 1 / 2 scale, the third image with a 1 / 2 scale is designated as the first target image.

[0049] In some selectable embodiments, the first target image may be trimmed based on trimming parameters to obtain a fourth image.

[0050] In some selectable embodiments, the sensing task model corresponding to the target distance range may be a multi-task sensing model, a single-task sensing model, or the like, and the specific sensing task model is not limited. A multi-task sensing model is a model that can simultaneously realize the sensing of multiple types of objects and / or multiple types of sensing results. The multiple types of objects may include at least two of the following: curbs, lane markings, other vehicles, cyclists, pedestrians, traffic lights, traffic cones, etc. The multiple types of sensing results may include, for example, target detection results and semantic segmentation results.

[0051] This embodiment obtains an input image for a sensing task model by scaling down and cropping the first image, significantly reducing the amount of data in the input image. This effectively reduces the computational complexity of the model, contributing to further improvements in sensing processing efficiency and facilitating the deployment of the model on terminals.

[0052] In some selectable embodiments, as shown in Figure 4 above, step 2022, which determines the narrow-angle sensing result corresponding to each second distance range based on a sensing task model of a second image and at least one second distance range corresponding to the second image, may include steps 20221 and 20222.

[0053] In step 20221, a fifth image is determined based on the second image, with multiple scales corresponding to the second image.

[0054] Here, the fifth image with multiple scales may include at least two of the following: the second image at the original scale, an image at half the scale of the second image, an image at quarter the scale of the second image, an image at eighth the scale of the second image, and so on. The specific operation for determining the fifth image with multiple scales corresponding to the second image is similar to that of the third image in the above embodiment and will not be described here.

[0055] In step 20222, the narrow-angle sensing results corresponding to each second distance range are determined based on the sensing task models corresponding to the fifth image of each scale and each second distance range.

[0056] Here, the sensing task models corresponding to different second distance ranges may utilize fifth images of different scales. Specifically, the scale corresponding to each second distance range can be determined by combining the computational requirements of the sensing task model and the effective sensing distance range that the sensing task model needs to cover, depending on the sensing status of objects within different second distance ranges by the narrow-angle camera. The specific operation for determining the narrow-angle sensing results corresponding to each second distance range is similar to that of the wide-angle sensing results in the previously described embodiment, and will not be explained here.

[0057] This embodiment achieves object detection for each second distance range by using multiple images of different scales corresponding to a second image and sensing task models for different second distance ranges. As a result, sensing task models for different second distance ranges can use images of different scales as input images, effectively reducing the scale of the input images for the sensing task models, thereby reducing the computational complexity of the sensing task models and further improving the model sensing efficiency.

[0058] In some selectable embodiments, step 20222, which determines narrow-angle sensing results corresponding to each second distance range based on a fifth image for each scale and a sensing task model corresponding to each second distance range, includes a step of determining a target scale and trimming parameters corresponding to a target distance range, with any of the second distance ranges being the target distance range, wherein different trimming parameters correspond to different distance ranges; a step of determining a second target image from each fifth image based on the target scale; a step of determining a sixth image based on the trimming parameters and the second target image; and a step of performing sensing processing on the sixth image based on the sensing task model corresponding to the target distance range to obtain narrow-angle sensing results corresponding to the target distance range.

[0059] Here, the target scale (which may also be called the second target scale) corresponding to the target distance range (which may also be called the second target distance range) is the scale of the fifth image required for the input image of the sensing task model corresponding to the target distance range. In other words, the fifth image of the target scale among multiple scales is used to determine the input image of the sensing task model for the target distance range. For example, the fifth image of the target scale is used as the input image of the sensing task model for the target distance range, or the fifth image of the target scale is cropped and the cropped area is used as the input image. The cropping parameters corresponding to the target distance range are the parameters required to crop the fifth image of the target scale. The cropping parameters corresponding to the target distance range may include parameters for determining the cropping area, for example, parameters for determining the boundary of the cropping area. Different cropping parameters correspond to different second distance ranges. The specific correspondence may be pre-set and stored according to the size of the area occupied by the actual second distance range in the image. The narrow-angle sensing result corresponding to the target distance range is called the narrow-angle sensing result because it is a sensing result obtained by sensing based on a narrow-angle image.

[0060] In some selectable embodiments, a target scale corresponding to each second distance range may be pre-set, and the correspondence between each second distance range and the target scale may be stored, allowing the target scale corresponding to the target distance range to be determined based on the correspondence when in use. For example, the target scale corresponding to the second distance range g may be a 1 / 4 scale, the target scale corresponding to the second distance range h may be a 1 / 2 scale, and the target scale corresponding to the second distance range t may be a 1 / 1 scale.

[0061] In some selectable embodiments, after determining the target scale and cropping parameters corresponding to the target distance range, a second target image can be determined from each fifth image based on the target scale; that is, the fifth image with the same scale as the target scale among the fifth images of multiple scales is designated as the second target image. For example, if the target scale is 1 / 2 scale, the fifth image at 1 / 2 scale is designated as the second target image.

[0062] In some selectable embodiments, a second target image may be trimmed based on trimming parameters to obtain a sixth image.

[0063] In several selectable embodiments, the sensing task model corresponding to the target distance range may be a multi-task sensing model, a single-task sensing model, or the like, and the specific sensing task model is not limited.

[0064] This embodiment obtains the input image for the sensing task model by scaling down and cropping the second image, significantly reducing the amount of data in the input image. This effectively reduces the computational complexity of the model, contributing to further improvements in sensing processing efficiency and facilitating the deployment of the model on terminals.

[0065] Figure 5 is a flowchart of a method for determining a sensing result according to yet another exemplary embodiment of the present disclosure.

[0066] In one selectable embodiment, the method of the embodiment of the present disclosure may further include steps 301 to 303, as shown in Figure 5.

[0067] In step 301, the target fifth image for the first preset scale is determined from the fifth image for each scale.

[0068] Here, the first preset scale may be set according to the sensing requirements of the preset object type. The preset object type may be any object type. For example, the preset object type may be a traffic light, a lane marking, etc. Different preset object types may have the same or different first preset scales. For example, the preset object type may be a traffic light and the first preset scale may be 1 / 2 scale. Or, the preset object type may be a lane marking and the first preset scale may be 1 / 4 scale. This is merely illustrative and does not limit the embodiments of the present disclosure. The fifth image of the first preset scale among the fifth images of each scale is defined as the target fifth image.

[0069] In step 302, the fifth target image is trimmed based on the preset trimming parameters of the preset object type, and the third target image is obtained.

[0070] Here, the preset object type may be set according to the actual sensing requirements. For example, the preset object type may be a traffic light, lane markings, etc. The preset trimming parameter of the preset object type may be set according to the sensing distance range requirements of the preset object type. For example, if the sensing purpose of the preset object type is to improve the recall rate of distant objects, the preset trimming parameter of the preset object type is capable of trimming the area corresponding to the distant range in the target fifth image. If the sensing purpose of the preset object type is to compensate for the problem of missed detection of near lane markings in the sensing task model for the second distance range, the preset trimming parameter is capable of trimming the area corresponding to the near range in the target fifth image. Based on the preset trimming parameter of the preset object type, the corresponding area is trimmed from the target fifth image as the third target image.

[0071] In step 303, sensing processing is performed on the third target image based on the sensing task model corresponding to the preset object type, and narrow-angle sensing results corresponding to the preset object type are obtained.

[0072] Here, the sensing task model corresponding to a preset object type is a model dedicated to sensing objects of the preset object type. For example, the sensing task model for a preset object type may be a narrow-angle lane marking detection model for sensing lane markings at close range, a narrow-angle traffic light detection model for sensing traffic lights at long range, etc., and is not specifically limited. A third target image is inferred based on the sensing task model, and a narrow-angle sensing result corresponding to the preset object type is obtained based on the inference result. The narrow-angle sensing result corresponding to the preset object type may include, for example, at least one sensing result from among target detection results and semantic segmentation results for each sensed object of the preset object type.

[0073] In some selectable embodiments, the number of preset object types may be one or more. For example, two preset object types may be set, such as traffic lights and lane markings, and each preset object type has a corresponding sensing task model for realizing the sensing task of the corresponding preset object type.

[0074] Step 203, which determines the target detection result based on the first detection result corresponding to each distance range, The step 2031 may include determining a target detection result based on a first detection result corresponding to each distance range and a narrow-angle detection result corresponding to a preset object type.

[0075] Here, the first sensing result corresponding to each distance range and the narrow-angle sensing result corresponding to the preset object type can be combined to obtain the target sensing result.

[0076] In some selectable embodiments, the first sensing results for each distance range and the narrow-angle sensing results corresponding to the preset object type can be merged to obtain a target sensing result. The merging process may include coordinate system transformation, deduplicate sensing results for repeatedly sensed objects, aggregation of non-duplicate objects, and splicing fusion of objects spanning multiple distance ranges (i.e., one object (such as a lane marking or road edge) exists in multiple distance ranges due to its large size). Here, the purpose of the coordinate system transformation is to integrate the wide-angle sensing results and narrow-angle sensing results into the same coordinate system, thereby determining object matching and whether or not an object has been repeatedly sensed. For repeatedly sensed objects, if the entire object is in the narrow-angle sensing result, the narrow-angle sensing result is used as the target sensing result for that object, and the wide-angle sensing result for that object is deleted.

[0077] In some selectable embodiments, the sensing results of sensing task models with different distance ranges may not require deduplication because they are already clearly separated by distance range, and deduplication is performed on the sensing results of sensing task models with overlapping distance ranges.

[0078] In some selectable embodiments, if the objects sensed by different sensing task models are different, there is no need to perform duplicate deduplicating. For example, if one sensing task model senses other vehicles, cyclists, and pedestrians, and another sensing task model senses curbs and lane markings, there is no need to perform duplicate deduplicating on the sensing results of these two sensing task models. The specific fusion method may be set according to the actual situation, and the embodiments of this disclosure are not limited thereto.

[0079] This embodiment enables the detection of objects of a specific object type, and in order to supplement the narrow-angle detection results in a second distance range, compensate for the defects in the narrow-angle detection results, and improve the effectiveness and reliability of the detection results, specific trimming parameters can be set for a specific object type. For example, a narrow-angle traffic light detection task model performs detection processing on a narrow-angle trimmed image in a long-range area to improve the recall rate of long-range traffic lights. A narrow-angle lane marking detection task model performs detection processing on a narrow-angle trimmed image in a short-range area to compensate for the defect that the narrow-angle detection results in a second distance range tend to miss detecting short-range lane markings, and improve the recall rate of narrow-angle detection of short-range lane markings.

[0080] In some selectable embodiments, step 302, which trims a fifth target image based on preset trimming parameters of a preset object type to obtain a third target image, may include the steps of determining the vanishing point pixel coordinates in the fifth target image, and trimming the fifth target image centered on the vanishing point pixel coordinates based on the preset trimming parameters to obtain a third target image.

[0081] Here, the vanishing point pixel coordinates in the target fifth image may be determined based on the calibration parameters of the narrow-angle camera. For example, the calibration parameters of the narrow-angle camera include the vanishing point pixel coordinates in the second image of the narrow-angle camera, and the vanishing point pixel coordinates in the target fifth image are determined based on the pixel mapping relationship between the target fifth image and the second image. Alternatively, the vanishing point pixel coordinates in the fifth image of each scale may be calculated and stored in advance based on the pixel mapping relationship between the second image and the fifth image of each scale and the calibrated vanishing point pixel coordinates, and the vanishing point pixel coordinates in the target fifth image can be directly obtained from the storage area when needed. The preset trimming parameters may include trimming amounts in the negative x, negative y, positive x, and positive y directions relative to the vanishing point pixel coordinates. For example, the preset trimming parameters may be represented as [s1, s2, s3, s4], and the trimming area may be represented as [xo-s1, yo-s2, xo+s3, yo+s4]. Alternatively, the preset trimming parameters may be marked to indicate direction, for example, s1 = -450, in which case the trimming area may be represented as [xo+s1, yo+s2, xo+s3, yo+s4]. The specific display format of the preset trimming parameters is not limited.

[0082] This embodiment uses a vanishing point in the image as a reference and crops it based on preset cropping parameters to obtain the input image for the corresponding sensing task model. Since the vanishing point represents the point where parallel lines visually intersect, the distance range that the cropping area can cover can be effectively controlled based on the vanishing point and cropping parameters. As a result, the cropped image can better satisfy the sensing distance range requirements of the sensing task model, improving the effectiveness of the input image for the sensing task model, and thereby improving the accuracy of the sensing results of the sensing task model and the recall rate for objects within the corresponding distance range.

[0083] Figure 6 is a flowchart of a method for determining a sensing result according to yet another exemplary embodiment of the present disclosure.

[0084] In some of the selectable embodiments, step 203, in the embodiment shown in Figure 2 above, determines the target sensing result based on the first sensing result corresponding to each distance range, Step 203a involves fusing the first sensing results corresponding to each distance range and obtaining the fused result, The procedure may also include step 203b, which determines the target sensing result based on the fusion result.

[0085] Here, the specific operating principles of steps 203a and 203b may be described by referring to step 2031 above, the difference being that in this embodiment, the narrow-angle sensing results of the preset object type may not be included.

[0086] In some selectable embodiments, Figure 7 is a flowchart for determining the sensing result according to one exemplary embodiment of the present disclosure. As shown in Figure 7, the wide-angle camera is a forward-view wide-angle camera 41, where *1 indicates that there is one wide-angle camera, and the narrow-angle camera is a forward-view narrow-angle camera 42, also of which there is one. The forward-view wide-angle camera 41 collects and acquires a first image. Subsampling is performed on the first image to acquire third images at multiple scales, the third images at multiple scales including 1 / 2 scale images and 1 / 4 scale images. The third images at multiple scales may further include the first image at the original scale. Sensing task models A, B, and C are sensing task models corresponding to distance ranges a, b, and c, respectively. Sensing task models A, B, and C may all be multi-task models, that is, they can simultaneously sense at least one type of sensing result for each of several different objects. Different objects include, for example, the rear of a vehicle, a vehicle, a pedestrian, a cyclist, a sign, a traffic light, and road markings. At least one type of sensing result may include, for example, a target detection result and a semantic segmentation result. Based on the trimming parameter 43, a corresponding region (i.e., a fourth image) is trimmed from a 1 / 4 scale image, and sensing processing is performed on the trimmed region by sensing task model A to obtain a wide-angle sensing result corresponding to distance range a. Based on the trimming parameter 44, a corresponding region (i.e., a fourth image) is trimmed from a 1 / 2 scale image, and sensing processing is performed on the trimmed region by sensing task model B to obtain a wide-angle sensing result corresponding to distance range b. Based on the trimming parameter 45, a corresponding region (i.e., a fourth image) is trimmed from the first image at the original scale, and sensing processing is performed on the trimmed region by sensing task model C to obtain a wide-angle sensing result corresponding to distance range c. Wide-angle sensing fusion is performed on the wide-angle sensing results for each distance range to obtain a wide-angle fusion result.Selectively, for sensing task model A, it is not necessary to crop the 1 / 4 scale image, and sensing processing may be performed directly on the 1 / 4 scale image using sensing task model A to obtain wide-angle sensing results corresponding to distance range a.

[0087] The forward-view narrow-angle camera 42 collects and acquires a second image, performs subsampling on the second image to acquire a fifth image at multiple scales, the fifth image at multiple scales includes a 1 / 2 scale image and a 1 / 4 scale image. Based on the trimming parameter 46, the corresponding region (i.e., the sixth image) is trimmed from the 1 / 2 scale image, and sensing processing is performed on the trimmed region by the sensing task model D to acquire a narrow-angle sensing result corresponding to the distance range d. Sensing task model D is a sensing task model corresponding to the distance range d. Sensing task model D may also be a multi-task model. Preset object types include traffic lights and lane markings. Traffic lights correspond to sensing task model E, and lane markings correspond to sensing task model F. That is, sensing task model E is a traffic light single-task sensing model, and sensing task model F is a lane marking single-task sensing model. Based on trimming parameter 47, a corresponding area is trimmed from a 1 / 4 scale image, and the sensing task model E performs traffic light detection processing on the trimmed area to obtain a narrow-angle traffic light detection result. Based on trimming parameter 48, a corresponding area is trimmed from a 1 / 4 scale image, and the sensing task model F performs lane marking detection processing on the trimmed area to obtain a narrow-angle lane marking detection result. The wide-angle fusion result, narrow-angle detection result, narrow-angle traffic light detection result, and narrow-angle lane marking detection result are fused to obtain the final target detection result. As shown in the figure, the target detection result includes the target detection result and semantic segmentation result for each object. The target detection result may include a two-dimensional detection frame for the object (e.g., a 2D box for each object in the figure). The semantic segmentation result may include the probability that each pixel in the image belongs to the object, or the set of pixel points in the image that belong to the object, or the classification label to which each pixel in the image belongs to each object. The specific content of the sensing result is not limited.The segmented LabelMap is a semantic segmentation result, represented as a pixel classification label map, where each pixel may correspond to a single classification label representing the type of object to which the pixel belongs; that is, the LabelMap contains semantic segmentation results corresponding to each sensed object. The forward view lane marking LabelMap represents the semantic segmentation results of lane markings.

[0088] Here, the trimming parameters corresponding to different sensing task models may be different. That is, parameters 43 to 48 may be different parameters, thereby allowing the trimming region to better meet the sensing requirements of the sensing task model, such as covering different distance ranges and satisfying the sensing characteristics of traffic lights, lane markings, etc., thereby improving the accuracy and precision of the sensing results of the sensing task model. Figure 7 is merely an exemplary embodiment of the present disclosure, and in actual application, it is not limited to the embodiment in Figure 7. For example, in Figure 7, only one sensing task model D for narrow-angle sensing (i.e., distance range d) is shown, and in actual application, multiple distance ranges may be set for narrow-angle sensing. Also, for example, the scale of the image on which each sensing task model is based is not limited to the scale shown in Figure 7.

[0089] In some optional embodiments, the narrow-angle sensing task model D may also be a further enhancement of the forward-view wide-angle sensing model and may include a 2D detection multitasking and semantic segmentation task for five types of objects: the entire vehicle, the rear of the vehicle, pedestrians, cyclists, and traffic cones, as well as a lane marking semantic segmentation task.

[0090] In some of the selectable embodiments, the distance ranges corresponding to the sensing task model for each distance range are shown in Table 1 below. [Table 1] Here, "actual physical size" refers to the actual size of the object. Based on the actual physical size of the object and the minimum pixel required for detection, the effective detection distance range for different objects can be estimated for each detection task model. Since the actual sizes of different objects may differ, and the minimum pixel required for detection of different objects may differ, the effective detection distance range for different objects of the same detection task model may differ. Table 1 is merely an illustrative description of the ranges, and in actual application, the distance range for different objects of a detection task model is not limited to the specific ranges in Table 1.

[0091] In some optional embodiments, sensing task models D, E, and F may utilize the same network structure to reduce model training costs. For example, a vargnet may be used as the backbone network, and a U-shaped network (unet) may be used as the neck network. Naturally, these are merely possible embodiments and do not limit the methods of the embodiments of this disclosure, and in actual application, each sensing task model may utilize a different network structure. The specific network structure is not limited to the vargnet and unet described above.

[0092] In some of the selectable embodiments, Figure 8 shows a schematic diagram of the input image trimming principle corresponding to the sensing task model D according to one exemplary embodiment of the present disclosure, above the embodiment shown in Figure 7. As shown in Figure 8, the fifth image (i.e., image pyramid) of multiple scales corresponding to the second image includes an image at the original scale (1 / 1 scale) (for example, H*W=2160*3840 in the figure, where H represents the image height and W represents the image width), an image at 1 / 2 scale (i.e., the height and width are each 1 / 2 of the original scale), an image at 1 / 4 scale (i.e., the height and width are each 1 / 4 of the original scale), and an image at 1 / 8 scale (i.e., the height and width are each 1 / 8 of the original scale). Sensing task model D is a narrow-angle multitask sensing model. The target scale corresponding to the distance range d is 1 / 2 scale, where xoy represents the pixel coordinate system of the image, and the pixel coordinates of the vanishing point P are represented as (xo,yo). Based on the vanishing point pixel coordinates (xo,yo) and the trimming parameters [s1,s2,s3,s4]=[-704,-284,540,228] corresponding to the distance range d, a 1 / 2 scale image (i.e., the second target image) is trimmed and represented as [xo-704,yo-284,xo+540,yo+228], obtaining a sixth image of 512*1344 pixels. Based on the sensing task model D, sensing processing is performed on the sixth image to obtain a narrow-angle sensing result corresponding to the distance range d. The narrow-angle multitask sensing task model D, as a further supplement to the wide-angle model, has a longer sensing distance and can effectively improve the recall rate of distant objects.

[0093] In some of the selectable embodiments, Figure 9 is a schematic diagram of the trimming principle of the input image corresponding to the sensing task model E according to one exemplary embodiment of the present disclosure, above the embodiment shown in Figure 7. As shown in Figure 9, the sensing task model E is a narrow-angle signal single-task sensing model, and in combination with the computational requirements and sensing distance requirements of the sensing task model, the first preset scale corresponding to the signal is set to a 1 / 2 scale and the corresponding preset trimming parameters are set to [s1,s2,s3,s4]=[704,540,540,164] depending on the spatial domain sensing range of the signal. The specific trimming principle is similar to that shown in Figure 8 and is therefore omitted from this explanation. By using narrow-angle signal single-task sensing, the sensing recall rate of long-distance signals can be greatly improved.

[0094] In some of the selectable embodiments, Figure 10 is a schematic diagram of the trimming principle of the input image corresponding to the sensing task model F according to one exemplary embodiment of the present disclosure, above the embodiment shown in Figure 7. As shown in Figure 10, sensing task model F is a lane marking single-task sensing model, and the specific trimming principle is similar to that of Figure 9 above, except that the first preset scale corresponding to the lane marking is a 1 / 4 scale, and the preset trimming parameters are [s1,s2,s3,s4]=[458,4,438,188], and a detailed explanation is omitted. By using the narrow-angle lane marking single-task sensing model (sensing task model F) as a supplement to the narrow-angle multi-task sensing model (sensing task model D), the shortcomings of the narrow-angle multi-task sensing model, which tend to lose close-range lane markings, are compensated for, and the completeness and accuracy of the lane marking sensing results are improved.

[0095] Figures 8 to 10 are merely illustrative examples, and in actual application, the trimming parameters may be set according to the actual situation and are not limited to the specific parameter values ​​shown in the figures.

[0096] Each of the embodiments described herein may be implemented individually or in any combination that does not conflict, and may be specifically configured according to the actual needs, but is not limited to the foregoing.

[0097] Any of the methods for determining the sensing result according to the embodiments of this disclosure may be performed on any suitable device having data processing capabilities, including but not limited to terminal devices and servers. Alternatively, any of the methods for determining the sensing result according to the embodiments of this disclosure may be performed by a processor, for example, by calling a corresponding instruction stored in memory, thereby performing any of the methods for determining the sensing result according to the embodiments of this disclosure. Further explanation is omitted below.

[0098] Exemplary device Figure 11 is a schematic diagram of the structure of an apparatus for determining sensing results according to one exemplary embodiment of the present disclosure. The apparatus of this embodiment can be used to implement an embodiment of the method corresponding to the present disclosure, and the apparatus shown in Figure 11 may include a first processing module 51, a second processing module 52, and a third processing module 53.

[0099] The first processing module 51 is used to determine a first image collected by a wide-angle camera and a second image collected by a narrow-angle camera, wherein the field of view of the narrow-angle camera is smaller than the field of view of the wide-angle camera.

[0100] The second processing module 52 is used to determine a first sensing result corresponding to each of the distance ranges, based on the first image, the second image, and a sensing task model corresponding to at least one distance range.

[0101] The third processing module 53 is used to determine the target detection result based on the first detection result corresponding to each of the distance ranges.

[0102] Figure 12 is a schematic diagram of the structure of an apparatus for determining sensing results according to another exemplary embodiment of the present disclosure.

[0103] In some optional embodiments, as shown in Figure 12 above, the second processing module 52 may include a first processing unit 521, a second processing unit 522, and a third processing unit 523, as shown in Figure 11.

[0104] The first processing unit 521 is used to determine wide-angle sensing results corresponding to each first distance range, based on the first image and a sensing task model of at least one first distance range corresponding to the first image.

[0105] The second processing unit 522 is used to determine narrow-angle sensing results corresponding to each second distance range, based on the second image and a sensing task model of at least one second distance range corresponding to the second image.

[0106] The third processing unit 523 is used to determine a first sensing result corresponding to each distance range, based on the wide-angle sensing result corresponding to each first distance range and the narrow-angle sensing result corresponding to each second distance range.

[0107] In some selectable embodiments, the first processing unit 521 is used specifically to determine a third image of a plurality of scales corresponding to the first image based on the first image, and to determine a wide-angle sensing result corresponding to each first distance range based on the third image of each scale and a sensing task model corresponding to each first distance range, respectively.

[0108] In some selectable embodiments, the first processing unit 521 specifically determines a target scale and trimming parameters corresponding to a target distance range, with any first distance range being the target distance range, the different trimming parameters corresponding to different distance ranges, determines a first target image from each third image based on the target scale, determines a fourth image based on the trimming parameters and the first target image, performs sensing processing on the fourth image based on a sensing task model corresponding to the target distance range, and is used to obtain wide-angle sensing results corresponding to the target distance range.

[0109] In some selectable embodiments, the second processing unit 522 is used specifically to determine a fifth image of a plurality of scales corresponding to the second image based on the second image, and to determine a narrow-angle sensing result corresponding to each second distance range based on the fifth image of each scale and the sensing task model corresponding to each second distance range, respectively.

[0110] In some selectable embodiments, the second processing unit 522 is used to obtain narrow-angle sensing results corresponding to the target distance range by using one of the second distance ranges as the target distance range, determining a target scale and trimming parameters corresponding to the target distance range, determining a second target image from each fifth image based on the target scale, determining a sixth image based on the trimming parameters and the second target image, performing sensing processing on the sixth image based on a sensing task model corresponding to the target distance range.

[0111] In some of the selectable embodiments, as shown in Figure 12, the apparatus of the embodiments of the present disclosure is further... A fourth processing module 54 for determining a target fifth image of a first preset scale from a fifth image of each scale, A fifth processing module 55 for trimming a fifth target image based on preset trimming parameters of a preset object type and obtaining a third target image, The system may also include a sixth processing module 56 for performing sensing processing on a third target image based on a sensing task model corresponding to a preset object type, and for obtaining narrow-angle sensing results corresponding to the preset object type.

[0112] The third processing module 53 may also include a fourth processing unit 531 for determining a target detection result based on a first detection result corresponding to each distance range and a narrow-angle detection result corresponding to a preset object type.

[0113] In some selectable embodiments, the fifth processing module 55 is specifically used to determine the vanishing point pixel coordinates in the target fifth image, to crop the target fifth image around the vanishing point pixel coordinates based on preset cropping parameters, and to obtain the third target image.

[0114] Figure 13 is a schematic diagram of the structure of an apparatus for determining sensing results according to yet another exemplary embodiment of the present disclosure.

[0115] In some of the selectable embodiments, in the embodiment shown in Figure 11, the third processing module 53 is: A fusion processing unit 53a for fusing the first sensing results corresponding to each distance range and obtaining the fusion result, The system may also include a decision unit 53b for determining the target sensing result based on the fusion result.

[0116] Beneficial technical effects corresponding to exemplary embodiments of this apparatus may be described by referring to the corresponding beneficial technical effects of the exemplary method portion described above, and are omitted here.

[0117] Exemplary electronic device Figure 14 is a structural diagram of an electronic device according to an embodiment of the present disclosure, wherein the electronic device 90 includes at least one processor 91 and memory 92.

[0118] The processor 91 may be a central processing unit (CPU) or another form of processing unit having data processing capability and / or instruction execution capability, and can control other components in the electronic device 90 to perform a desired function.

[0119] The memory 92 may include one or more computer program products, which include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored in the computer-readable storage media, and the processor 91 can implement the methods and / or other desired functions in each embodiment of the present disclosure described above by executing one or more computer program instructions.

[0120] In one example, the electronic device 90 may further include an input device 93 and an output device 94, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0121] The input device 93 may include a keyboard, mouse, or the like.

[0122] The output device 94 is capable of outputting various types of information to the outside and may include a display, speaker, printer, communication network, and remote output devices connected thereto.

[0123] Of course, for the sake of simplicity, Figure 14 shows only some of the components of the electronic device 90 relevant to this disclosure, omitting components such as buses and input / output interfaces. Furthermore, depending on the specific application, the electronic device 90 may further include any other suitable components.

[0124] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of the present disclosure may further provide computer program products that, when executed by a processor, cause the processor to perform steps in the various embodiments of the present disclosure described in the “Exemplary Methods” section above.

[0125] Computer program products can be created using any combination of one or more programming languages ​​to produce program code for performing the operations of the embodiments of this disclosure, and the programming languages ​​include object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as the C language or similar programming languages. The program code may run entirely on the user's computing device, partially on the user's device, as separate software packages, partly on the user's computing device and partly on a remote computing device, or entirely on a remote computing device or server.

[0126] Furthermore, embodiments of the present disclosure may also be computer-readable storage media that, when executed by a processor, stores computer program instructions causing the processor to perform steps in the various embodiments of the present disclosure described in the “Exemplary Methods” section above.

[0127] Computer-readable storage media may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, electrical, magnetic, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0128] While the basic principles of this disclosure have been explained above with reference to specific examples, the advantages, advantages, and effects mentioned herein are not limited to those mentioned above, but are merely illustrative, and these advantages, advantages, and effects are not necessarily present in every example of this disclosure. Furthermore, the specific details disclosed above are not limited to those mentioned above, but are merely illustrative and intended to facilitate understanding, and these details do not necessarily limit this disclosure to being realized by those specific details.

[0129] Those skilled in the art can make various modifications and alterations to this disclosure without departing from the spirit and scope of this disclosure. Thus, if such modifications and alterations of this disclosure fall within the scope of the claims of this disclosure and the equivalent art, this disclosure is intended to include such modifications and alterations.

Claims

1. A step of determining a first image collected by a wide-angle camera and a second image collected by a narrow-angle camera, wherein the field of view of the narrow-angle camera is smaller than the field of view of the wide-angle camera, A step of determining a first sensing result corresponding to each of the distance ranges based on the first image, the second image, and a sensing task model corresponding to at least one distance range, A method for determining a sensing result, performed by a device for determining sensing results, comprising the step of determining a target sensing result based on the first sensing result corresponding to each of the aforementioned distance ranges.

2. The step of determining a first sensing result corresponding to each of the distance ranges based on the first image, the second image, and a sensing task model corresponding to at least one distance range is: A step of determining a wide-angle sensing result corresponding to each of the first distance ranges based on the first image and the sensing task model of at least one first distance range corresponding to the first image, A step of determining a narrow-angle sensing result corresponding to each of the second distance ranges based on the sensing task model of the second image and at least one second distance range corresponding to the second image, A method for determining a sensing result according to claim 1, comprising the step of determining the first sensing result corresponding to each distance range based on the wide-angle sensing result corresponding to each of the first distance ranges and the narrow-angle sensing result corresponding to each of the second distance ranges.

3. The step of determining wide-angle sensing results corresponding to each of the first distance ranges based on the sensing task model of the first image and at least one first distance range corresponding to the first image is: A step of determining a third image of a plurality of scales corresponding to the first image based on the first image, A method for determining a sensing result according to claim 2, comprising the step of determining the wide-angle sensing result corresponding to each first distance range based on the sensing task model corresponding to each of the third images of each scale and each of the first distance ranges, respectively.

4. The step of determining the wide-angle sensing result corresponding to each first distance range, based on the sensing task model corresponding to each of the third images of each scale and each of the first distance ranges, is: A step of determining a target scale and trimming parameters corresponding to the target distance range, with any of the first distance ranges being the target distance range, wherein different trimming parameters correspond to different distance ranges, A step of determining a first target image from each of the third images based on the target scale, A step of determining a fourth image based on the trimming parameters and the first target image, A method for determining a sensing result according to claim 3, comprising the steps of performing sensing processing on the fourth image based on the sensing task model corresponding to the target distance range and obtaining the wide-angle sensing result corresponding to the target distance range.

5. The step of determining a narrow-angle sensing result corresponding to each of the second distance ranges based on the sensing task model of the second image and at least one second distance range corresponding to the second image is: A step of determining a fifth image of a plurality of scales corresponding to the second image based on the second image, A method for determining a sensing result according to claim 2, comprising the step of determining the narrow-angle sensing result corresponding to each second distance range based on the fifth image of each scale and the sensing task model corresponding to each second distance range, respectively.

6. The step of determining the narrow-angle sensing result corresponding to each second distance range based on the fifth image of each scale and the sensing task model corresponding to each second distance range is: A step of determining a target scale and trimming parameters corresponding to the target distance range, with any of the second distance ranges being the target distance range, wherein different trimming parameters correspond to different distance ranges, A step of determining a second target image from each of the fifth images based on the aforementioned target scale, A step of determining a sixth image based on the trimming parameters and the second target image, A method for determining a sensing result according to claim 5, comprising the steps of performing sensing processing on the sixth image based on the sensing task model corresponding to the target distance range and obtaining the narrow-angle sensing result corresponding to the target distance range.

7. A step of determining a target fifth image of a first preset scale from the fifth image of each scale, The steps include: trimming the target fifth image based on the preset trimming parameters of the preset object type to obtain the third target image; The process further includes the steps of performing a sensing process on the third target image based on a sensing task model corresponding to the preset object type, and obtaining a narrow-angle sensing result corresponding to the preset object type, The step of determining the target detection result based on the first detection result corresponding to each of the aforementioned distance ranges is as follows: A method for determining a sensing result according to claim 5, comprising the step of determining the target sensing result based on the first sensing result corresponding to each of the distance ranges and the narrow-angle sensing result corresponding to the preset target type.

8. The step of trimming the target fifth image based on the preset trimming parameters of the preset object type and obtaining the third target image is: The steps include determining the coordinates of the vanishing point pixels in the fifth target image, A method for determining a sensing result according to claim 7, comprising the steps of: trimming the target fifth image based on the preset trimming parameters with respect to the vanishing point pixel coordinates to obtain the third target image.

9. The step of determining the target detection result based on the first detection result corresponding to each of the aforementioned distance ranges is as follows: A step of fusing the first sensing results corresponding to each of the aforementioned distance ranges and obtaining the fusing result, A method for determining a sensing result according to any one of claims 1 to 8, comprising the step of determining the target sensing result based on the fusion result.

10. A first processing module for determining a first image collected by a wide-angle camera and a second image collected by a narrow-angle camera, wherein the field of view of the narrow-angle camera is smaller than the field of view of the wide-angle camera. A second processing module for determining a first sensing result corresponding to each of the distance ranges, based on the first image, the second image, and a sensing task model corresponding to at least one distance range, A device for determining a sensing result, comprising: a third processing module for determining a target sensing result based on the first sensing result corresponding to each of the aforementioned distance ranges.

11. A computer-readable storage medium storing a computer program for performing a method for determining a sensing result according to any one of claims 1 to 8.

12. Processor and The processor includes a memory for storing executable instructions, The processor is an electronic device used to realize a method for determining a sensing result according to any one of claims 1 to 8, by reading and executing the executable instructions from the memory.

Images