Object recognition method, storage medium, program product, electronic device and vehicle

Abstract

The invention relates to an object recognition method, a storage medium, a program product, electronic equipment and a vehicle. The object identification method comprises the following steps: eliminating an object in a target area in a first image to obtain a second image; and identifying a target object in the target area according to the first image and the second image. According to the embodiment of the invention, the method does not depend on a laser point cloud radar or a plurality of cameras, can achieve the recognition of the obstacle even through the image collected by one monocular camera, does not need the hardware equipment for image collection, and can adapt to different hardware configurations.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to an object recognition method, a storage medium, a program product, an electronic device, and a vehicle. Background Art

[0002] With the development of vehicle technology, intelligent driving is gaining increasing attention. Obstacle recognition is a key component of intelligent driving, and its accuracy is crucial to driving safety. Related technologies typically use laser point cloud radar or multiple cameras to capture images of the road the vehicle is traveling on, ensuring the accuracy of the road data and thus improving obstacle recognition. However, not all vehicles are equipped with laser point cloud radar or multiple cameras. Ensuring obstacle recognition across different hardware configurations is a pressing technical challenge. Summary of the Invention

[0003] The embodiments of the present application provide an object recognition method, storage medium, program product, electronic device and vehicle, which do not require hardware equipment for image acquisition and can realize obstacle recognition under different hardware configurations, so as to at least partially solve the above-mentioned technical problems.

[0004] In order to achieve the above object, according to a first aspect of the present application, a method for object recognition is provided, the method comprising:

[0005] Eliminating the object within the target area in the first image to obtain a second image;

[0006] A target object within the target area is identified according to the first image and the second image.

[0007] Optionally, eliminating objects within the target area in the first image to obtain the second image includes: for each sub-image of the first image, eliminating objects within the target area in the sub-image to obtain an intermediate image; and acquiring the second image based on the multiple intermediate images.

[0008] Optionally, for each sub-image of the first image, eliminating objects within a target area in the sub-image to obtain an intermediate image includes: for each sub-image of the first image, erasing image content within the target area in the sub-image to obtain an image to be repaired; and repairing the image to be repaired to obtain an intermediate image.

[0009] Optionally, obtaining the second image based on the multiple intermediate images includes: for each first pixel in the target area in the first image, obtaining a target image containing the first pixel from the multiple intermediate images; and determining the pixel value of the first pixel in the second image based on the pixel value of the first pixel in the target image.

[0010] Optionally, determining the pixel value of the first pixel in the second image based on the pixel value of the first pixel in the target image includes: determining the pixel value of the first pixel in the second image based on the pixel value of the first pixel in the target image and the weight value of the target image.

[0011] Optionally, the method further includes: determining a weight value of the target image according to a pixel value of the first pixel in the target image, a pixel value of a reference pixel in the target image, and size data of the target image.

[0012] Optionally, the reference pixel is a central pixel of the target image.

[0013] Optionally, the method further includes: traversing the first image based on a target sliding window; and intercepting a sub-image from the first image according to the target sliding window at each sliding window position of the target sliding window.

[0014] Optionally, traversing the first image based on the target sliding window includes: sliding the target sliding window sequentially in the first image according to a target step length; wherein the target step length is less than or equal to size data of the target sliding window.

[0015] Optionally, at each sliding window position of the target sliding window, intercepting a sub-image from the first image according to the target sliding window includes: at each sliding window position of the target sliding window, intercepting a sub-image concentric with the target sliding window from the first image; wherein size data of the sub-image is smaller than or equal to size data of the target sliding window.

[0016] Optionally, identifying the target object within the target area based on the first image and the second image includes: inputting the first image and the second image into a recognition model, so that the recognition model outputs recognition data of the target object within the target area.

[0017] Optionally, the identification data includes mask data.

[0018] Optionally, the recognition model includes a first extraction network, a second extraction network and a fusion network; inputting the first image and the second image into the recognition model so that the recognition model outputs recognition data of the target object within the target area includes: inputting the first image into the first extraction network so that the first extraction network extracts a first feature from the first image; inputting the second image into the second extraction network so that the second extraction network extracts a second feature from the second image; and inputting the first feature and the second feature into the fusion network so that the fusion network outputs recognition data of the target object within the target area.

[0019] Optionally, the first extraction network includes first extraction modules corresponding to multiple image sizes, the second extraction network includes second extraction modules corresponding to the multiple image sizes, and the fusion network includes a stitching module and an upsampling module corresponding to the multiple image sizes; the inputting the first image and the second image into the recognition model so that the recognition model outputs recognition data of the target object in the target area includes: inputting the first image into the first extraction network so that the first extraction network outputs first features corresponding to the multiple image sizes; wherein each of the first extraction modules in the first extraction network extracts the first features corresponding to one image size from the first image; inputting the second image into the second extraction network so that the second extraction network outputs second features corresponding to the multiple image sizes; wherein each of the second extraction modules in the second extraction network extracts the second features corresponding to one image size from the second image; inputting the first features and the second features corresponding to the multiple image sizes into the fusion network so that the fusion network outputs recognition data of the target object in the target area; wherein each of the stitching modules in the fusion network stitches the first features and the second features corresponding to one image size; and each of the upsampling modules in the fusion network upsamples the output of one of the stitching modules.

[0020] Optionally, the method further includes: constructing a training data set; wherein each training sample in the training data set includes a first historical image, a second historical image and a historical recognition data; and training a recognition model based on the training data set.

[0021] Optionally, constructing a training data set includes: obtaining a first historical image and the historical identification data corresponding to the first historical image; eliminating objects within a target area in the first historical image to obtain a second historical image; and obtaining a training data set based on multiple first historical images, and the historical identification data and second historical images corresponding to multiple first historical images respectively.

[0022] According to a second aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the object recognition method described above is implemented.

[0023] According to a third aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program implements the above-mentioned object recognition method when executed by a processor.

[0024] According to a fourth aspect of the present application, an electronic device is provided, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the above-mentioned object recognition method.

[0025] According to a fifth aspect of the present application, a vehicle is provided, comprising the above-mentioned electronic device.

[0026] The embodiment of the present application first eliminates the objects in the target area in the image, and then identifies the target objects in the target area based on the images before and after elimination. By eliminating the images before and after, target objects such as obstacles in the target area can be identified more accurately and quickly, thereby improving the accuracy and efficiency of obstacle identification. Moreover, the embodiment of the present application does not rely on laser point cloud radar or multiple cameras, etc., and even images captured by a monocular camera can realize obstacle identification. The embodiment of the present application does not require the hardware equipment for image acquisition and can be adapted to different hardware configurations. In addition, the embodiment of the present application first eliminates the objects in the target area, and then performs object identification based on the images before and after elimination, thereby realizing any type of object recognition. When the embodiment of the present application is applied to the field of vehicle technology, it can cover a large number of obstacle types in actual traffic scenes, thereby improving the versatility and compatibility of object recognition.

[0027] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0029] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0030] Figure 1 This is a flowchart of an object recognition method provided by an embodiment of the present application;

[0031] Figure 2 is a schematic diagram of a target area provided in an embodiment of the present application;

[0032] Figure 3 is a schematic diagram of an obstacle provided in an embodiment of the present application;

[0033] Figure 4 is a schematic diagram of an image restoration process provided by an embodiment of the present application;

[0034] Figure 5 is a schematic diagram of an object elimination process provided by an embodiment of the present application;

[0035] Figure 6 is a schematic diagram of a recognition model provided in an embodiment of the present application;

[0036] Figure 7 is a flowchart of another object recognition method provided by an embodiment of the present application;

[0037] Figure 8 It is a schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0039] According to a first aspect of the present application, an embodiment of the present application provides an object recognition method.

[0040] See also Figure 1 , Figure 1 This is a flow chart of an object recognition method provided by an embodiment of the present application. Figure 1 As shown, the object recognition method may include the following steps:

[0041] Step S100: Eliminate objects within the target area of the first image to obtain a second image;

[0042] Step S200: identifying a target object within a target area according to the first image and the second image.

[0043] The first image can be any type of image, and the embodiments of the present application do not limit this. For example, the first image can be a driving road image, an indoor image, a sports field image, a security monitoring image, etc. Taking the first image as an example of a driving road image, the first image can be a road image collected during the vehicle's driving process. For example, the first image can be collected by a camera installed in the vehicle (such as a monocular camera). The embodiments of the present application do not limit the acquisition device of the first image, and the first image can also be collected by a radar or sensor. The first image can refer to an image of the road in front of the vehicle, and of course it can also refer to an image of the road to the side or rear of the vehicle. The embodiments of the present application do not limit this.

[0044] In an embodiment of the present application, the first image can be input into a target area detection model so that the target area detection model can identify the target area of the first image. For example, the target area detection model outputs mask data of the target area of the first image, and the mask data indicates the pixel position of the target area. The target area can be any type of area. For example, when the first image is a driving road image, the target area can be a drivable area; when the first image is an indoor image, the target area can be a desk area; when the first image is a sports field image, the target area can be a lawn area, etc. For example, if the first image is a driving road image and the target area is a drivable area, Figure 2 As shown, Figure 2 The green pixel position in the image is the mask data of the target area output by the target area detection model, which indicates the pixel position of the target area. The target area detection model can be a traditional model such as a texture feature extraction model, an edge detection model, or a support vector machine, or a deep learning model such as BiFPN (Bidirectional Feature Pyramid Network) or YOLO (You Only Look Once). The specific structure of the target area detection model is not limited in this embodiment of the application.

[0045] In the embodiment of the present application, objects within the target area of the first image are eliminated to obtain a second image. For example, taking the first image as a driving road image and the target area as a driving area, the objects within the target area include but are not limited to obstacles such as cones, water barriers, earth piles, and fallen rocks. Figure 3 As shown; other vehicles; pedestrians; animals; plants, etc. The embodiment of the present application can eliminate all objects in the target area. After eliminating the objects in the target area in the first image, the location of the target area in the second image is all the road surface.

[0046] Based on the first image and its corresponding second image, a target object within the target area of the first image can be identified. For example, the first image and its corresponding second image can be compared to identify the target object within the target area. The embodiments of the present application do not limit the type of target object; the target object can be any object or a specific type of object, such as an obstacle.

[0047] For other descriptions of object elimination and target object recognition, please refer to the following embodiments, which will not be elaborated here.

[0048] In summary, the object recognition method provided by the embodiment of the present application first eliminates the objects in the target area in the image, and then identifies the target objects in the target area based on the images before and after elimination. By eliminating the images before and after, target objects such as obstacles in the target area can be identified more accurately and quickly, thereby improving the accuracy and efficiency of obstacle recognition. In addition, the embodiment of the present application does not rely on laser point cloud radar or multiple cameras, etc., and even images captured by a monocular camera can realize obstacle recognition. The embodiment of the present application does not require the hardware equipment for image acquisition and can be adapted to different hardware configurations, such as the hardware equipment of the vehicle, and can be adapted to different vehicle configurations. In addition, the embodiment of the present application first eliminates the objects in the target area, and then performs object recognition based on the images before and after elimination, thereby realizing any type of object recognition. When the embodiment of the present application is applied to the field of vehicle technology, it can cover a large number of obstacle types in actual traffic scenes, thereby improving the versatility and compatibility of object recognition.

[0049] In some embodiments, the above step S100 may include the following steps:

[0050] Step S110: for each sub-image of the first image, eliminate the object in the target area of the sub-image to obtain an intermediate image;

[0051] Step S120: Acquire a second image based on the multiple intermediate images.

[0052] When performing the elimination operation, the embodiment of the present application crops the first image into multiple sub-images, and then performs the elimination operation on each sub-image to eliminate the objects within the target area of each sub-image. The dimensional data of the multiple sub-images can be the same, but the embodiment of the present application does not exclude the situation where the dimensional data of the multiple sub-images are different. The dimensional data of the sub-images includes but is not limited to the length, width, area, and other data of the sub-images. In addition, there may be partial overlap between two adjacent sub-images, or there may be no overlapping area. Generally, it is possible to set the partial overlap between two adjacent sub-images to cover the entire area of the first image.

[0053] In some embodiments, before step S110, the object recognition method may further include the following steps:

[0054] Step S010: traversing the first image based on the target sliding window;

[0055] Step S020: at each sliding window position of the target sliding window, a sub-image is captured from the first image according to the target sliding window.

[0056] When traversing the first image, the target window can slide in a certain order within the first image to traverse all pixel positions in the first image. Thus, in some embodiments, step S010 may include sliding the target window sequentially within the first image according to a target step size. For example, the target window may be slid starting from the upper left corner of the first image and moving from left to right or from top to bottom. Of course, in practical applications, the target window may also be slid in other ways, such as from right to left or from bottom to top. The target step size during the sliding of the target window is less than or equal to the size data of the target window. For example, if the target window is square, the target step size may be less than or equal to the side length of the target window; if the target window is rectangular, the target step size may be less than or equal to the length or width of the target window.

[0057] In the embodiment of the present application, a sub-image is captured at each sliding window position. In some embodiments, step S020 may include: capturing, at each sliding window position of the target sliding window, a sub-image concentric with the target sliding window from the first image. The sub-image's dimensions are less than or equal to the target sliding window's dimensions. For example, if the sub-image and the target sliding window are both square, the side length of the sub-image is less than or equal to the side length of the target sliding window; if the sub-image and the target sliding window are both square, the length of the sub-image is less than or equal to the length of the target sliding window, and / or the width of the sub-image is less than or equal to the width of the target sliding window.

[0058] Because not all locations in the first image may be the target area, the sub-image does not necessarily include the target area, that is, the sub-image does not necessarily overlap with the target area. For sub-images that do not overlap with the target area, the embodiment of the present application does not need to perform a removal operation; for sub-images that overlap with the target area, the embodiment of the present application performs a removal operation to remove objects within the target area in the sub-image.

[0059] In some embodiments, the above step S110 may include: for each sub-image of the first image, erasing the image content in the target area of the sub-image to obtain the image to be repaired; repairing the image to be repaired to obtain an intermediate image. Among them, for each sub-image, it can be first determined whether the sub-image overlaps with the target area. If it overlaps, continue to erase the image content in the target area of the sub-image, otherwise there is no need to perform the elimination operation. In the embodiment of the present application, erasing the image content in the target area of the sub-image may refer to setting the pixel values corresponding to the image content in the target area to preset pixel values, such as setting them to zero. After erasing, the image to be repaired corresponding to the sub-image can be obtained, and then the image to be repaired can be repaired to obtain the intermediate image corresponding to the sub-image. In some embodiments, an image repair algorithm can be used to repair the image to be repaired. The image repair algorithm includes but is not limited to PatchMatch, FMM (Fast Marching Method), Criminisi, LaMa (LArge MAskin painting), PowerPaint, BrushNet, etc. For example, Figure 4 As shown, Figure 4 (1) is the original input image, Figure 4 (2) is the image mask that needs to be erased. Figure 4 (3) is the restoration effect after using the image restoration algorithm.

[0060] If the sub-images do not overlap, the intermediate images corresponding to the multiple sub-images can be spliced together to obtain the second image. However, in the embodiments of the present application, overlapping areas may exist between the sub-images, so that part of the target area may be included in multiple sub-images at the same time. The intermediate images corresponding to the multiple sub-images may have eliminated this part. Therefore, the intermediate images corresponding to the multiple sub-images need to be fused to obtain the second image.

[0061] In some embodiments, the above step S120 may include the following steps:

[0062] Step S121: for each first pixel in the target area of the first image, obtaining a target image containing the first pixel from a plurality of intermediate images;

[0063] Step S122: Determine the pixel value of the first pixel in the second image according to the pixel value of the first pixel in the target image.

[0064] For each first pixel in the target area, an intermediate image containing the first pixel, that is, the target image, can be first obtained; and then the pixel value of the first pixel in the second image can be determined based on the pixel value of the first pixel in the target image. The target image containing the first pixel can be one image or multiple images. In the embodiment of the present application, the pixel value of the first pixel in at least one target image can be averaged, weighted summed, maximized, minimized, etc. to obtain the pixel value of the first pixel in the second image. In some embodiments, the above-mentioned step S122 may include: determining the pixel value of the first pixel in the second image based on the pixel value of the first pixel in the target image and the weight value of the target image. The embodiment of the present application does not limit the method for determining the weight value of the target image. For example, the weight value of the target image can be determined in combination with the area size of the target area in the target image. The larger the area of the target area included, the larger the weight value of the target image. In some embodiments, to determine the weight value of the target image, the object recognition method may further include: determining the weight value of the target image based on the pixel value of the first pixel in the target image, the pixel value of a reference pixel in the target image, and the size data of the target image. The reference pixel may be any pixel in the target image, such as the upper left pixel, upper right pixel, lower left pixel, lower right pixel, or center pixel in the target image.

[0065] Exemplarily, for pixel P, the weight value w of one of the target images including pixel P can be calculated using Formula 1.

[0066] Formula 1:

[0067] Where [u, v] refers to pixel P, Π(u, v) is at least one target image containing pixel P, [u j ,v j ] is the center pixel of one of the target images containing pixel P, i refers to the index value of the target image, and s refers to the length or width of the target image.

[0068] For example, Figure 5 As shown, at the sliding window position 510 of the target sliding window, a sub-image 520 is captured from the first image 500 according to the target sliding window; then, the image content in the sub-image 520 is erased to obtain the image to be repaired, and then image repair is performed on the image to be repaired to obtain an intermediate image 530; finally, weighted fusion is performed on the intermediate image 530 to obtain a second image 540.

[0069] In summary, the object recognition method provided by the embodiment of the present application crops an image into multiple sub-images and performs a removal operation on each sub-image separately, which can improve the accuracy of object removal and can remove smaller objects. Furthermore, the embodiment of the present application traverses the image to avoid missing objects in the target area. Furthermore, the embodiment of the present application performs weighted fusion on the intermediate image after image restoration to improve the accuracy of removal of the entire image.

[0070] In some embodiments, the above step S200 may include: inputting the first image and the second image into a recognition model, so that the recognition model outputs recognition data of the target object in the target area.

[0071] The recognition model can compare the first image and the second image to identify a target object within a target area in the first image and output recognition data for the target object. In some embodiments, the recognition data includes mask data that can indicate information such as the size, position, and dimensions of the target object. The present embodiments do not limit the specific structure of the recognition model; in actual applications, it can be flexibly configured based on actual needs.

[0072] Taking the recognition model as a comparison network based on a dual-stream architecture as an example, in some embodiments, the recognition model includes a first extraction network, a second extraction network, and a fusion network; the above step S200 may include the following steps:

[0073] Step S210: inputting the first image into the first extraction network, so that the first extraction network extracts a first feature from the first image;

[0074] Step S220: inputting the second image into the second extraction network, so that the second extraction network extracts second features from the second image;

[0075] Step S230: inputting the first feature and the second feature into the fusion network, so that the fusion network outputs recognition data of the target object in the target area.

[0076] The first extraction network and the second extraction network may be the same extraction network to respectively extract the first feature in the first image and the second feature in the second image.

[0077] In some embodiments, the first extraction network includes a first extraction module corresponding to multiple image sizes, the second extraction network includes a second extraction module corresponding to multiple image sizes, and the fusion network includes a stitching module corresponding to multiple image sizes and an upsampling module. The extraction modules in the first and second extraction networks can be implemented as convolutional neural networks, for example, and the upsampling module in the fusion network can form an image pyramid structure.

[0078] Step S210 may include inputting the first image into a first extraction network, so that the first extraction network outputs first features corresponding to multiple image sizes. Each first extraction module in the first extraction network is configured to extract first features corresponding to one image size from the first image.

[0079] Step S220 may include inputting the second image into a second extraction network, so that the second extraction network outputs second features corresponding to multiple image sizes. Each second extraction module in the second extraction network is configured to extract a second feature corresponding to one image size from the second image.

[0080] Step S230 may include inputting the first features and the second features corresponding to the multiple image sizes into a fusion network, so that the fusion network outputs recognition data of the target object within the target area. Each stitching module in the fusion network is configured to stitch the first features and the second features corresponding to a specific image size, and each upsampling module in the fusion network is configured to upsample the output of a stitching module.

[0081] For example, Figure 6 It is a schematic diagram of a recognition model provided by an embodiment of the present application. The first image 500 is input into the first extraction network 610 in the recognition model 600, and the multiple first extraction modules 611 corresponding to the multiple image sizes in the first extraction network 610 respectively output the first features corresponding to one image size. The second image 540 is input into the second extraction network 620 in the recognition model 600, and the multiple second extraction modules 621 corresponding to the multiple image sizes in the second extraction network 620 respectively output the second features corresponding to one image size. Then, the first features and the second features corresponding to the multiple image sizes are respectively input into the splicing modules 631 corresponding to the multiple image sizes in the fusion network 630 in the recognition model 600, and each splicing module 631 splices the first features and the second features corresponding to one image size. The spliced first feature and second feature are then input into the upsampling module 632 in the fusion network 630. Each upsampling module 632 upsamples the output of a splicing module 631 and the output of the previous upsampling module 632. The output of the last upsampling module 632 is the recognition data 550 of the target object.

[0082] In some embodiments, the object recognition method may further include the following steps:

[0083] Step S001: construct a training data set;

[0084] Step S002: training a recognition model based on a training data set.

[0085] The recognition model trained in step S002 can be used to identify the target object in step S200.

[0086] Among them, each training sample in the training data set includes a first historical image, a second historical image and a historical recognition data. The embodiment of the present application does not limit the generation method of the first historical image. The first historical image can be obtained by camera capture, or the first historical image can be obtained by further processing based on the image captured by the camera. For example, for each image captured by the camera, objects such as obstacles can be arbitrarily added to the target area of the image to synthesize the first historical image. The historical recognition data corresponding to the first historical image refers to the recognition data of the target object in the target area of the first historical image, such as the mask data of the target object. The second historical image corresponding to the first historical image refers to the image obtained after eliminating the objects in the target area of the first historical image.

[0087] Based on this, in some embodiments, step S001 may include: obtaining a first historical image and historical recognition data corresponding to the first historical image; eliminating objects within a target area in the first historical image to obtain a second historical image; and obtaining a training dataset based on multiple first historical images and the historical recognition data and second historical images corresponding to each of the multiple first historical images. For details on eliminating objects within the target area in the first historical image to obtain the second historical image, please refer to the description of step S100 above. The implementation principles of these steps are the same and are not further elaborated here.

[0088] Training a recognition model based on a training data set may include: inputting a first historical image and its corresponding second historical image in the training data set into the recognition model so that the recognition model outputs predicted recognition data of a target object within a target area in the first historical image; determining a loss value of the recognition model based on the predicted recognition data and historical recognition data corresponding to the first historical image; and adjusting parameters of the recognition model so that the loss value converges.

[0089] In summary, the object recognition method provided in the embodiments of the present application improves object recognition efficiency by using a recognition model to compare images before and after elimination to output recognition data for the target object in the target area. Furthermore, the embodiments of the present application construct a recognition model based on a comparison network using a dual-stream architecture. The model structure and training method are simple, making it easier to adapt to different hardware configurations, such as different vehicle configurations.

[0090] Below, an example is used to introduce and illustrate the object recognition method provided in an embodiment of the present application.

[0091] See also Figure 7 , Figure 7This is a flow chart of another object recognition method provided by an embodiment of the present application. Figure 7 As shown, the object recognition method may include the following steps S701 to S708.

[0092] Step S701: Acquire a first image.

[0093] Step S702: inputting the first image into the target region detection algorithm, so that the target region detection algorithm outputs the target region of the first image.

[0094] Step S703: traverse the first image from left to right and from top to bottom based on the target sliding window.

[0095] Step S704: at each sliding window position of the target sliding window, a sub-image having the same center as the target sliding window is intercepted from the first image.

[0096] Step S705: For each sub-image, erase the image content within the target area of the sub-image to obtain the image to be repaired corresponding to the sub-image.

[0097] Step S706: Repair the image to be repaired corresponding to the sub-image to obtain an intermediate image corresponding to the sub-image.

[0098] Step S707: performing weighted fusion on the intermediate images corresponding to the multiple sub-images to obtain a second image.

[0099] Step S708: Input the first image and the second image into the recognition model, so that the recognition model outputs recognition data of the target object in the target area.

[0100] For the description of each step in steps S701 to S708, please refer to the above embodiment. Their implementation principles are the same and will not be described in detail here.

[0101] According to a second aspect of the present application, embodiments of the present application further provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the object recognition method described above. This non-transitory computer-readable storage medium has all the beneficial effects of the object recognition method described above, and this application will not further elaborate on them.

[0102] According to the third aspect of the present application, an embodiment of the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above-mentioned object recognition method and has all the beneficial effects of the above-mentioned object recognition method. This application will not go into details here.

[0103] According to a fourth aspect of the present application, an embodiment of the present application further provides an electronic device comprising: a memory and a processor, wherein the memory stores a computer program; the processor is configured to execute the computer program in the memory to implement the steps of the object recognition method described above. This electronic device has all the beneficial effects of the object recognition method described above, and this application will not further elaborate on them.

[0104] The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof, and this application does not specifically limit this. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0105] In some embodiments of the present application, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0106] The computer-readable storage medium may be included in the electronic device or may exist independently without being incorporated into the electronic device. The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0107] Eliminating the object within the target area in the first image to obtain a second image;

[0108] A target object within the target area is identified according to the first image and the second image.

[0109] Computer program code for performing the operations of some embodiments of the present application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (for example, using an Internet service provider to connect via the Internet).

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function.

[0111] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures.

[0112] For example, two blocks shown in succession may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow charts, and combinations of blocks in the block diagrams and / or flow charts, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or may be implemented using a combination of dedicated hardware and computer instructions.

[0113] The units described in some embodiments of the present application may be implemented in software or hardware, and may also be provided in a processor.

[0114] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and the like.

[0115] According to the fifth aspect of this application, Figure 8 As shown, the embodiment of the present application further provides a vehicle 10, which includes the above-mentioned electronic device. The vehicle has all the beneficial effects of the above-mentioned electronic device, etc., which will not be described in detail in this application.

[0116] The vehicle may be a fuel vehicle, a plug-in hybrid vehicle or a new energy vehicle, etc., and this application does not make any specific restrictions on this.

[0117] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0118] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0119] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0120] The above are merely preferred embodiments of the present application and do not constitute any form of limitation to the present application. Although the descriptions of each embodiment in the embodiments of the present application have different focuses, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A method for object recognition, characterized in that: The method comprises: Eliminating the object within the target area in the first image to obtain a second image; A target object within the target area is identified according to the first image and the second image.

2. The method according to claim 1, characterized in that Eliminating the object within the target area in the first image to obtain the second image includes: For each sub-image of the first image, eliminating objects within a target area of the sub-image to obtain an intermediate image; A second image is acquired based on the plurality of intermediate images.

3. The method according to claim 2, characterized in that Eliminating the object within the target area of each sub-image of the first image to obtain an intermediate image includes: For each sub-image of the first image, erasing the image content within the target area of the sub-image to obtain an image to be repaired; Repair the image to be repaired to obtain an intermediate image.

4. The method according to claim 2, characterized in that The step of acquiring a second image based on the plurality of intermediate images comprises: For each first pixel in the target area in the first image, acquiring a target image containing the first pixel from the plurality of intermediate images; The pixel value of the first pixel in the second image is determined according to the pixel value of the first pixel in the target image.

5. The method according to claim 4, characterized in that Determining the pixel value of the first pixel in the second image according to the pixel value of the first pixel in the target image includes: The pixel value of the first pixel in the second image is determined according to the pixel value of the first pixel in the target image and the weight value of the target image.

6. The method according to claim 5, characterized in that The method further comprises: A weight value of the target image is determined according to a pixel value of the first pixel in the target image, a pixel value of a reference pixel in the target image, and size data of the target image.

7. The method according to claim 6, characterized in that The reference pixel is the center pixel of the target image.

8. The method according to claim 2, characterized in that The method further comprises: traversing the first image based on the target sliding window; At each sliding window position of the target sliding window, a sub-image is captured from the first image according to the target sliding window.

9. The method according to claim 8, characterized in that The traversing the first image based on the target sliding window includes: Slide the target sliding window sequentially in the first image according to the target step size; The target step size is smaller than or equal to the size data of the target sliding window.

10. The method according to claim 8, characterized in that The step of intercepting a sub-image from the first image according to the target sliding window at each sliding window position of the target sliding window includes: At each sliding window position of the target sliding window, intercepting a sub-image concentric with the target sliding window from the first image; The size data of the sub-image is smaller than or equal to the size data of the target sliding window.

11. The method according to claim 1, wherein The identifying the target object within the target area according to the first image and the second image includes: The first image and the second image are input into a recognition model so that the recognition model outputs recognition data of the target object in the target area.

12. The method according to claim 11, characterized in that The identification data includes mask data.

13. The method according to claim 11, characterized in that The recognition model includes a first extraction network, a second extraction network and a fusion network; Inputting the first image and the second image into a recognition model so that the recognition model outputs recognition data of the target object in the target area includes: inputting the first image into the first extraction network so that the first extraction network extracts a first feature from the first image; inputting the second image into the second extraction network so that the second extraction network extracts second features from the second image; The first feature and the second feature are input into the fusion network, so that the fusion network outputs recognition data of the target object in the target area.

14. The method according to claim 13, characterized in that The first extraction network includes a first extraction module corresponding to a plurality of image sizes, the second extraction network includes a second extraction module corresponding to the plurality of image sizes, and the fusion network includes a stitching module and an upsampling module corresponding to the plurality of image sizes; Inputting the first image and the second image into a recognition model so that the recognition model outputs recognition data of the target object in the target area includes: Inputting the first image into the first extraction network so that the first extraction network outputs first features corresponding to the multiple image sizes; wherein each of the first extraction modules in the first extraction network extracts the first features corresponding to one image size from the first image; Inputting the second image into the second extraction network so that the second extraction network outputs second features corresponding to the multiple image sizes; wherein each second extraction module in the second extraction network extracts the second feature corresponding to one image size from the second image; The first features and the second features corresponding to the multiple image sizes are respectively input into the fusion network, so that the fusion network outputs the recognition data of the target object in the target area; wherein each of the stitching modules in the fusion network stitches the first features and the second features corresponding to an image size; and each of the upsampling modules in the fusion network upsamples the output of a stitching module.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the object recognition method according to any one of claims 1 to 14 is implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the object recognition method according to any one of claims 1 to 14 is implemented.

17. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the object recognition method according to any one of claims 1 to 14.

18. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 17.

Images