Vehicle blind area target detection method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle blind area target detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a real-time image of any blind area around a vehicle, and enabling the real-time image to comprise a detection target; determining a region-of-interest image according to the real-time image; according to the attention area image and the real-time image, an image to be detected is determined, and the image to be detected comprises a first zoom image corresponding to the real-time image and a second zoom image corresponding to the attention area image; inputting a to-be-detected image to the target detection model, and generating a first detection frame corresponding to the detection target in the first zoom graph and a second detection frame corresponding to the detection target in the second zoom graph; and determining a blind area target detection result according to the first detection frame and the second detection frame. The accuracy of a vehicle blind area target detection result can be improved.

Description

Technical Field

[0001] The present application relates to the field of automobile safety technology, and in particular to a method, device, electronic device and storage medium for detecting targets in vehicle blind spots. Background Art

[0002] At present, many vehicles are equipped with blind spot detection systems, which mainly rely on cameras installed around the vehicle to collect real-time images of the blind spot range, analyze the real-time images, obtain detection targets (for example, pedestrians or vehicles), output blind spot detection results, and improve vehicle driving safety based on the blind spot detection results. However, the blind spot detection system may make misjudgments when detecting targets, which is not conducive to ensuring the safety of vehicle driving. Summary of the invention

[0003] In view of the above, it is necessary to propose a vehicle blind spot target detection method, device, electronic device and storage medium to crop and splice the real-time image of the blind spot to solve the technical problem of how to quickly and accurately detect the detection target far away in the vehicle blind spot.

[0004] The present application provides a method for detecting a target in a vehicle blind spot, the method comprising:

[0005] Get real-time images of any blind spot around the vehicle;

[0006] Determine a region of interest image according to the real-time image;

[0007] Determine an image to be detected according to the image of the region of interest and the real-time image, wherein the image to be detected includes a first zoomed image corresponding to the real-time image and a second zoomed image corresponding to the image of the region of interest;

[0008] Input the image to be detected into the target detection model, generate a first detection frame corresponding to the detection target in the real-time image in the first zoomed image, and a second detection frame corresponding to the detection target in the focus area image in the second zoomed image;

[0009] Determine a blind spot target detection result according to the first detection frame and the second detection frame.

[0010] It can be seen from the above technical scheme that the present application collects real-time images of any vehicle blind spot, captures a pre-set area of ​​interest in the real-time image, obtains an image of the area of ​​interest, further scales the image of the area of ​​interest and the real-time image at different ratios and splices them into an image to be detected of a preset size; the image to be detected is used as the input of the target detection model, and the detection frames of all detection targets in the image of the area of ​​interest and the real-time image at different scales can be simultaneously obtained through only one target detection inference, thereby improving the detection speed and detection accuracy; at the same time, all detection frames are screened according to the overlap of the detection frames in the image of the area of ​​interest and the real-time image, thereby further improving the detection accuracy, and then the detection targets in the blind spot of the vehicle can be detected quickly and accurately, so as to leave the driver with sufficient reaction time to make corresponding judgments.

[0011] In some embodiments, determining the region of interest image according to the real-time image includes: determining the region of interest in the real-time image according to a preset field of view vanishing point and a preset size in the real-time image; and intercepting the region of interest to obtain the region of interest image.

[0012] In some embodiments, determining the image to be detected based on the area of ​​interest image and the real-time image includes: creating a background image that meets a preset background size; scaling the real-time image according to a preset first scaling ratio to obtain a first scaled image, and scaling the area of ​​interest image according to a preset second scaling ratio to obtain a second scaled image; and pasting the first scaled image and the second scaled image to the background image to obtain the image to be detected.

[0013] In some embodiments, the background image includes a first area, a second area and a third area, the second area is located between the first area and the third area, wherein the first area is occupied by the first zoom image and the third area is occupied by the second zoom image.

[0014] In some embodiments, before determining the blind spot target detection result based on the first detection frame and the second detection frame, the method also includes: if it is determined that there is an intersection between the first detection frame and the second area, deleting the first detection frame; if it is determined that there is an intersection between the second detection frame and the second area, deleting the second detection frame.

[0015] In some embodiments, determining the blind spot target detection result based on the first detection frame and the second detection frame includes: projecting the first detection frame to the real-time image and projecting the second detection frame to the focus area image; calculating the overlap between the first detection frame in the real-time image and the second detection frame in the focus area image; and filtering the first detection frame and the second detection frame based on the size of the overlap to obtain the blind spot target detection result.

[0016] In some embodiments, calculating the degree of overlap between the first detection frame in the real-time image and the second detection frame in the focus area image includes: restoring the focus area image to the focus area in the real-time image; calculating the intersection-and-union ratio of the first detection frame of the detection target in the real-time image and the second detection frame of the detection target in the focus area image, and setting the intersection-and-union ratio as the degree of overlap.

[0017] An embodiment of the present application also provides a vehicle blind spot target detection device, the device comprising: an acquisition unit, used to acquire a real-time image of any blind spot around the vehicle, wherein the real-time image contains a detection target; a first determination unit, used to determine a focus area image based on the real-time image; a second determination unit, used to determine an image to be detected based on the focus area image and the real-time image, wherein the image to be detected includes a first zoomed image corresponding to the real-time image and a second zoomed image corresponding to the focus area image; a target detection unit, used to input the image to be detected into a target detection model, generate a first detection frame corresponding to the detection target in the first zoomed image, and a second detection frame corresponding to the detection target in the second zoomed image; a screening unit, used to determine a blind spot target detection result based on the first detection frame and the second detection frame.

[0018] An embodiment of the present application also provides an electronic device, which includes: a memory storing at least one instruction; and a processor executing the instructions stored in the memory to implement the vehicle blind spot target detection method.

[0019] An embodiment of the present application also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the vehicle blind spot target detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is an application scenario diagram of the vehicle blind spot target detection method involved in this application.

[0021] Figure 2 It is a schematic diagram of the blind spot image involved in this application.

[0022] Figure 3 It is a flow chart of an embodiment of the vehicle blind spot target detection method involved in the present application.

[0023] Figure 4 It is a schematic diagram of an embodiment of acquiring an image to be detected involved in the present application.

[0024] Figure 5It is a schematic diagram of another embodiment of acquiring an image to be detected involved in the present application.

[0025] Figure 6 Schematic diagram of the anomaly detection frame involved in this application.

[0026] Figure 7 It is a functional module diagram of an embodiment of the vehicle blind spot target detection device involved in the present application.

[0027] Figure 8 It is a structural schematic diagram of an electronic device of an embodiment of a vehicle blind spot target detection method involved in the present application. DETAILED DESCRIPTION

[0028] In order to more clearly understand the purpose, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth to facilitate a full understanding of the present application, and the embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments.

[0029] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0031] like Figure 1 The following is an application scenario diagram of the technical solution of the present application. The present method is applied to an on-board system in a vehicle. When the vehicle 10 is driving, in order to avoid accidents caused by pedestrians or vehicles moving into the blind spots of the left and right rearview mirrors, it is necessary to perform blind spot detection on all blind spots around the vehicle to promptly detect pedestrians or vehicles in the blind spots of the vehicle. Figure 1As shown, targets 211 and 212 are traveling in blind spot 210 on the right side of vehicle 10, and targets 311 and 312 are traveling in blind spot 310 on the left side of vehicle 10. To ensure that vehicle 10 and targets can maintain a safe distance to avoid accidents, camera 20 on the right side of vehicle 10 can be used to monitor blind spot 210, and camera 30 on the left side of vehicle 10 can be used to monitor blind spot 310. Targets (pedestrians or vehicles) in real-time images captured by cameras 20 and 30 are detected with the aid of a target detection model to obtain blind spot detection results, so as to assist the driver of vehicle 10 or the autonomous driving program running in the vehicle system to avoid accidents in a timely manner.

[0032] Under the same reaction time, the faster the vehicle travels, the farther away pedestrians or vehicles in the blind spot should be detected, so as to give the driver sufficient reaction time to make corresponding judgments. Figure 2 The imaging schematic diagram of the blind spot 310 is shown. Due to the characteristics of the camera imaging that objects are larger near and smaller far away, and are clear near and fuzzy far away, in the imaging picture of the blind spot 310 monitored by the camera 30, the target 311 is smaller and more blurred than the target 312. Therefore, when the existing blind spot target detection method detects the real-time image of the blind spot 310, the target 312 may be misjudged, and the target 312 cannot be detected quickly and accurately, resulting in a short reaction time for the driver, thereby reducing the safety of the car driving.

[0033] The embodiment of the present application provides a vehicle blind spot target detection method, which can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable gate array (Field-Programmable Gate Array, FPGA), a digital processor (Digital Signal Processor, DSP), an embedded device, etc.

[0034] An electronic device can be any electronic product that can interact with a customer, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0035] The electronic device may also include a network device and / or a client device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0036] The network where the electronic device is located includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0037] like Figure 3 FIG. 1 is a flow chart of a preferred embodiment of the vehicle blind spot target detection method of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0038] S10, obtaining a real-time image of any blind spot around the vehicle, wherein the real-time image includes a detection target.

[0039] In one embodiment, multiple cameras can be deployed around the vehicle, and the cameras are used to collect real-time images in the blind spots around the vehicle and send the real-time images to the electronic device. The real-time images are collected by any camera deployed around the vehicle, and the cameras include cameras installed on the front, rear, and left and right rearview mirrors, that is, there is at least one blind spot around the vehicle, and the real-time image of each blind spot is collected by the corresponding camera.

[0040] The real-time image contains detection targets, including pedestrians, animals, vehicles, and static obstacles.

[0041] S11, determining a region of interest image according to the real-time image.

[0042] In one embodiment, the focus area image is the image information of the focus area in the real-time image, and the focus area can be an area far from the vehicle, used to reflect the position of the detection target far away in the blind spot; it can also be any area pre-set in the real-time image, used to reflect any focus area in the blind spot, such as the area close to the edge of the blind spot, the area close to the vehicle in the blind spot, etc. The detection targets include pedestrians, animals, vehicles, static obstacles, etc.

[0043] In one embodiment, determining the region of interest image according to the real-time image comprises:

[0044] Determining a region of interest in the real-time image according to a preset field of view vanishing point and a preset size in the real-time image;

[0045] The focus area is intercepted to obtain the focus area image.

[0046] Among them, the field of view vanishing point is the disappearance position of the detection target in the distance in the real-time image. Since the position and angle of the blind spot monitoring cameras deployed around the vehicle are fixed, the position of the field of view vanishing point remains unchanged. A pixel point in the real-time image can be marked as the field of view vanishing point in advance. For example, the disappearance position of the lane line from near to far in the real-time image can be marked as the field of view vanishing point.

[0047] The preset size refers to a preset rectangular size. An area that meets the preset size is cut out from the real-time image, and the field of view vanishing point is included in the area that meets the preset size to obtain a region of interest image. Since the field of view vanishing point is included in the region of interest, and the field of view vanishing point is used to represent a pixel point at a farther position in the real-time image, the region of interest image can be used to represent image information represented by the farther region in the real-time image.

[0048] In another embodiment, the region of interest may be directly determined in the real-time image according to a preset size, and image information in the region of interest may be intercepted to obtain an image of the region of interest.

[0049] In this embodiment, the focus area is used to characterize any area in the real-time image that needs to be analyzed intensively. For example, the focus area may be an area in the blind spot that is closer to the vehicle, or an area close to the edge of the blind spot.

[0050] In this way, the image of the area of ​​interest that needs to be analyzed is extracted from the real-time image, thereby narrowing the receptive field of interest in the blind spot, providing data support for the subsequent construction of the image to be detected, thereby improving the accuracy of blind spot target detection.

[0051] S12, determining an image to be detected according to the region of interest image and the real-time image, wherein the image to be detected includes a first zoomed image corresponding to the real-time image and a second zoomed image corresponding to the region of interest image.

[0052] In one embodiment, a first zoomed image corresponding to a real-time image and a second zoomed image corresponding to an area of ​​interest image are spliced ​​into an image to be detected. Since the area of ​​interest image is image information of a partial area in the real-time image, multi-scale information can be added to the image to be detected after the first zoomed image and the second zoomed image are spliced, thereby improving the accuracy of subsequent target detection; and it is convenient to use a single target detection model for target detection in the subsequent target detection process, thereby reducing the complexity of the model and further improving the efficiency of blind spot target detection.

[0053] In one embodiment, determining the image to be detected according to the area of ​​interest image and the real-time image specifically includes:

[0054] Create a background image that meets the preset background size;

[0055] Scaling the real-time image according to a preset first scaling ratio to obtain a first scaling image, and scaling the region of interest image according to a preset second scaling ratio to obtain a second scaling image;

[0056] Paste the first zoom image and the second zoom image to the background image to obtain an image to be detected.

[0057] In this embodiment, the preset background size refers to a preset rectangular size, the background image refers to a rectangular image that meets the preset background size, and the background image is an empty image that does not contain any image information.

[0058] In this embodiment, the preset first zoom ratio is used to scale the size of the real-time image to obtain a first zoomed image having the same image information as the real-time image but a different size; the preset second zoom ratio is used to scale the size of the image of the area of ​​interest to obtain a second zoomed image having the same image information as the image of the area of ​​interest but a different size.

[0059] In some embodiments, the preset background size includes a preset width and a preset height, and the setting method of the preset first zoom ratio includes: when the preset width and the preset height are the same, any one of the preset width and the preset height is used as the reference size; when the preset width and the preset height are different, the smaller value of the preset width and the preset height is used as the reference size. The reference size can be used to calculate the first zoom ratio to zoom the real-time image, which can ensure that all pixel information of the real-time image can be included in the image to be detected. The larger value of the width and height in the real-time image is used as the zoom size, and the ratio of the reference size to the zoom size is used as the first zoom ratio.

[0060] Exemplarily, when the width of the real-time image is 1920 pixels and the height is 1080 pixels, when the preset first zoom ratio is 0.6, the width of the first zoom image is 1920*0.6=1152 pixels, and the height of the first zoom image is 1080*0.6=648 pixels; when the width of the area of ​​interest image is 640 pixels and the height is also 640 pixels, when the preset second zoom ratio is 0.4, the width of the second zoom image is 640*0.4=256 pixels, and the height of the second zoom image is also 640*0.4=256 pixels.

[0061] In this embodiment, the first zoom image can be pasted to the upper side, lower side, left side or right side of the background image. When the first zoom image is pasted to the upper side or lower side of the background image, the first zoom image and the second zoom image are arranged along the direction of columns in the background image; when the first zoom image is pasted to the left side or right side of the background image, the first zoom image and the second zoom image are arranged along the direction of rows in the background image.

[0062] In one embodiment, the background image includes a first area, a second area and a third area, the second area is located between the first area and the third area, wherein the first area is occupied by the first zoom image and the third area is occupied by the second zoom image.

[0063] In this embodiment, after the first zoom image is pasted to one side of the background image, the background image is divided into a first area, a second area and a third area, wherein the first area is occupied by the first zoom image, and the second area is located between the third area and the first area, and is used to separate the first zoom image and the third area.

[0064] In one embodiment, the second zoom ratio can be calculated based on the third region and the region of interest image, specifically including: calculating a height ratio between the third region and the region of interest image, and a width ratio between the third region and the region of interest image; and taking the smaller value between the height ratio and the width ratio as the second zoom ratio. The second zoom ratio can ensure that the second zoom image can be pasted into the third region.

[0065] For example, when the width of the third region is 1920 pixels and the height is 324 pixels, and the width of the image of the region of interest is 1920 pixels and the height is 648 pixels, the height ratio between the third region and the image of the region of interest is If the width ratio between the third region and the focus region image is 1, the second scaling ratio may be set to 0.5.

[0066] In one embodiment, after obtaining the second zoom ratio, the image of the region of interest is zoomed according to the second zoom ratio to obtain a second zoomed image, and the second zoomed image is pasted to the third region to obtain an image to be detected that meets the preset size. The first zoomed image, the second region, and the second zoomed image can completely fill the background image; the second region and the third region are both rectangular regions; the image to be detected includes a first zoomed image corresponding to the real-time image, a second zoomed image corresponding to the image of the region of interest, and a second region located between the first zoomed image and the second zoomed image.

[0067] For example, see Figure 4 , taking the real-time image 400 captured by the camera of the left rearview mirror as an example, assuming that the preset width and the preset height are both N, and the width W of the real-time image is greater than the height H, and the background image is an N×N square; then the first zoom ratio is The first zoom ratio The real-time image 400 is scaled, and the obtained first scaled image 401 is pasted on the upper side of the background image, and the area occupied by the first scaled image 401 in the background image is the first area 402; further, the pixels of a preset number of rows adjacent to the first scaled image 401 in the background image are divided into a second area 403, and the area of ​​the background image other than the second area and the first scaled image is used as a third area 404; the central point of the highest point in the third area is overlapped with the field of view vanishing point 405 in the real-time image to construct a focus area in the real-time image 400, and the image information within the focus area in the real-time image 400 is cropped to obtain a focus area image 406; further, a second scaling ratio is calculated based on the third area 404 and the focus area image 405, and the focus area image 406 is scaled according to the second scaling ratio to obtain a second scaling image 407, and the second scaling image 407 is pasted to the third area 403 to obtain an image to be detected 408 with a size of N×N.

[0068] In another embodiment, a region of interest may be constructed based on the field of view vanishing point in the real-time image and a preset region of interest size, and the real-time image may be cropped based on the region of interest to obtain a region of interest image, wherein the region of interest is used to characterize a region in the real-time image that is far from the vehicle.

[0069] For example, see Figure 5 Taking the real-time image 500 captured by the camera of the left rearview mirror as an example, assuming that the preset width and the preset height are both N, and the width W of the real-time image is greater than the height H, the background image is an N×N square. The real-time image 500 is scaled, and the obtained first scaled image 501 is pasted on the upper side of the background image, and the area occupied by the first scaled image 501 in the background image is the first area 502; at the same time, the center point of the height of the preset focus area size is overlapped with the field of view vanishing point 503 in the real-time image to construct the focus area, and the real-time image is cropped according to the focus area to obtain the focus area image 504; further, the pixels of a preset number of rows adjacent to the first scaled image 501 in the background image are divided into a second area 505, and the area of ​​the background image except the second area 505 and the first scaled image 501 is used as a third area 506; further, a second scaling ratio is calculated based on the third area 506 and the focus area image 504, and the focus area image 504 is scaled according to the second scaling ratio to obtain a second scaling image 507, and pasted to the blank area to obtain an image to be detected 508 with a size of N×N.

[0070] In this way, an image to be detected that meets the preset size is obtained, and the image to be detected includes real-time images and images of the area of ​​interest at different scaling ratios, and a reduction area is provided between the real-time images and images of the area of ​​interest at different scaling ratios, so as to provide support for subsequent screening of detection frames that span both the real-time image and the image of the area of ​​interest, so as to improve the accuracy of blind spot target detection.

[0071] S13, inputting the image to be detected into the target detection model, generating a first detection frame corresponding to the detection target in the first zoomed image, and a second detection frame corresponding to the detection target in the second zoomed image.

[0072] In one embodiment, a target detection model is used to perform a detection on the image to be detected, and two detection frames belonging to the first zoom image and the second zoom image, namely the first detection frame and the second detection frame, are obtained, which can improve the efficiency of blind spot target detection; and provide data support for the subsequent screening of blind spot target detection results based on the two detection frames, thereby improving the accuracy of blind spot detection results.

[0073] In one embodiment, the target detection model can be any existing target detection model such as YOLOV3, CenterNet, etc., and this application does not limit it. The input of the target detection model is the image to be detected, and the output is the first detection box corresponding to the detection target in the first zoom image, and the second detection box corresponding to the detection target in the second zoom image. The first detection box and the second detection box of any detection target are both rectangular boxes surrounding the detection target, and each first detection box or second detection box corresponds to a confidence level, and the value range of the confidence level is [0,1].

[0074] In this embodiment, the preset size corresponding to the image to be detected is related to the target detection model, and the preset size includes a preset width and a preset height, and the preset width and the preset height are the same or different.

[0075] For example, when the dimension of the input image of the target detection model is 1920*1080 pixels, the preset width of the image to be detected is 1920 pixels, and the preset height is 1080 pixels; for another example, when the dimension of the input image of the target detection model is 600*600 pixels, the preset width and preset height of the image to be detected are both 600 pixels.

[0076] In one embodiment, before determining the blind spot target detection result according to the first detection frame and the second detection frame, the method further includes:

[0077] If it is determined that the first detection frame and the second area have an intersection, deleting the first detection frame;

[0078] If it is determined that the second detection frame and the second area have an intersection, the second detection frame is deleted.

[0079] like Figure 6 As shown, in the image to be detected 600, since the second area 602 is located between the first zoom image 601 and the second zoom image 603, and there is no detection target in the second area 602, when any detection frame (e.g. Figure 6 606) and the second area 602, it indicates that the detection box spans the first zoom image 601 and the second zoom image 602, so the detection box is an abnormal detection box, and the first detection box 604 in the first zoom image 601 has no intersection with the second area 602, and the second detection box 605 in the second zoom image 603 has no intersection with the second area 602, so the first detection box 604 and the second detection box 605 are both normal detection boxes. The abnormal detection box 606 can be deleted, and the first detection box 604 and the second detection box 605 are retained to reduce misjudgment in the target detection results.

[0080] In this way, by determining whether there is an intersection between the detection frame and the second area of ​​the image to be detected, the detection frame is screened, and abnormal detection frames can be eliminated before obtaining the blind spot target detection result, thereby improving the accuracy of the blind spot target detection result.

[0081] S14: Determine a blind spot target detection result according to the first detection frame and the second detection frame.

[0082] In one embodiment, in order to improve the accuracy of the blind spot target detection result, the blind spot target detection result may be determined based on the first detection frame and the second detection frame to eliminate the detection frames in which misjudgment occurs in the blind spot target detection result.

[0083] In one embodiment, determining the blind spot target detection result according to the first detection frame and the second detection frame includes:

[0084] Projecting the first detection frame onto the real-time image, and projecting the second detection frame onto the focus area image;

[0085] Calculating the degree of overlap between the first detection frame in the real-time image and the second detection frame in the focus area image;

[0086] The first detection frame and the second detection frame are selected based on the overlap degree to obtain a blind area target detection result.

[0087] In this embodiment, the first detection frame may be projected into the real-time image based on the first zoom ratio; and the second detection frame may be projected into the focus area image based on the second zoom ratio.

[0088] For example, if the first zoom ratio is 1 / 2, it means that the first zoom image can be obtained by zooming the real-time image by 1 / 2. On the contrary, after the first detection frame A in the first zoom image is zoomed by 2 times, the first detection frame A can be projected into the real-time image. By traversing all the first detection frames and second detection frames in the image to be detected in the same way, the first detection frame can be projected into the real-time image, and the second detection frame can be projected into the image of the area of ​​interest.

[0089] In this way, the detection frame in the image to be detected is projected into the focus area image or the real-time image. The detection frames of all detected targets in the focus area image and the real-time image can be obtained simultaneously through only one target detection inference, thereby improving the detection speed. At the same time, the scaling ratios of the focus area image and the real-time image in the image to be detected are different, thereby realizing target detection at different scales and improving the detection accuracy.

[0090] In one embodiment, calculating the overlap between the first detection frame in the real-time image and the second detection frame in the focus area image includes:

[0091] Restoring the region of interest image to the region of interest in the real-time image;

[0092] An intersection-and-union ratio of a first detection frame of the detection target in the real-time image and a second detection frame of the detection target in the focus area image is calculated, and the intersection-and-union ratio is set as the overlap degree.

[0093] After projecting each of the detection frames onto the focus area image or the real-time image, the focus area image is overlapped with the focus area in the real-time image to restore the focus area image to the focus area in the real-time image, and for each of the detection targets, the intersection-over-Union (IoU) between the second detection frame of the detection target in the focus area image and the first detection frame in the real-time image is calculated, and the IoU is set as the overlap degree.

[0094] Exemplarily, for a certain detection target, if the area of ​​the second detection frame of the detection target in the focus area image is 30*30 pixels, and the area of ​​the first detection frame of the detection target in the real-time image is 30*30 pixels, after the focus area image and the focus area in the real-time image overlap, when the area of ​​intersection between the second detection frame and the first detection frame of the detection target is 10*10 pixels, the intersection and union ratio of the first detection frame and the second detection frame corresponding to the detection target is calculated as follows:

[0095]

[0096] In one embodiment, since the focus area image is obtained by cropping the real-time image based on the focus area, the real-time image includes all detection targets in the focus area. It should be noted that, ideally, all second detection frames in the focus area image should completely overlap with the first detection frame in the real-time image. Therefore, for the second detection frame in the focus area image, all detection frames can be screened according to the overlap of the second detection frame in the focus area image and the real-time image to further improve the detection accuracy.

[0097] In one embodiment, the screening of the first detection frame and the second detection frame based on the overlap to obtain a blind area target detection result includes:

[0098] For any detection frame within the focus area in the real-time image, screening the first detection frame according to the overlap degree and the confidence level corresponding to the first detection frame to obtain a first blind spot detection result;

[0099] For each detection frame outside the focus area of ​​the real-time image, compare the confidence of the detection frame with a preset confidence, and when the confidence of the detection frame is greater than the preset confidence, retain the detection frame until all detection frames outside the focus area are traversed, and use the retained detection frame outside the focus area as the second blind spot detection result;

[0100] The first blind spot detection result and the second blind spot detection result are combined as a blind spot detection result.

[0101] The preset confidence level may be 0.5.

[0102] In one embodiment, for any detection frame within the focus area in the real-time image, screening the detection frame according to the overlap of the detection frame and the confidence to obtain the first blind spot detection result specifically includes:

[0103] If the overlap of the detection frames is not greater than a preset overlap, deleting the detection frames;

[0104] If the overlap of the detection frame is greater than the preset overlap, and the confidence of the detection frame is greater than the preset confidence, the detection frame is retained, and the retained detection frame is used as the first blind spot detection result.

[0105] The preset overlap degree may be 0.5.

[0106] In this embodiment, the real-time image also includes detection frames outside the focus area. When the focus area includes the field of view vanishing point, the focus area represents the area farther away in the real-time image. The detection frames outside the focus area are detection targets in the area closer to the vehicle in the real-time image. Therefore, the detection frames outside the focus area can be recorded as near detection frames. Since the detection targets closer to the vehicle have obvious features, the near detection frames can be screened only based on the confidence. If the confidence of any near detection frame is greater than the preset confidence, the near detection frame is retained. If the confidence of any near detection frame is not greater than the preset confidence, the near detection frame is deleted. After traversing all the near detection frames, the near detection frame retained in the real-time image is used as the second blind spot detection result.

[0107] Finally, the first blind spot detection result and the second blind spot detection result are combined as a blind spot detection result.

[0108] In this way, all detection frames are screened according to the confidence and overlap of the detection frames, thereby improving the detection accuracy. Not only can the detection targets that are closer to the vehicle in the real-time image be identified, but also the detection targets that are farther away from the vehicle in the real-time image can be identified.

[0109] It can be seen from the above technical scheme that the present application collects real-time images of any blind spots, and captures the preset concerned area in the real-time image to obtain the concerned area image, and further scales the concerned area image and the real-time image at different proportions and splices them into an image to be detected of a preset size; the image to be detected is input into the target detection model, and the detection frames of all detection targets in the concerned area image and the real-time image at different scales can be simultaneously obtained through only one target detection inference, thereby improving the detection speed and detection accuracy; at the same time, all detection frames are screened according to the overlap of the detection frames in the concerned area image and the real-time image, thereby further improving the detection accuracy, and thus the detection targets in the blind spots of the vehicle can be detected quickly and accurately, so as to leave the driver with sufficient reaction time to make corresponding judgments.

[0110] See also Figure 7 , Figure 7 It is a functional module diagram of a preferred embodiment of the vehicle blind spot target detection device of the present application. The vehicle blind spot target detection device 11 includes an acquisition unit 110, a first determination unit 111, a second determination unit 112, a target detection unit 113, and a screening unit 114. The module / unit referred to in the present application refers to a series of computer-readable instruction segments that can be executed by the processor 13 and can complete fixed functions, which are stored in the memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0111] The acquisition unit 110 is used to acquire a real-time image of any blind spot around the vehicle.

[0112] The first determining unit 111 is configured to determine a region of interest image according to the real-time image.

[0113] The second determining unit 112 is used to determine the image to be detected according to the area of ​​interest image and the real-time image, wherein the image to be detected includes a first zoom image corresponding to the real-time image and a second zoom image corresponding to the area of ​​interest image.

[0114] The target detection unit 113 is used to input the image to be detected into the target detection model, generate a first detection frame corresponding to the detection target in the real-time image in the first zoom image, and a second detection frame corresponding to the detection target in the focus area image in the second zoom image.

[0115] The screening unit 114 is configured to determine a blind spot target detection result according to the first detection frame and the second detection frame.

[0116] In an optional embodiment, the first determining unit 111 is specifically configured to:

[0117] Determining a region of interest in the real-time image according to a preset field of view vanishing point and a preset size in the real-time image;

[0118] The focus area is intercepted to obtain a focus area image.

[0119] In an optional embodiment, the second determining unit 112 is specifically configured to:

[0120] Create a background image that meets the preset background size;

[0121] Scaling the real-time image according to a preset first scaling ratio to obtain a first scaling image, and scaling the region of interest image according to a preset second scaling ratio to obtain a second scaling image;

[0122] Paste the first zoom image and the second zoom image to the background image to obtain an image to be detected.

[0123] In an optional embodiment, the background image includes a first area, a second area and a third area, the second area is located between the first area and the third area, wherein the first area is occupied by the first zoom image and the third area is occupied by the second zoom image.

[0124] In an optional embodiment, before the screening unit 114 determines the blind spot target detection result according to the first detection frame and the second detection frame, the screening unit 114 is further configured to:

[0125] If it is determined that the first detection frame and the second area have an intersection, deleting the first detection frame;

[0126] If it is determined that the second detection frame and the second area have an intersection, the second detection frame is deleted.

[0127] In an optional embodiment, the screening unit 114 is specifically used for:

[0128] Projecting the first detection frame onto the real-time image, and projecting the second detection frame onto the focus area image;

[0129] Calculating the degree of overlap between the first detection frame in the real-time image and the second detection frame in the focus area image;

[0130] The first detection frame and the second detection frame are selected based on the overlap degree to obtain a blind area target detection result.

[0131] In an optional embodiment, the screening unit 114 is further configured to:

[0132] Restoring the region of interest image to the region of interest in the real-time image;

[0133] An intersection-and-union ratio of a first detection frame of the detection target in the real-time image and a second detection frame of the detection target in the focus area image is calculated, and the intersection-and-union ratio is set as the overlap degree.

[0134] It can be seen from the above technical scheme that the present application collects real-time images of any vehicle's blind spot, and crops the area in the real-time image that is far away from the vehicle to obtain an image of the area of ​​interest, and further scales the image of the area of ​​interest and the real-time image at different ratios and splices them into an image to be detected of a preset size; the image to be detected is used as the input of the target detection model, and the detection frames of all detection targets in the image of the area of ​​interest and the real-time image at different scales can be simultaneously obtained through only one target detection inference, thereby improving the detection speed and detection accuracy; at the same time, all detection frames are screened according to the overlap of the detection frames in the image of the area of ​​interest and the real-time image, thereby further improving the detection accuracy, and thus the detection targets in the distance in the blind spot of the vehicle can be quickly and accurately detected, so as to leave the driver with sufficient reaction time to make corresponding judgments.

[0135] See also Figure 8 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 is used to execute the computer-readable instructions stored in the memory to implement the vehicle blind spot target detection method described in any of the above embodiments.

[0136] In an optional embodiment, the electronic device 1 further includes a bus, a computer program stored in the memory 12 and executable on the processor 13, such as a vehicle blind spot target detection program.

[0137] Figure 8 Only the electronic device 1 having the memory 12 and the processor 13 is shown, and those skilled in the art can understand that Figure 8 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0138] Combination Figure 3The memory 12 in the electronic device 1 stores a plurality of computer-readable instructions to implement a vehicle blind spot target detection method, and the processor 13 can execute the plurality of instructions to implement: obtaining a real-time image of any blind spot around the vehicle; determining a focus area image according to the real-time image; determining an image to be detected according to the focus area image and the real-time image, wherein the image to be detected includes a first zoomed image corresponding to the real-time image and a second zoomed image corresponding to the focus area image; inputting the image to be detected into a target detection model to generate a first detection frame corresponding to the detection target in the real-time image in the first zoomed image, and a second detection frame corresponding to the detection target in the focus area image in the second zoomed image; and determining a blind spot target detection result according to the first detection frame and the second detection frame.

[0139] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 3 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0140] Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may be a bus structure or a star structure. The electronic device 1 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the electronic device 1 may also include input and output devices, network access devices, etc.

[0141] It should be noted that the electronic device 1 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.

[0142] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium can be non-volatile or volatile. The readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the vehicle blind spot target detection program, etc., but also can be used to temporarily store data that has been output or is to be output.

[0143] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and uses various interfaces and lines to connect various components of the entire electronic device 1, and executes or executes programs or modules stored in the memory 12 (for example, executing a vehicle blind spot target detection program, etc.), and calls data stored in the memory 12 to execute various functions of the electronic device 1 and process data.

[0144] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned vehicle blind spot target detection method embodiments, for example Figure 3 Steps shown.

[0145] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a collection unit 110, a splicing unit 111, a detection unit 112, a calculation unit 113, and a screening unit 114.

[0146] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute the vehicle blind spot target detection method described in each embodiment of the present application.

[0147] If the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware devices to complete through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0148] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory, etc.

[0149] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0150] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. Figure 8 Only one arrow is used in the figure, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.

[0151] An embodiment of the present application also provides a computer-readable storage medium (not shown), in which computer-readable storage medium is stored computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the vehicle blind spot target detection method described in any of the above embodiments.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0153] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0155] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the specification can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A vehicle blind spot target detection method, characterized in that: The method comprises: Get real-time images of any blind spot around the vehicle; Determine a region of interest image according to the real-time image; Determine an image to be detected according to the image of the region of interest and the real-time image, wherein the image to be detected includes a first zoomed image corresponding to the real-time image and a second zoomed image corresponding to the image of the region of interest; Input the image to be detected into the target detection model, generate a first detection frame corresponding to the detection target in the real-time image in the first zoomed image, and a second detection frame corresponding to the detection target in the focus area image in the second zoomed image; Determine a blind spot target detection result according to the first detection frame and the second detection frame.

2. The vehicle blind spot target detection method according to claim 1, characterized in that: Determining the region of interest image according to the real-time image comprises: Determining a region of interest in the real-time image according to a preset field of view vanishing point and a preset size in the real-time image; The focus area is intercepted to obtain the focus area image.

3. The vehicle blind spot target detection method according to claim 1, characterized in that: Determining an image to be detected according to the focus area image and the real-time image includes: Create a background image that meets the preset background size; Scaling the real-time image according to a preset first scaling ratio to obtain a first scaling image, and scaling the region of interest image according to a preset second scaling ratio to obtain a second scaling image; Paste the first zoom image and the second zoom image to the background image to obtain an image to be detected.

4. The vehicle blind spot target detection method according to claim 3, characterized in that: The background image includes a first area, a second area and a third area, the second area is located between the first area and the third area, wherein the first area is occupied by the first zoom image and the third area is occupied by the second zoom image.

5. The vehicle blind spot target detection method according to claim 4, characterized in that: Before determining the blind spot target detection result according to the first detection frame and the second detection frame, the method further includes: If it is determined that the first detection frame and the second area have an intersection, deleting the first detection frame; If it is determined that the second detection frame and the second area have an intersection, the second detection frame is deleted.

6. The vehicle blind spot target detection method according to claim 1, characterized in that: The determining the blind spot target detection result according to the first detection frame and the second detection frame includes: Projecting the first detection frame onto the real-time image, and projecting the second detection frame onto the focus area image; Calculating the degree of overlap between the first detection frame in the real-time image and the second detection frame in the focus area image; The first detection frame and the second detection frame are selected based on the overlap degree to obtain a blind area target detection result.

7. The vehicle blind spot target detection method according to claim 6, characterized in that: The calculating the overlap between the first detection frame in the real-time image and the second detection frame in the focus area image includes: Restoring the region of interest image to the region of interest in the real-time image; An intersection-and-union ratio of a first detection frame of the detection target in the real-time image and a second detection frame of the detection target in the focus area image is calculated, and the intersection-and-union ratio is set as the overlap degree.

8. A vehicle blind spot target detection device, characterized in that: The device comprises: An acquisition unit, used to acquire a real-time image of any blind spot around the vehicle; A first determining unit, configured to determine a region of interest image according to the real-time image; a second determining unit, configured to determine an image to be detected according to the image of the area of ​​interest and the real-time image, wherein the image to be detected includes a first zoomed image corresponding to the real-time image and a second zoomed image corresponding to the image of the area of ​​interest; A target detection unit, used for inputting the image to be detected into a target detection model, generating a first detection frame corresponding to the detection target in the real-time image in the first zoomed image, and a second detection frame corresponding to the detection target in the focus area image in the second zoomed image; A screening unit is used to determine a blind spot target detection result according to the first detection frame and the second detection frame.

9. An electronic device, characterized in that: The electronic device comprises: a memory storing computer-readable instructions; and A processor executes computer-readable instructions stored in the memory to implement the vehicle blind spot target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the vehicle blind spot target detection method according to any one of claims 1 to 7 is implemented.

Images