Method and device for correcting driving area, storage medium and electronic device

Abstract

The invention provides a method and device for correcting a drivable area, a storage medium and an electronic device, and the method comprises the steps: obtaining an original image collected by a vehicle camera; calculating initial boundary point information of a drivable area of the vehicle according to the original image, and calculating position information of each target in the original image; and correcting the initial boundary point information according to the position information to obtain target boundary point information of a drivable area of the vehicle. According to the embodiment of the invention, the technical problem that the relative position relation between the boundary point of the travelable area and the target frame generated in the related technology is inaccurate is solved, a reliable bounded safety area can be provided for the automatic driving vehicle in the data layer through the accurate information of the travelable area, and the safety of the automatic driving vehicle is effectively guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and device for correcting a drivable area, a storage medium, and an electronic device. Background Art

[0002] Among the related technologies, the development of intelligent driving technology has been particularly important. The importance of vision-based drivable area detection technology as a supporting task module for underlying security is self-evident. When faced with unfamiliar environments, non-whitelist obstacles and long-tail effects in various complex scenarios, the drivable area information in the image can provide autonomous driving vehicles with underlying, reliable, bounded safety areas in the current environment, effectively ensuring the safety of autonomous driving vehicles.

[0003] In the process of promoting relevant technologies and solutions, there are two main ways to realize the drivable area of ​​autonomous driving. The first "multi-sensor fusion" solution collects the surrounding information of the vehicle through cameras, millimeter wave radars, lasers, radars and other devices. The solution can obtain deeper spatial information due to the addition of laser radars, and the position, distance and size of objects are more accurately perceived. Moreover, since laser radars are self-luminous and are not affected by ambient light. However, the hardware cost of laser radars themselves is not low, and "multi-sensor fusion" often requires higher computing power for computing chips, so there is no cost advantage. The second "pure vision" solution is favored by automobile manufacturers or solution providers for its relatively lower cost, closer to human driving, and richer environmental information obtained through high-resolution and high-frame rate imaging technology. The detection results of the drivable area of ​​the vehicle are obtained according to the image segmentation detection model, and the detection results are input into the confidence network to obtain the confidence of this drivable area. The drivable area of ​​the current "pure vision" solution is obtained based on the image segmentation model, and dynamic targets (such as vehicles, pedestrians, cone barrels, etc.) are obtained by adding appropriate post-processing according to the image detection model. Therefore, the current relative position relationship between the existing drivable area and the target frame in the vehicle coordinate system is inconsistent with each other.

[0004] With respect to the above-mentioned problems existing in the related technologies, no efficient and accurate solutions have been found yet. Summary of the invention

[0005] The present invention provides a method and device for correcting a drivable area, a storage medium, and an electronic device to solve the technical problems in related technologies.

[0006] According to one embodiment of the present invention, a method for correcting a drivable area is provided, comprising: acquiring an original image captured by a vehicle camera; calculating initial boundary point information of the vehicle's drivable area based on the original image, and calculating position information of each target in the original image; and correcting the initial boundary point information based on the position information to obtain target boundary point information of the vehicle's drivable area.

[0007] Optionally, calculating the initial boundary point information of the vehicle's drivable area based on the original image includes: dividing the original image into multiple segmentation maps, wherein each segmentation map corresponds to a target; merging at least two segmentation maps belonging to the same object category in the multiple segmentation maps into a connected domain to obtain multiple connected domains; screening out the regional connected domain belonging to the vehicle's drivable area from the multiple connected domains; scanning the regional connected domain row by row or column by column to determine the boundary coordinates of the initial boundary points of the drivable area, and converting the boundary coordinates from the pixel coordinate system into a vehicle coordinate system with the vehicle coordinate system as the origin.

[0008] Optionally, filtering out a regional connected domain belonging to the drivable area of ​​the vehicle from the multiple connected domains includes: filtering out a road connected domain and an open-space connected domain from the multiple connected domains; and determining the road connected domain and the open-space connected domain as the regional connected domain belonging to the drivable area of ​​the vehicle.

[0009] Optionally, calculating the position information of each target in the original image includes: inputting the original image into a target detection model, and outputting detection frames of all targets; for each detection frame, estimating the first coordinate information and first size information of the target in the vehicle coordinate system according to the camera parameters of the camera; correcting the first size information to obtain second size information; and calculating the second coordinate information of each target in the vehicle coordinate system according to the camera parameters and the second size information.

[0010] Optionally, correcting the first size information to obtain the second size information includes: identifying the target category of the target; obtaining a reference size range corresponding to the target category; determining whether the first size information is within the reference size range; if the first size information is not within the reference size range, correcting the first size information based on the reference size range to obtain the second size information.

[0011] Optionally, the initial boundary point information is corrected according to the position information to obtain the target boundary point information of the vehicle's drivable area, including: determining an initial value of the drivable area, wherein the initial value includes the number of boundary points and the area size; generating a boundary point confirmation line according to the initial value and the position information, wherein the boundary point confirmation line is a ray extending outward from the vehicle as the starting point in the area size; based on the boundary point confirmation line, the initial boundary point information is corrected to obtain the target boundary point information of the vehicle's drivable area.

[0012] Optionally, generating a boundary point confirmation line based on the initial value and the position information includes: determining the centerline point coordinates of the first target, wherein the first target is a target whose target size is smaller than a preset threshold, and the position information includes the centerline point coordinates and the target size; taking the origin of the vehicle coordinate system as the starting point, extending toward the centerline point coordinates of each first target to generate a first number of first boundary point confirmation lines; taking the origin of the vehicle coordinate system as the starting point, extending outward a second number of rays of the same angle to generate a second number of second boundary point confirmation lines, wherein the sum of the first number and the second number is the number of boundary points.

[0013] Optionally, modifying the initial boundary point information based on the boundary point confirmation line includes: initializing the first intersection between the boundary point confirmation line and the box where the area size is located as the first boundary point; using any two adjacent initial boundary points in the initial boundary point information to generate an initial line segment; using the second intersection between the initial line segment and any of the boundary point confirmation lines to generate a second boundary point; generating a third boundary point based on the position information and the target category; and determining the target boundary point based on the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line.

[0014] Optionally, generating a third boundary point based on the position information and the target category includes: identifying the target category of the target, wherein the target category includes: non-vehicle category, vehicle category; if the target category is a non-vehicle category, determining the centerline point as the third boundary point, wherein the position information includes the centerline point; if the target category is a vehicle category, calculating four vertices of the target based on the center point, length, width, and heading angle of the target in the vehicle coordinate system, respectively calculating third intersections between the four vertices and the boundary point confirmation line, and selecting the third intersection closest to the vehicle among the four third intersections as the third boundary point.

[0015] Optionally, determining the target boundary point based on the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line includes: for each boundary point confirmation line, respectively calculating the first distance, the second distance, and the third distance between the first boundary point, the second boundary point, the third boundary point on the line and the vehicle; determining whether the third distance is greater than the second distance, and whether the fourth distance between the second boundary point and the third boundary point is less than a preset threshold; if the third distance is greater than the second distance, and the fourth distance between the second boundary point and the third boundary point is less than a preset threshold, determining the third boundary point as the target boundary point; if the third distance is less than or equal to the second distance, or the fourth distance between the second boundary point and the third boundary point is greater than or equal to the preset threshold, determining the smallest boundary point among the first distance, the second distance, and the third distance as the target boundary point.

[0016] According to another embodiment of the present invention, a device for correcting a drivable area is provided, comprising: an acquisition module for acquiring an original image captured by a vehicle camera; a calculation module for calculating initial boundary point information of the vehicle's drivable area based on the original image, and calculating position information of each target in the original image; and a correction module for correcting the initial boundary point information based on the position information to obtain target boundary point information of the vehicle's drivable area.

[0017] Optionally, the calculation module includes: a segmentation unit, used to divide the original image into multiple segmentation maps, wherein each segmentation map corresponds to a target; a merging unit, used to merge at least two segmentation maps belonging to the same object category in the multiple segmentation maps into a connected domain, so as to obtain multiple connected domains; a screening unit, used to screen out the regional connected domain belonging to the drivable area of ​​the vehicle from the multiple connected domains; a determination unit, used to scan the regional connected domain row by row or column by column, determine the boundary coordinates of the initial boundary points of the drivable area, and convert the boundary coordinates from the pixel coordinate system into a vehicle coordinate system with the vehicle coordinate system as the origin.

[0018] Optionally, the screening unit includes: a screening subunit, used to screen out road connected domains and open-space connected domains from the multiple connected domains; and a determination subunit, used to determine the road connected domains and the open-space connected domains as regional connected domains belonging to the drivable area of ​​the vehicle.

[0019] Optionally, the calculation module includes: a processing unit, used to input the original image into a target detection model and output detection frames of all targets; an estimation unit, used to estimate, for each detection frame, the first coordinate information and first size information of the target in the vehicle coordinate system according to the camera parameters of the camera; a correction unit, used to correct the first size information to obtain second size information; and a calculation unit, used to calculate the second coordinate information of each target in the vehicle coordinate system according to the camera parameters and the second size information.

[0020] Optionally, the correction unit includes: an identification subunit, used to identify the target category of the target; an acquisition subunit, used to acquire a reference size range corresponding to the target category; a judgment subunit, used to judge whether the first size information is within the reference size range; and a correction subunit, used to correct the first size information based on the reference size range to obtain second size information if the first size information is not within the reference size range.

[0021] Optionally, the correction module includes: a determination unit, used to determine the initial value of the drivable area, wherein the initial value includes the number of boundary points and the area size; a generation unit, used to generate a boundary point confirmation line based on the initial value and the position information, wherein the boundary point confirmation line is a ray extending outward from the vehicle as the starting point in the area size; a correction unit, used to correct the initial boundary point information based on the boundary point confirmation line to obtain the target boundary point information of the drivable area of ​​the vehicle.

[0022] Optionally, the generation unit includes: a determination subunit, used to determine the centerline point coordinates of the first target, wherein the first target is a target whose target size is smaller than a preset threshold, and the position information includes the centerline point coordinates and the target size; a first generation subunit, used to take the origin of the vehicle coordinate system as the starting point, extend toward the centerline point coordinates of each first target, and generate a first number of first boundary point confirmation lines; a second generation subunit, used to take the origin of the vehicle coordinate system as the starting point, extend outward a second number of rays of the same angle, and generate a second number of second boundary point confirmation lines, wherein the sum of the first number and the second number is the number of boundary points.

[0023] Optionally, the correction unit includes: a processing subunit, used to initialize the first intersection between the boundary point confirmation line and the box where the area size is located as a first boundary point; a first generation subunit, used to generate an initial line segment using any two adjacent initial boundary points in the initial boundary point information; a second generation subunit, used to generate a second boundary point using the second intersection between the initial line segment and any of the boundary point confirmation lines; a third generation subunit, used to generate a third boundary point based on the position information and the target category; and a determination subunit, used to determine the target boundary point based on the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line.

[0024] Optionally, the third generating subunit is also used to: identify the target category of the target, wherein the target category includes: non-vehicle category, vehicle category; if the target category is a non-vehicle category, determine the centerline point as the third boundary point, wherein the position information includes the centerline point; if the target category is a vehicle category, calculate the four vertices of the target according to the center point, length, width, and heading angle of the target in the vehicle coordinate system, calculate the third intersection points between the four vertices and the boundary point confirmation line respectively, and select the third intersection point closest to the vehicle among the four third intersection points as the third boundary point.

[0025] Optionally, the determination subunit is also used to: for each boundary point confirmation line, calculate the first distance, the second distance, and the third distance between the first boundary point, the second boundary point, the third boundary point on the line and the vehicle respectively; determine whether the third distance is greater than the second distance, and whether the fourth distance between the second boundary point and the third boundary point is less than a preset threshold; if the third distance is greater than the second distance, and the fourth distance between the second boundary point and the third boundary point is less than a preset threshold, determine the third boundary point as the target boundary point; if the third distance is less than or equal to the second distance, or the fourth distance between the second boundary point and the third boundary point is greater than or equal to the preset threshold, determine the boundary point with the smallest of the first distance, the second distance, and the third distance as the target boundary point.

[0026] According to another aspect of an embodiment of the present application, a storage medium is further provided, which includes a stored program, and the above steps are executed when the program is run.

[0027] According to another aspect of an embodiment of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; wherein: the memory is used to store computer programs; and the processor is used to execute the steps in the above method by running the program stored in the memory.

[0028] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps in the above method.

[0029] Beneficial effects of the present invention:

[0030] 1. Provide a drivable area correction method based on target detection to solve the problem that the relative position relationship between the edge points of the drivable area and the target frame is inaccurate in the vehicle coordinate system;

[0031] 2. Accurate drivable area information can provide a reliable bounded safety area for autonomous driving vehicles at the data layer, effectively ensuring the safety of autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0033] Figure 1 is a hardware structure block diagram of a car according to an embodiment of the present invention;

[0034] Figure 2 is a flow chart of a method for correcting a drivable area according to an embodiment of the present invention;

[0035] Figure 3 is a schematic diagram of generating a boundary point confirmation line in a vehicle coordinate system according to an embodiment of the present invention;

[0036] Figure 4 is a schematic diagram of correcting target boundary point information of a drivable area in an embodiment of the present invention;

[0037] Figure 5 is a schematic diagram of generating a third boundary point in an embodiment of the present invention;

[0038] Figure 6 is a schematic diagram of boundary point correction based on target detection in an embodiment of the present invention;

[0039] Figure 7 4 is a structural block diagram of a device for correcting a drivable area according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] Example 1

[0043] The method embodiment provided in the first embodiment of the present application can be executed in a car, a server, a processor, a controller or a similar processing device. Taking running on a car as an example, Figure 1 1 is a hardware structure diagram of a car according to an embodiment of the present invention. Figure 1 As shown, a car may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the above-mentioned automobile may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above-mentioned automobile. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0044] The memory 104 can be used to store automobile programs, for example, software programs and modules of application software, such as an automobile program corresponding to a method for correcting a drivable area of ​​an automobile in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the automobile program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the automobile via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0045] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the car. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0046] In this embodiment, a method for correcting a drivable area is provided. Figure 2 is a flow chart of a method for correcting a drivable area according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0047] Step S202, obtaining an original image captured by a vehicle camera;

[0048] Optionally, after acquiring the original image, multiple frames of images captured by multiple cameras may be stitched, denoised, and processed.

[0049] Step S204, calculating the initial boundary point information of the drivable area of ​​the vehicle according to the original image, and calculating the position information of each target in the original image;

[0050] The vehicle's freespace is a safe area for the vehicle to travel in and is used to make decisions during autonomous driving or active safety.

[0051] Step S206: modify the initial boundary point information according to the position information to obtain target boundary point information of the drivable area of ​​the vehicle.

[0052] Through the above steps, the original image captured by the vehicle camera is obtained; the initial boundary point information of the vehicle's drivable area is calculated according to the original image, and the position information of each target in the original image is calculated; the initial boundary point information is corrected according to the position information to obtain the target boundary point information of the vehicle's drivable area. By calculating the position information of each target in the image and correcting the initial boundary point information, a drivable area correction method based on target detection is implemented, which solves the technical problem of inaccurate relative position relationship between the drivable area boundary points and the target frame generated in the related technology. Through accurate drivable area information, a reliable bounded safety area can be provided for the autonomous driving vehicle at the data layer, effectively ensuring the safety of the autonomous driving vehicle.

[0053] In one implementation of the present embodiment, calculating the initial boundary point information of the drivable area of ​​the vehicle based on the original image includes: dividing the original image into multiple segmentation maps, wherein each segmentation map corresponds to a target; merging at least two segmentation maps belonging to the same object category in the multiple segmentation maps into a connected domain to obtain multiple connected domains; screening out the regional connected domain belonging to the drivable area of ​​the vehicle from the multiple connected domains; scanning the regional connected domain row by row or column by column to determine the boundary coordinates of the initial boundary points of the drivable area, and converting the boundary coordinates from the pixel coordinate system to the vehicle coordinate system with the vehicle coordinate system as the origin.

[0054] The boundary coordinates of the initial boundary points of the drivable area are a set, including boundary coordinates corresponding to multiple initial boundary points. Optionally, the boundary coordinates after conversion to the vehicle coordinate system can be stored through an initial boundary point list or an initial boundary point matrix.

[0055] In one example, screening out a regional connected domain belonging to the drivable area of ​​the vehicle from the multiple connected domains includes: screening out a road connected domain and an open-space connected domain from the multiple connected domains; and determining the road connected domain and the open-space connected domain as the regional connected domain belonging to the drivable area of ​​the vehicle.

[0056] Identify the drivable area from the original image, convert it into the vehicle coordinate system, and perform subsequent corrections, including image segmentation, connected domain analysis, coordinate system conversion and other processes. Here is an explanation and description:

[0057] Image segmentation: The camera captures the real-time scene image and uses it as the original input. Then, a series of preprocessing operations such as data normalization, cropping and scaling are performed. The deep learning model is then used for image segmentation. This image segmentation model can identify the boundaries of different object categories from the original image and output a label map of the same size as the original image, where each pixel corresponds to a specific category (such as vehicles, pedestrians, two-wheeled vehicles, cones, static targets, background, etc.).

[0058] Connected domain analysis is used to identify continuous regions belonging to the same category in the segmented image. At this stage: Filter connected domains: Only those areas that are considered to be drivable (such as roads or open spaces) are retained, and obstacles, static targets, etc. are excluded. Determine boundary points: Scan the selected connected domain line by line from bottom to top to find edge pixels as boundary points of freespace.

[0059] Coordinate system conversion, since segmentation and analysis are usually performed in image space, it is necessary to convert these boundary points from pixel coordinates (usually pixels on the screen) to the vehicle coordinate system (in meters). It is mainly based on the camera's internal parameters, external parameters, and the size information of the vehicle to define a pixel value mapped to the actual physical distance in the vehicle coordinate system. The final output is a list or matrix of freespace boundary points in the vehicle coordinate system.

[0060] In one implementation of the present embodiment, calculating the position information of each target in the original image includes: inputting the original image into a target detection model and outputting detection frames of all targets; for each detection frame, estimating the first coordinate information and first size information of the target in the vehicle coordinate system according to the camera parameters of the camera; correcting the first size information to obtain second size information; and calculating the second coordinate information of each target in the vehicle coordinate system according to the camera parameters and the second size information.

[0061] The target is detected in 2D based on the original image captured by the camera, the detection frame information of the target is output, and the coordinates of the target in the vehicle coordinate system and its attribute information are calculated.

[0062] The original image captured by the camera is input into the object detection model. This object detection model is trained to identify and locate specific objects in the image (such as vehicles, pedestrians, and cones), and provides the approximate position, size, and corresponding confidence score of each detection frame. If it is a vehicle, the full vehicle frame, the rear frame, and the wheel frame will be detected at the same time. For pedestrians or cones, only one pedestrian detection frame or cone detection frame will be output.

[0063] Optionally, after detecting the detection frame in the original image of the current frame, the historical detection frame in the original image of the historical frame can be obtained for target tracking. The authenticity of the detection frame in the original image of the current frame can be determined by combining the historical detection frame in the historical frame. The target that does not appear in the current frame but appears in the historical frame can also be added to the target output by the current frame. When tracking the target, non-maximum suppression (NMS) and the Hungarian algorithm are used to match the target detection frame between the two frames. This process is based on the intersection over union (IoU) to determine whether the detection frame in the current frame corresponds to an object in the previous frame. Combined with the feature map vector of the detection frame output by the image detection model, the detection frame tracking of the previous and next two frames is performed according to the cosine similarity of the feature map vector.

[0064] In one example, correcting the first size information to obtain the second size information includes: identifying the target category of the target; obtaining a reference size range corresponding to the target category; determining whether the first size information is within the reference size range; if the first size information is not within the reference size range, correcting the first size information based on the reference size range to obtain the second size information.

[0065] Optionally, target categories include vehicles, vehicles, traffic cones, stones, etc. Vehicles can be divided into cars, trucks, mechanical engineering vehicles, large trucks, small trucks, tricycles, electric vehicles, bicycles, motorcycles, etc.

[0066] Optionally, a reference size range is pre-set for each target category. For example, the reference size range for the length, width and height of a car is 4500*1800*1600 (mm). For car 1 in the target frame detected in the original image, if its estimated first size information is 6500*1600*1500 (mm), its length has exceeded the normal reference size range and there is visual distortion. It needs to be corrected with the reference size range as the maximum allowable size to be within 4500 mm.

[0067] When calculating the first coordinate information and the first size information, firstly, the three-dimensional position (first coordinate information) of the target in the vehicle coordinate system is estimated by combining the camera intrinsic and extrinsic information of the vehicle camera with the position information of the target detection frame. Secondly, the first size information such as the length, width, and height of the target is estimated according to the pinhole camera model of the camera. Then, the detected size information is adjusted according to the target type. If the target type is a vehicle, the heading angle of the target can also be calculated based on the full vehicle frame, the rear frame, or the wheel frame.

[0068] In the process of correcting the first size information, the relationship between the width and height of the target in the image and the camera's intrinsic parameter matrix is ​​used, combined with known information such as focal length, and the pinhole imaging principle is used to reversely solve the precise position of the target in the vehicle coordinate system (second coordinate information). The final output will include the position information of each detected target object (such as the center point position, midline position, center of mass position, etc.), and for vehicle-type targets, additional attributes such as length, width, and heading angle will be provided.

[0069] In one implementation of the present embodiment, the initial boundary point information is corrected according to the position information to obtain the target boundary point information of the drivable area of ​​the vehicle, including: determining an initial value of the drivable area, wherein the initial value includes the number of boundary points and the area size; generating a boundary point confirmation line according to the initial value and the position information, wherein the boundary point confirmation line is a ray extending outward from the vehicle as the starting point in the area size; based on the boundary point confirmation line, the initial boundary point information is corrected to obtain the target boundary point information of the drivable area of ​​the vehicle.

[0070] Based on the initial boundary point information generated above, as well as the generated target frame and target attributes and other position information, the corrected target boundary point information is output.

[0071] In one example, when setting the initial value, the number of output target boundary points is set to Fnum (such as 256 points), and the maximum area Area enclosed by the target boundary points in the vehicle coordinate system is set (such as the maximum lateral distance is 50m, and the maximum longitudinal distance is 500m).

[0072] In one example, generating a boundary point confirmation line based on the initial value and the position information includes: determining the centerline point coordinates of a first target, wherein the first target is a target whose target size is smaller than a preset threshold, and the position information includes the centerline point coordinates and the target size; starting from the origin of the vehicle coordinate system, extending toward the centerline point coordinates of each first target to generate a first number of first boundary point confirmation lines; starting from the origin of the vehicle coordinate system, extending outward a second number of rays of the same angle to generate a second number of second boundary point confirmation lines, wherein the sum of the first number and the second number is the number of boundary points.

[0073] Figure 3 Schematic diagram of generating boundary point confirmation lines in a vehicle coordinate system according to an embodiment of the present invention. The vehicle coordinate system takes the center of the vehicle as the origin, the longitudinal extension direction is the x-axis direction, and the lateral extension direction is the y-axis direction. Since the projections of pedestrians and cones in the vehicle coordinate system are relatively small, they are taken as the first target. First, a freespace confirmation line is generated according to the center point positions of pedestrians and cones (see Figure 3 The thick dotted line indicates the first boundary point confirmation line for pedestrians / cones). Assume that the coordinates of the midline point of the pedestrian or cone are (x 0 ,y 0 ), then the equation of the pedestrian / cone exclusive freespace confirmation line is:

[0074]

[0075] If there are Pnum cones and pedestrians, Pnum pedestrian / cone-specific freespace confirmation lines will be generated.

[0076] Secondly, according to the number of output freespace points Fnum, the default maximum area of ​​freespace in the vehicle coordinate system Area, generate the remaining second boundary point confirmation lines. With the origin of the vehicle coordinate system as the center, divide the default maximum area of ​​freespace in the vehicle coordinate system Area into rays with (Fnum-Pnum) angles (see Figure 3 The gray thin dashed line indicates the second boundary point confirmation line). The equation of the second boundary point confirmation line is: y = tan (n × α) × x;

[0077] in,

[0078] In one example, modifying the initial boundary point information based on the boundary point confirmation line includes: initializing the first intersection between the boundary point confirmation line and the box where the area size is located as the first boundary point; using any two adjacent initial boundary points in the initial boundary point information to generate an initial line segment; using the second intersection between the initial line segment and any of the boundary point confirmation lines to generate a second boundary point; generating a third boundary point based on the position information and the target category; and determining the target boundary point based on the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line.

[0079] In this example, the first boundary point is freespace0, the second boundary point is freespace1, and the third boundary point is freespace2. Each type of boundary point is a set, and its position information in the vehicle coordinate system is stored in a list or matrix.

[0080] Figure 4This is a schematic diagram of the target boundary point information of the drivable area corrected in the embodiment of the present invention. First, an array with a dimension of Fnum is generated. According to the generated boundary point confirmation line, the intersection formed with the default maximum area Area of ​​the freespace in the vehicle coordinate system is initialized to obtain a freespace point (freespace0). Secondly, according to the intersection formed by the line segment formed by two adjacent points in the generated initial boundary point and the boundary point confirmation line, a new freespace1 is generated.

[0081] Optionally, generating a third boundary point based on the position information and the target category includes: identifying the target category of the target, wherein the target category includes: non-vehicle category, vehicle category; if the target category is a non-vehicle category, determining the centerline point as the third boundary point, wherein the position information includes the centerline point; if the target category is a vehicle category, calculating four vertices of the target based on the center point, length, width, and heading angle of the target in the vehicle coordinate system, respectively calculating third intersections between the four vertices and the boundary point confirmation line, and selecting the third intersection closest to the vehicle among the four third intersections as the third boundary point.

[0082] In this example, a new freespace 2 (third boundary point) is generated based on the target frame midline points and target frame attributes of each target generated and the intersection point closest to the vehicle generated by the boundary point confirmation line. Figure 5 : is a schematic diagram of generating the third boundary point in an embodiment of the present invention, a is the heading angle of the vehicle, W is the width of the vehicle, L is the length of the vehicle, P0 is the center point of the vehicle (also the origin of the vehicle coordinate system), P1-P4: the four vertices (corner points) of the vehicle, and F2 is the closest intersection point between the boundary point confirmation line and the vehicle, that is, the generated third boundary point.

[0083] Optionally, determining the target boundary point based on the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line includes: for each boundary point confirmation line, respectively calculating the first distance, the second distance, and the third distance between the first boundary point, the second boundary point, the third boundary point on the line and the vehicle; determining whether the third distance is greater than the second distance, and whether the fourth distance between the second boundary point and the third boundary point is less than a preset threshold; if the third distance is greater than the second distance, and the fourth distance between the second boundary point and the third boundary point is less than a preset threshold, determining the third boundary point as the target boundary point; if the third distance is less than or equal to the second distance, or the fourth distance between the second boundary point and the third boundary point is greater than or equal to the preset threshold, determining the smallest boundary point among the first distance, the second distance, and the third distance as the target boundary point.

[0084] In one implementation scenario, the freespace0(x 0 ,y 0 ), freespace1(x 1 ,y 1 ) and freespace2(x 2 ,y 2 ) to confirm the freespace point of the final output. The specific calculation method is as follows:

[0085] Define R as the distance from any freespace point to the vehicle, that is, the distance from freespace0, freespace1, and freespace2 to the vehicle is defined as R 0 , R 1 , R 2 If R 2 > R1, and the pixel coordinates of freespace2 and freespace1 are similar (less than the set threshold), then freespace2 is determined as the final output of the freespace point. Otherwise, according to the freespace0 (x 0 ,y 0 ), freespace1(x 1 ,y 1 ) and freespace2(x 2 ,y 2 ) three points, and select the point closest to the vehicle as the freespace point finally output on this freespace confirmation line.

[0086] A drivable area correction method based on target detection is provided to solve the problem that the relative position relationship between the edge points of the drivable area and the target box in the vehicle coordinate system is inaccurate. Figure 6 : is a schematic diagram of a boundary point correction based on target detection according to an embodiment of the present invention, comprising the following steps:

[0087] Step 1 - Calculation of drivable area:

[0088] Based on the camera's original image information and image segmentation (parsing) information, calculate the boundary point information in the drivable area (freespace) and send it to step 3-freespace point position correction module. It mainly includes the following contents:

[0089] First, the image captured by the camera is input into the image segmentation model, which detects and outputs the image segmentation map, which represents the type of each pixel in the image (such as vehicle, pedestrian, cone, static target, background, etc.).

[0090] Secondly, the connected domains belonging to the freespace are screened out from all the connected domains in the image segmentation graph (parsing graph), and then the freespace connected domains are scanned row by row from bottom to top to determine the boundary points and boundary point types of the freespace.

[0091] Finally, the freespace boundary points are converted from the pixel coordinate system (in pixels) to the vehicle coordinate system (in meters) and output to the step 3-freespace point position correction module.

[0092] Step 2-Calculation of target location and attribute information:

[0093] According to the original camera image information and the target 2D detection frame information, the position and attribute information of the target in the vehicle coordinate system are calculated and sent to step 3-freespace point position correction module. It mainly includes the following contents:

[0094] First, the image captured by the camera is input into the image detection frame model, which detects the target 2D detection frame, target type and confidence. The target type includes vehicle, pedestrian or cone. If it is a vehicle, the full vehicle frame, rear frame and wheel frame will be detected at the same time. Pedestrians or cones will only have one detection frame.

[0095] Secondly, the IOU matching is performed based on the target detection box of the previous frame and the detection box of the current frame, and the detection box tracking of the previous and next two frames is performed based on the feature map vector of the detection box output by the image detection model.

[0096] Then, the extrinsic distance measurement of the target frame in the vehicle coordinate system is calculated based on the extrinsic distance measurement principle. The length, width and height of the target are estimated based on the pinhole imaging. The length, width and height of the target are then corrected based on the target type. If it is a vehicle, the heading angle of the vehicle can also be calculated based on the full vehicle frame, the rear frame or the wheel frame.

[0097] Finally, the target calculates the position of the target in the vehicle coordinate system based on the width and height of the target according to the camera intrinsic parameter ranging principle. This module will output the center point position of the target in the vehicle coordinate system. The vehicle target will also output the length, width, and heading angle of the target.

[0098] Step 3 - Freespace point position correction:

[0099] According to the free space points generated in step 1, the target frame and target attributes generated in step 2, the corrected free space point calculation is output. The final optimized effect is as follows Figure 3 As shown, the present invention successfully solves the technical problem of inaccurate relative position relationship between edge points of the drivable area and the target box in the vehicle coordinate system.

[0100] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0101] Example 2

[0102] In this embodiment, a device for correcting a drivable area is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0103] Figure 7 is a structural block diagram of a device for correcting a drivable area according to an embodiment of the present invention. Figure 7 As shown, the device comprises:

[0104] An acquisition module 70 is used to acquire an original image captured by a vehicle camera;

[0105] A calculation module 72, used to calculate the initial boundary point information of the drivable area of ​​the vehicle according to the original image, and calculate the position information of each target in the original image;

[0106] The correction module 74 is used to correct the initial boundary point information according to the position information to obtain the target boundary point information of the drivable area of ​​the vehicle.

[0107] Optionally, the calculation module includes: a segmentation unit, used to divide the original image into multiple segmentation maps, wherein each segmentation map corresponds to a target; a merging unit, used to merge at least two segmentation maps belonging to the same object category in the multiple segmentation maps into a connected domain, so as to obtain multiple connected domains; a screening unit, used to screen out the regional connected domain belonging to the drivable area of ​​the vehicle from the multiple connected domains; a determination unit, used to scan the regional connected domain row by row or column by column, determine the boundary coordinates of the initial boundary points of the drivable area, and convert the boundary coordinates from the pixel coordinate system into a vehicle coordinate system with the vehicle coordinate system as the origin.

[0108] Optionally, the screening unit includes: a screening subunit, used to screen out road connected domains and open-space connected domains from the multiple connected domains; and a determination subunit, used to determine the road connected domains and the open-space connected domains as regional connected domains belonging to the drivable area of ​​the vehicle.

[0109] Optionally, the calculation module includes: a processing unit, used to input the original image into a target detection model and output detection frames of all targets; an estimation unit, used to estimate, for each detection frame, the first coordinate information and first size information of the target in the vehicle coordinate system according to the camera parameters of the camera; a correction unit, used to correct the first size information to obtain second size information; and a calculation unit, used to calculate the second coordinate information of each target in the vehicle coordinate system according to the camera parameters and the second size information.

[0110] Optionally, the correction unit includes: an identification subunit, used to identify the target category of the target; an acquisition subunit, used to acquire a reference size range corresponding to the target category; a judgment subunit, used to judge whether the first size information is within the reference size range; and a correction subunit, used to correct the first size information based on the reference size range to obtain second size information if the first size information is not within the reference size range.

[0111] Optionally, the correction module includes: a determination unit, used to determine the initial value of the drivable area, wherein the initial value includes the number of boundary points and the area size; a generation unit, used to generate a boundary point confirmation line based on the initial value and the position information, wherein the boundary point confirmation line is a ray extending outward from the vehicle as the starting point in the area size; a correction unit, used to correct the initial boundary point information based on the boundary point confirmation line to obtain the target boundary point information of the drivable area of ​​the vehicle.

[0112] Optionally, the generation unit includes: a determination subunit, used to determine the centerline point coordinates of the first target, wherein the first target is a target whose target size is smaller than a preset threshold, and the position information includes the centerline point coordinates and the target size; a first generation subunit, used to take the origin of the vehicle coordinate system as the starting point, extend toward the centerline point coordinates of each first target, and generate a first number of first boundary point confirmation lines; a second generation subunit, used to take the origin of the vehicle coordinate system as the starting point, extend outward a second number of rays of the same angle, and generate a second number of second boundary point confirmation lines, wherein the sum of the first number and the second number is the number of boundary points.

[0113] Optionally, the correction unit includes: a processing subunit, used to initialize the first intersection between the boundary point confirmation line and the box where the area size is located as a first boundary point; a first generation subunit, used to generate an initial line segment using any two adjacent initial boundary points in the initial boundary point information; a second generation subunit, used to generate a second boundary point using the second intersection between the initial line segment and any of the boundary point confirmation lines; a third generation subunit, used to generate a third boundary point based on the position information and the target category; and a determination subunit, used to determine the target boundary point based on the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line.

[0114] Optionally, the third generating subunit is also used to: identify the target category of the target, wherein the target category includes: non-vehicle category, vehicle category; if the target category is a non-vehicle category, determine the centerline point as the third boundary point, wherein the position information includes the centerline point; if the target category is a vehicle category, calculate the four vertices of the target according to the center point, length, width, and heading angle of the target in the vehicle coordinate system, calculate the third intersection points between the four vertices and the boundary point confirmation line respectively, and select the third intersection point closest to the vehicle among the four third intersection points as the third boundary point.

[0115] Optionally, the determination subunit is also used to: for each boundary point confirmation line, calculate the first distance, the second distance, and the third distance between the first boundary point, the second boundary point, the third boundary point on the line and the vehicle respectively; determine whether the third distance is greater than the second distance, and whether the fourth distance between the second boundary point and the third boundary point is less than a preset threshold; if the third distance is greater than the second distance, and the fourth distance between the second boundary point and the third boundary point is less than a preset threshold, determine the third boundary point as the target boundary point; if the third distance is less than or equal to the second distance, or the fourth distance between the second boundary point and the third boundary point is greater than or equal to the preset threshold, determine the boundary point with the smallest of the first distance, the second distance, and the third distance as the target boundary point.

[0116] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0117] Example 3

[0118] An embodiment of the present invention further provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0119] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0120] S1, obtaining the original image captured by the vehicle camera;

[0121] S2, calculating the initial boundary point information of the drivable area of ​​the vehicle according to the original image, and calculating the position information of each target in the original image;

[0122] S3, modifying the initial boundary point information according to the position information to obtain target boundary point information of the drivable area of ​​the vehicle.

[0123] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0124] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0125] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0126] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0127] S1, obtaining the original image captured by the vehicle camera;

[0128] S2, calculating the initial boundary point information of the drivable area of ​​the vehicle according to the original image, and calculating the position information of each target in the original image;

[0129] S3, modifying the initial boundary point information according to the position information to obtain target boundary point information of the drivable area of ​​the vehicle.

[0130] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0131] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0133] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0134] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for correcting a drivable area, characterized in that: include: Get the original image collected by the vehicle camera; Calculating initial boundary point information of a drivable area of ​​the vehicle according to the original image, and calculating position information of each target in the original image; The initial boundary point information is corrected according to the position information to obtain the target boundary point information of the drivable area of ​​the vehicle.

2. The method according to claim 1, characterized in that Calculating the initial boundary point information of the drivable area of ​​the vehicle according to the original image includes: Dividing the original image into multiple segmentation maps, wherein each segmentation map corresponds to an object; Merging at least two segmentation maps belonging to the same object category in the multiple segmentation maps into one connected domain to obtain multiple connected domains; Screening out a regional connected domain belonging to a drivable area of ​​the vehicle from the multiple connected domains; The connected domain of the region is scanned row by row or column by column to determine the boundary coordinates of the initial boundary points of the drivable area, and the boundary coordinates are converted from a pixel coordinate system to a vehicle coordinate system with the vehicle coordinate system as the origin.

3. The method according to claim 2, characterized in that Screening out a regional connected domain belonging to the drivable area of ​​the vehicle from the multiple connected domains includes: Screening out a road connected domain and an open space connected domain from the multiple connected domains; The road connected domain and the open space connected domain are determined as regional connected domains belonging to a drivable area of ​​the vehicle.

4. The method according to claim 1, characterized in that Calculating the position information of each target in the original image includes: Input the original image into the target detection model and output the detection boxes of all targets; For each detection frame, estimating first coordinate information and first size information of the target in the vehicle coordinate system according to the camera parameters of the camera; Correcting the first size information to obtain second size information; The second coordinate information of each target in the vehicle coordinate system is calculated according to the camera parameters and the second size information.

5. The method according to claim 4, characterized in that Correcting the first size information to obtain the second size information includes: identifying a target category of said target; Obtaining a reference size range corresponding to the target category; Determine whether the first size information is within the reference size range; If the first size information is not within the reference size range, the first size information is corrected based on the reference size range to obtain second size information.

6. The method according to claim 1, characterized in that The initial boundary point information is corrected according to the position information to obtain the target boundary point information of the drivable area of ​​the vehicle, including: Determining an initial value of the drivable area, wherein the initial value includes the number of boundary points and the area size; Generate a boundary point confirmation line according to the initial value and the position information, wherein the boundary point confirmation line is a ray extending outward from the vehicle as the starting point in the area size; The initial boundary point information is corrected based on the boundary point confirmation line to obtain target boundary point information of the drivable area of ​​the vehicle.

7. The method according to claim 6, characterized in that Generating a boundary point confirmation line according to the initial value and the position information includes: Determine the coordinates of the center point of a first target, wherein the first target is a target whose target size is smaller than a preset threshold, and the position information includes the coordinates of the center point and the target size; Starting from the origin of the vehicle coordinate system, extending toward the midline point coordinates of each first target to generate a first number of first boundary point confirmation lines; Taking the origin of the vehicle coordinate system as the starting point, a second number of rays with the same angle are extended outward to generate a second number of second boundary point confirmation lines, wherein the sum of the first number and the second number is the number of boundary points.

8. The method according to claim 6, characterized in that Correcting the initial boundary point information based on the boundary point confirmation line includes: Initializing a first intersection point between the boundary point confirmation line and the frame where the area size is located as a first boundary point; Using any two adjacent initial boundary points in the initial boundary point information to generate an initial line segment; Generate a second boundary point using a second intersection point between the initial line segment and any of the boundary point confirmation lines; generating a third boundary point according to the position information and the target category; A target boundary point is determined according to the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line.

9. The method according to claim 8, characterized in that Generating a third boundary point according to the position information and the target category includes: Identify the target category of the target, wherein the target category includes: non-vehicle category and vehicle category; If the target category is a non-vehicle category, the centerline point is determined as the third boundary point, wherein the position information includes the centerline point; if the target category is a vehicle category, the four vertices of the target are calculated according to the center point, length, width, and heading angle of the target in the vehicle coordinate system, and the third intersection points between the four vertices and the boundary point confirmation line are calculated respectively, and the third intersection point closest to the vehicle is selected from the four third intersection points as the third boundary point.

10. The method according to claim 8, characterized in that Determining a target boundary point according to the first boundary point, the second boundary point, and the third boundary point on each boundary point confirmation line includes: For each boundary point confirmation line, respectively calculating a first distance, a second distance, and a third distance between the first boundary point, the second boundary point, and the third boundary point on the line and the vehicle; Determine whether the third distance is greater than the second distance, and whether a fourth distance between the second boundary point and the third boundary point is less than a preset threshold; If the third distance is greater than the second distance, and the fourth distance between the second boundary point and the third boundary point is less than a preset threshold, the third boundary point is determined as the target boundary point; if the third distance is less than or equal to the second distance, or the fourth distance between the second boundary point and the third boundary point is greater than or equal to the preset threshold, the smallest boundary point among the first distance, the second distance, and the third distance is determined as the target boundary point.

11. A device for correcting a drivable area, characterized in that: include: An acquisition module is used to acquire the original image collected by the vehicle camera; A calculation module, used to calculate the initial boundary point information of the drivable area of ​​the vehicle according to the original image, and calculate the position information of each target in the original image; The correction module is used to correct the initial boundary point information according to the position information to obtain the target boundary point information of the drivable area of ​​the vehicle.

12. The device according to claim 11, characterized in that The correction module comprises: A determination unit, configured to determine an initial value of the drivable area, wherein the initial value includes the number of boundary points and the area size; A generating unit, configured to generate a boundary point confirmation line according to the initial value and the position information, wherein the boundary point confirmation line is a ray extending outward from the vehicle as a starting point in the area size; The correction unit is used to correct the initial boundary point information based on the boundary point confirmation line to obtain the target boundary point information of the drivable area of ​​the vehicle.

13. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 10 when executed.

14. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 10.

Images