An image processing method and apparatus

Abstract

The present application discloses an image processing method and apparatus for accurately identifying a road operation area, thereby improving road traffic safety and reducing the occurrence of traffic accidents. The method provided by the present application includes: acquiring an image collected by an image acquisition device for a road, and identifying and tracking a moving target and a road operation target according to the image to obtain a target tracking result; wherein, the road operation target includes a stationary target; when the target tracking result includes information of a road operation target and the road operation target meets a preset condition, identifying the road operation area according to the target tracking result.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and particularly to an image processing method and device. Background Art

[0002] With the rapid development of domestic road traffic construction, the total mileage of domestic roads is nearly 5 million kilometers, of which the total mileage of expressways also exceeds 1.4 million kilometers. A huge amount of personnel input is required for daily road maintenance and road traffic management. Accidents caused by road operations (surveying, construction, maintenance) are particularly prominent. Timely detection of road operations and supervision are particularly important. Standardizing road operations and timely synchronizing them to vehicle drivers can greatly improve the safety of road operations and traffic safety. Precise control of road operations plays an important role in road traffic management.

[0003] With the development of computer vision technology based on deep learning and machine learning, computer vision technology has rapidly developed in the field of traffic monitoring. The advantages of road operation recognition based on computer vision are obvious. It can not only reduce costs, but also monitor and supervise road operations in real time. For example, whether the operation duration, operation lane, and operation area will affect normal driving, and timely notify the operators of irregular behaviors in the operation area and correct them in time to reduce the occurrence of traffic accidents.

[0004] However, as shown in the road operation scenario Figure 1 Due to the lack and non-standard layout of road operation warning markers, there are no obvious road operation signs, and only road cones can be used as auxiliary signs for identifying road operation areas. This current situation makes it difficult to identify road operation areas and unable to accurately identify the road construction area ahead. Summary of the Invention

[0005] Embodiments of this application provide an image processing method and device to accurately identify road operation areas, thereby improving road traffic safety and reducing the occurrence of traffic accidents.

[0006] An image processing method provided by an embodiment of this application includes:

[0007] Obtain the image collected by the image acquisition device for the road, and identify and track moving targets and road operation targets based on the image to obtain the target tracking result; wherein, the road operation targets include stationary targets;

[0008] When the target tracking result includes information of road operation targets and the road operation targets meet the preset conditions, identify the road operation area based on the target tracking result.

[0009] Through this method, an image collected by an image acquisition device for a road is obtained, and moving targets and road operation targets are identified and tracked based on the image to obtain a target tracking result; wherein, the road operation targets include stationary targets; when the target tracking result includes information on road operation targets and the road operation targets meet preset conditions, road operation area identification is performed based on the stationary targets in the target tracking result, so that accurate identification of the road operation area is achieved through the target tracking result including stationary targets, road traffic safety is improved, and the occurrence of traffic accidents is reduced.

[0010] In some embodiments, performing road operation area identification based on the target tracking result specifically includes:

[0011] Grid the image;

[0012] According to the target tracking result, binarize the values in the grid; wherein, set the grid where the moving target is located to a first value; set the grid where the stationary target is located to a second value;

[0013] Determine the road operation area according to the values in the grid.

[0014] Thus, the determination of the road operation area can be made more convenient, accurate, and detailed.

[0015] In some embodiments, determining the road operation area according to the values in the grid specifically includes:

[0016] Update the values in the grid according to the subsequent multi-frame image target tracking results of the image;

[0017] Determine the road operation area using the updated values in the grid.

[0018] Thus, the values in the grid can be updated in real time, and further, the determination of the road operation area can be made more accurate, detailed, and in line with the actual situation.

[0019] In some embodiments, determining the road operation area using the updated values in the grid specifically includes:

[0020] According to the updated values in the grid, determine the connected components and number the connected components; wherein, the values of the grids in any connected component are all the second value;

[0021] Use the connected components that meet the preset conditions to determine the road operation area.

[0022] Thus, screening of the connected components can be achieved, removing grid areas that are less likely to be road operation areas, and further improving the accuracy of determining the road operation area.

[0023] In some embodiments, a road operation area is determined by using a connected region that meets a preset condition. Specifically, it includes:

[0024] Map the grid coordinates within the connected region that meets the preset condition to the image to obtain the coordinates of the region on the image corresponding to this connected region;

[0025] According to the coordinates of the region, determine the polygonal region on the image, and use this polygonal region as the road operation area. The polygonal region covers the region on the image corresponding to this connected region.

[0026] Thus, a complete polygonal region can be determined as the road operation area, and this road operation area covers all possible road operation targets, thereby further improving the accuracy of determining the road operation area.

[0027] In some embodiments, in order to further improve the accuracy of determining the road operation area, the method further includes:

[0028] Use the lane lines in the image to correct the road operation area.

[0029] In some embodiments, using the lane lines in the image to correct the road operation area specifically includes:

[0030] According to the position information of the road operation marker in the target tracking result and the marked position of the lane line, determine the lane where the road operation marker is located and its adjacent lanes;

[0031] Merge the regions where the lane where the road operation marker is located and its adjacent lanes are located to form a candidate operation lane area;

[0032] Use the region jointly covered by the road operation area and the candidate operation lane area as the corrected road operation area.

[0033] Another embodiment of the present application provides an image processing device, which includes a memory and a processor. Among them, the memory is used to store program instructions, and the processor is used to call the program instructions stored in the memory and execute any of the above methods according to the obtained program.

[0034] In addition, according to an embodiment, for example, a computer program product for a computer is provided, which includes software code portions that, when the product runs on the computer, are used to execute the steps of the method defined above. The computer program product may include a computer-readable medium on which the software code portions are stored. In addition, the computer program product may be directly loaded into the internal memory of the computer and / or sent via at least one of an upload process, a download process, and a push process through a network.

[0035] Another embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions for causing the computer to execute any of the above methods. Description of the Drawings

[0036] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0037] Figure 1 It is a schematic diagram of a road operation scenario;

[0038] Figure 2 It is a schematic flowchart of an image processing method provided by an embodiment of the present application;

[0039] Figure 3 It is a schematic diagram of the overall process for identifying a road operation area provided by an embodiment of the present application;

[0040] Figure 4 It is a schematic flowchart of the process for extracting a road operation area provided by an embodiment of the present application;

[0041] Figure 5a It is a schematic diagram of grid initialization provided by an embodiment of the present application;

[0042] Figure 5b It is a schematic diagram of updating the setting of the grid where the center point of a road operation marker is located provided by an embodiment of the present application;

[0043] Figure 5c It is a schematic diagram of updating the setting of the grid where the center point of a moving vehicle is located provided by an embodiment of the present application;

[0044] Figure 6 It is a schematic diagram of the principle for determining a connected domain provided by an embodiment of the present application;

[0045] Figure 7Schematic diagram of a polygon area obtained by using a convex hull solving algorithm provided by an embodiment of the present application;

[0046] Figure 8 Schematic diagram of the process for correcting the road operation area provided by an embodiment of the present application;

[0047] Figure 9 Schematic diagram of the final effect display provided by an embodiment of the present application;

[0048] Figure 10 Schematic diagram of the structure of an image processing device provided by an embodiment of the present application;

[0049] Figure 11 Schematic diagram of the structure of another image processing device provided by an embodiment of the present application. Detailed implementation manners

[0050] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0051] The embodiments of the present application provide an image processing method and device to achieve accurate identification of the road operation area, thereby improving road traffic safety and reducing the occurrence of traffic accidents.

[0052] Among them, the method and the device are based on the same inventive concept. Since the principles for the method and the device to solve problems are similar, the implementation of the device and the method can be referred to each other, and the repeated parts will not be elaborated.

[0053] The terms "first", "second", etc. (if any) in the description and claims of the embodiments of the present application and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments described here can be implemented in an order different from that shown or described here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0054] The following examples and embodiments are to be understood as illustrative examples only. Although the present specification may refer to "one", "a", or "some" examples or embodiments in several places, this does not mean that each such reference relates to the same example or embodiment, nor does it mean that the feature applies only to a single example or embodiment. The individual features of different embodiments can also be combined to provide other embodiments. In addition, terms such as "including" and "comprising" should be understood not to limit the described embodiments to only those features that have been mentioned; such examples and embodiments can also include features, structures, units, modules, etc. that have not been specifically mentioned.

[0055] The following describes each embodiment of the present application in detail with reference to the accompanying drawings of the specification. It should be noted that the display order of the embodiments of the present application only represents the sequence of the embodiments, and does not represent the superiority or inferiority of the technical solutions provided by the embodiments.

[0056] In the embodiments of the present application, the road operation area is determined through background segmentation and road operation target recognition, and the road operation area is refined with the assistance of lane lines.

[0057] See Figure 2 , an image processing method provided by the embodiments of the present application includes:

[0058] S001. Obtain an image collected by an image acquisition device for a road, and perform recognition and tracking of moving targets and road operation targets based on the image to obtain a target tracking result; wherein, the road operation targets include stationary targets;

[0059] Among them, the moving targets in the road-collected image, such as non-road operation targets, such as a normally moving vehicle, etc.

[0060] That is to say, in the embodiments of the present application, the road operation targets in the road-collected image can be tracked, as well as non-road operation targets (such as a normally moving vehicle, etc.).

[0061] The stationary targets included in the road operation targets, such as road operation markers (warning signs, road cones, road operation fences), of course, the road operation targets can also include moving targets, such as operators and vehicles.

[0062] S002. When the target tracking result includes information on road operation targets and the road operation targets meet a preset condition, perform road operation area recognition based on the target tracking result.

[0063] The information on the road operation targets, for example, includes the coordinate position, target category, confidence level, etc. of the road operation targets;

[0064] The road operation target meets a preset condition, for example, the number of road operation targets exceeds a preset threshold.

[0065] In some embodiments, road operation area recognition is performed according to the target tracking result, which specifically includes:

[0066] Grid the image;

[0067] According to the target tracking result, binarize the values in the grid; wherein, the grid where the moving target is located is set to a first value; the grid where the stationary target is located is set to a second value;

[0068] Determine the road operation area according to the values in the grid.

[0069] Among them, the grid where the moving target is located, for example, the grid corresponding to the center point position coordinates of the tracking frame of the moving target;

[0070] The first value is, for example, 1; the second value is, for example, 0.

[0071] In some embodiments, in addition to setting the grid where the stationary target is located to the second value, further, the adjacent grids (if there are corresponding grids) above, below, left, and right of the grid where the stationary target is located can be set to the second value.

[0072] In some embodiments, determining the road operation area according to the values in the grid specifically includes:

[0073] Update the values in the grid according to the target tracking results of subsequent multiple frames of the image (which can be continuous or discontinuous);

[0074] Use the updated values in the grid to determine the road operation area.

[0075] Of course, in the embodiments of the present application, the values in the grid may not be updated, and the road operation area may be determined only using one frame of image.

[0076] In some embodiments, using the updated values in the grid to determine the road operation area specifically includes:

[0077] According to the updated values in the grid, determine the connected regions and number the connected regions; wherein, the values of the grids in any connected region are all the second value;

[0078] Use the connected regions that meet the preset conditions to determine the road operation area.

[0079] The connected regions that meet the preset conditions, for example, the number of grids in the connected region exceeds a preset threshold.

[0080] In some embodiments, a road operation area is determined by using a connected component that meets a preset condition. Specifically, it includes:

[0081] Map the grid coordinates within the connected component that meets the preset condition onto the image to obtain the coordinates of the area on the image corresponding to this connected component;

[0082] Determine a polygonal area on the image according to the coordinates of the area, and use this polygonal area as the road operation area, where the polygonal area covers the area on the image corresponding to this connected component.

[0083] For example, a connected component edge solving algorithm: convex hull solving algorithm can be adopted to determine the polygonal area on the image according to the coordinates of the area.

[0084] In some embodiments, the method further includes:

[0085] Use the lane lines in the image to correct the road operation area.

[0086] Among them, the lane lines can be manually marked or recognized by an algorithm.

[0087] In some embodiments, using the lane lines in the image to correct the road operation area specifically includes:

[0088] According to the position information of the road operation markers in the target tracking result and the marked positions of the lane lines, determine the lane where the road operation markers are located and its adjacent lanes;

[0089] Merge the areas where the lane where the road operation markers are located and its adjacent lanes are located to form a candidate operation lane area;

[0090] Take the intersection of the road operation area and the candidate operation lane area, that is, the area jointly covered by the road operation area and the candidate operation lane area, as the corrected road operation area.

[0091] In summary, the embodiments of the present application propose a road operation area recognition solution. Some specific embodiments are described below.

[0092] Refer to Figure 3 As shown, it is the specific process of road operation area recognition. The specific implementation steps are as follows:

[0093] Step S101, obtain at least one frame of road acquisition image.

[0094] For example, the video stream in a highway scene is obtained in this step, that is, it includes multiple frames of images, and each frame of image is processed in the following steps.

[0095] In some embodiments, this step may further include determining detailed information of the image, such as frame rate and resolution.

[0096] For each frame of the image:

[0097] Step S102 (optional step): Draw lane lines on the image according to the road scene on the image.

[0098] Among them, there are two ways to draw the lane lines: one is manual drawing, and the other is to identify the lane lines in the image according to the lane line recognition algorithm (prior art).

[0099] Step S103: Perform target tracking on the image using a pre-set tracking algorithm (tracker, prior art) to obtain the target tracking result.

[0100] In some embodiments, the targets include road pedestrians, vehicles, and road operation targets;

[0101] In some embodiments, the road operation targets mainly include road operation vehicles, operators wearing road operation uniforms, road operation markers (warning signs, road cones, road operation fences), etc. Among them, road operation vehicles and operators wearing road operation uniforms can be regarded as moving targets, and road operation markers (warning signs, road cones, road operation fences) can be regarded as stationary targets.

[0102] In some embodiments, the target tracking result mainly includes basic information such as the coordinate information of the target on the image, the target category, and the confidence level of the target;

[0103] In some embodiments, the center point coordinates of the tracking box (i.e., the circumscribed rectangle) of each target can be used as the coordinates of the target.

[0104] In some embodiments, the target category, for example: the target is in categories such as pedestrians, vehicles, and road operation targets.

[0105] In some embodiments, the confidence level of the target, for example, is the confidence level used to further determine whether the identified target is a real target. For example, if the confidence level of the target is greater than a preset threshold, it is considered that the target is correctly identified and is a real target; otherwise, it is considered that the identification is incorrect.

[0106] Step S104: Determine whether the number of road operation targets in the target tracking result of the image is greater than a preset threshold. If not, end the process; if so, execute step S105.

[0107] Step S105: Perform road operation area recognition.

[0108] It should be noted that in the embodiments of the present application, one frame of image can be used for road operation area recognition, or multiple frames of images can be used for road operation area recognition.

[0109] When using one frame of image for road operation area recognition, that is, when the number of road operation targets in the object tracking result of the image is greater than a preset threshold, directly use the object tracking result of the image for road operation area recognition.

[0110] In some embodiments, when using multiple frames of images for road operation area recognition, for example, when the number of road operation targets in the object tracking result of the current frame image is greater than a preset threshold, use the object tracking results of the current frame image and subsequent consecutive multiple frames of images for road operation area recognition. At this time, for example, it specifically includes:

[0111] First, grid the current frame image, and according to the moving vehicle detection frame in the object tracking result of the image, mark the moving foreground in the image with grids, and then segment out the static background, and the static background can be initially considered as a candidate area for the road operation area;

[0112] Then, combined with the information of road operation targets in subsequent consecutive multiple frames of images of the image, update the candidate area of the road operation area in real time;

[0113] Finally, the road operation area can be recognized from the updated candidate area through an edge solution algorithm, for example, a convex hull solution algorithm.

[0114] Step S106 (optional step), according to the lane lines drawn in the image in step S102, correct the road operation area determined in step S105.

[0115] Among them, in step S105, if multiple frames of images (all these images meet the judgment conditions of step S104) are used for road operation area recognition, then the determination process of the road operation area is as Figure 4 shown, specifically as follows:

[0116] Step S201, grid the current frame image.

[0117] For example, divide the image into M*N grids, M and N can be set as fixed constants, or can be dynamically adjusted according to the change of the image resolution (such as 1920*1080). For example, the values of M and N used for the current frame image and the values of M and N used for subsequent images can be different.

[0118] Each grid should not be too large or too small. If it is too large, it will affect the accuracy. If the grid is divided too small, the calculation amount will increase and the running speed of the algorithm will be affected. The specific values of M and N can be determined according to actual needs and are not limited in this application.

[0119] Step S202: Perform binary initialization on the values within the grid.

[0120] For the first frame image among the multiple frame images used for road operation area recognition, all grids are initialized once. For example, if 0 is used for grid initialization, it is defaulted that the whole image is a static background. If 1 is used for grid initialization, it is defaulted that the whole image is a moving panorama. For example Figure 5a As shown, that is to say, the grid where the static target is located is represented by 0, namely the static background, and the grid where the moving target is located is represented by 1, namely the moving foreground.

[0121] For each subsequent frame image among the multiple frame images used for road operation area recognition, according to the newly recognized road operation marker in this image (in a static state, and the ID of the newly recognized road operation marker is different from that of the existing road operation markers), and taking the grid where the center point of the newly detected road operation marker is located, mark the positions of this grid and its adjacent grids (up, down, left, right) as 0. For example Figure 5b As shown.

[0122] Step S203: Update the values within the grid according to the target tracking result of each frame, that is, update the candidate area of the road operation area in real time.

[0123] For example, when the newly detected target is a moving vehicle, as Figure 5c shown, then the marker position of the grid where the center point of the tracking box of this target is located is set to 1. Thus, the candidate area of the road operation area is updated to Figure 5c the dotted box area shown, that is, the area composed of multiple consecutive grids with the marker position of 0.

[0124] Step S204: Solve the connected components.

[0125] Among them, the connected components are the candidate areas of the road operation area.

[0126] This step mainly determines the numbers and the number of the candidate areas of the road operation area by obtaining one or more independent grid areas.

[0127] For example, see Figure 6, starting from the first row of the grid, traverse the flag bits of the grid row by row. Taking the grid with a flag bit of 0 as the center, the grids adjacent to this grid (for example, considering the upper, lower, left, and right as adjacent) and with a flag bit of 0 are grouped into a cluster. This cluster constitutes a connected domain, and an identifier (or serial number) of this connected domain is given. For example, Figure 6 If the flag bits of the first two grids in the first row shown are 0, then these two grids form a connected domain, and this connected domain is identified as 1. Traverse and solve the connected domains formed by the grids with a flag bit of 0 in the grid, so as to finally determine each connected domain and its corresponding serial number identifier.

[0128] Specifically, as Figure 6 shown, Figure 6 In the left figure in, for example, it is the result of the grid flag bits generated according to the target tracking results of consecutive multi-frame images. As time changes, the position of the vehicle will also change, so the corresponding position in the grid is marked as 1. Figure 6 In the right figure in, there are two connected regions calculated by using the connected domain solving algorithm, and different connected domains are respectively identified by 1 and 2. Figure 6 In the right figure in, the numbers in the grid no longer represent moving targets and stationary targets, but represent different connected domains. As can be seen from Figure 6 the right figure in, at this time, the number in the grid where the moving target is located is set to 0, indicating that the connected domain calculation is not performed on this grid. Specifically: Traverse Figure 6 the M*N rectangular grid shown in the left figure in, and finally obtain Figure 6 the two connected domains shown by the dashed boxes in the right figure in, that is, the connected domain identified by 1 and the connected domain identified by 2.

[0129] Step S205 (optional step): Screen the candidate areas of the road operation area.

[0130] Specifically, traverse the connected domains obtained in step S204, that is, the serial numbers of the connected domains are incremented by one in turn. The connected domains with the number of grids in the connected domain > the threshold are finally regarded as the candidate areas of the road operation area, and step S206 is executed. Otherwise, judge whether the number of grids in the next connected domain is greater than the threshold.

[0131] Among them, the number of grids in the connected domain > the threshold indicates that this connected domain is more likely to be the road operation area, that is, if the connected domain is too small, it is not considered as the road operation area. For example, the threshold can be set to 3. Then, for the connected domain composed of two grids, it is discarded and not considered as the road operation area. That is, Figure 6 in, the connected domain 1 is to be discarded, and the connected domain 2 is continued as the candidate area of the road operation area.

[0132] Step S206: Map each grid coordinate (the four vertex coordinates of the grid box) in the candidate area of the road operation area onto the image, so as to obtain the candidate area coordinates of the road operation area in the image.

[0133] Step S207: Use the connected component edge solving algorithm to determine the polygonal area of the candidate area that completely contains the road operation area on the image according to the candidate area coordinates of the road operation area determined in Step S206.

[0134] For example, the convex hull solving algorithm can be used. The image coordinates corresponding to the four vertices of each grid in the candidate area of the road operation area are used as the input of the convex hull algorithm to solve the polygonal area. Specifically, regarding the convex hull algorithm, for example, given a set of points on a two-dimensional plane arbitrarily, the convex hull is a convex polygon formed by connecting the outermost points, which contains all the points in the point set. Usually, the Graham scan method is used to complete the convex hull solving, and the result of the solving is as Figure 7 shown. The outermost polygon covers the candidate area of the road operation area. It can be understood that the points in the polygonal area include the four vertices of each grid in the candidate area of the road operation area.

[0135] In some embodiments, the determined polygonal area above can be used as the finally determined road operation area.

[0136] In some embodiments, the determined polygonal area above can also be corrected. For example, Step S106 for correcting the road operation area according to the lane line above, and the specific process is as Figure 8 shown, for example, including:

[0137] Step S301: Determine the lane where the road operation marker is located and its adjacent lanes according to the tracking result (coordinate position information in the image) of the road operation marker target in the target tracking result and the marked position of the lane line (manually marked or automatically recognized by the algorithm).

[0138] For example, by judging whether the center point coordinate of the road operation marker target is within a certain lane area, determine the lane where the road operation marker is located, and then obtain the adjacent lanes through the lane position;

[0139] Step S302: Merge the lane where the road operation marker is located and its adjacent lane areas to form a candidate operation lane area;

[0140] Step S303: Traverse the road operation area obtained in step S105 (i.e., the determined polygon area above), and take the intersection with the candidate operation lane area determined in step S302 to obtain one or more corrected road operation area results (i.e., the intersection may have only one area or multiple areas), making the determination of the road operation area more refined.

[0141] Final effect display:

[0142] As Figure 9 shown, the rectangular box shown in the final effect is the finally determined road operation area. Since only unilateral road cones are placed on the road, the complete area cannot be obtained directly based on the road cones, while the present application can more accurately determine the complete road operation area.

[0143] Next, the equipment or device provided in the embodiments of the present application will be introduced. For the explanations or examples of the same or corresponding technical features as those described in the above method, they will not be repeated hereinafter.

[0144] See Figure 10 , an image processing device provided in an embodiment of the present application includes:

[0145] A memory 11 for storing program instructions;

[0146] A processor 12 for calling the program instructions stored in the memory and executing the method provided in the embodiments of the present application according to the obtained program. For the specific processing process, refer to the content in the above method part and will not be elaborated here.

[0147] For example, see Figure 11 , the image processing device provided in an embodiment of the present application may be a computing device on the terminal side. The computing device may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), etc. The computing device may include a processor 600, a memory 620, etc.

[0148] The memory 620 may include a read-only memory (ROM) and a random access memory (RAM), and provide the program instructions and data stored in the memory to the processor. In the embodiments of the present application, the memory may be used to store the program of any of the methods provided in the embodiments of the present application.

[0149] The processor 600 is used to execute any of the methods provided in the embodiments of the present application by calling the program instructions stored in the memory.

[0150] A transceiver 610 for receiving and sending data under the control of the processor 600.

[0151] Among them, in Figure 11 , the bus architecture may include any number of interconnected buses and bridges, specifically, various circuits represented by one or more processors represented by processor 600 and a memory represented by memory 620 are linked together. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art, and thus will not be further described herein. The bus interface provides an interface. The transceiver 610 may be a plurality of elements, that is, including a transmitter and a receiver, and provides a unit for communicating with various other devices on a transmission medium.

[0152] In some embodiments, a user interface 630 is further included. The user interface 630 may be an interface capable of externally or internally connecting to a required device, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, etc.

[0153] The processor 600 is responsible for managing the bus architecture and general processing, and the memory 620 may store data used by the processor 600 when performing operations.

[0154] In some embodiments, the processor 600 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a CPLD (Complex Programmable Logic Device).

[0155] The embodiments of the present application further provide a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes any of the methods described in the above embodiments. The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] An embodiment of the present application provides a computer-readable storage medium for storing computer program instructions used by the device provided in the embodiment of the present application above, which includes a program for executing any method provided in the embodiment of the present application above. The computer-readable storage medium may be a non-transitory computer-readable medium.

[0157] The computer-readable storage medium may be any available medium or data storage device accessible by a computer, including but not limited to magnetic memory (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical memory (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drives (SSD)), etc.

[0158] It should be understood that:

[0159] The access technology through which entities in a communication network transmit traffic to and from each other can be any suitable current or future technology, such as WLAN (Wireless Local Area Network), WiMAX (Worldwide Interoperability for Microwave Access), LTE, LTE-A, 5G, Bluetooth, infrared, etc.; additionally, the embodiment can also apply wired technologies, for example, IP-based access technologies, such as wired networks or fixed lines.

[0160] Embodiments suitable for being implemented as software code or a part thereof and running using a processor or processing function are independent of the software code and can be specified using any known or future-developed programming language, such as high-level programming languages, such as objective-C, C, C++, C#, Java, Python, Javascript, other scripting languages, etc., or low-level programming languages, such as machine language or assembly programs.

[0161] The implementation of the embodiment is independent of the hardware and can be implemented using any known or future-developed hardware technology or any combination thereof, such as microprocessors or CPUs (Central Processing Units), MOS (Metal Oxide Semiconductor), CMOS (Complementary MOS), BiMOS (Bipolar MOS), BiCMOS (Bipolar CMOS), ECL (Emitter Coupled Logic), and / or TTL (Transistor-Transistor Logic).

[0162] Embodiments can be implemented as a separate device, apparatus, unit, component or function, or in a distributed manner. For example, one or more processors or processing functions can be used or shared in a process, or one or more processing segments or processing parts can be used and shared in a process, where one physical processor or more than one physical processor can be used to implement one or more processing parts dedicated to a specific process as described.

[0163] The apparatus can be implemented by a semiconductor chip, a chipset, or a (hardware) module including such a chip or chipset.

[0164] Embodiments can also be implemented as any combination of hardware and software, such as ASIC (Application Specific IC (Integrated Circuit)) components, FPGA (Field Programmable Gate Array) or CPLD (Complex Programmable Logic Device) components, or DSP (Digital Signal Processor) components.

[0165] Embodiments can also be implemented as a computer program product, including a computer-usable medium having embodied therein computer-readable program code adapted to perform the processes as described in the embodiments, where the computer-usable medium can be a non-transitory medium.

[0166] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories and optical memories, etc.) containing computer-usable program code.

[0167] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0168] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in the function.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the processes and / or blocks Figure 1 one or more of the processes and / or blocks Figure 1 specified in the function.

[0170] It is apparent that those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to cover these modifications and variations.

Claims

1. An image processing method, characterized in that, comprising: Obtain the current frame image collected by the image acquisition device for the road, and obtain the target tracking result of the current frame image according to the recognition and tracking of moving targets and road operation targets in the current frame image; wherein, the road operation targets include stationary targets; When the target tracking result of the current frame image includes information on road operation targets and the road operation targets meet the preset conditions, use the target tracking result of the current frame image and the target tracking results of subsequent multiple frames of the current frame image to perform road operation area recognition; Among them, using the target tracking result of the current frame image and the target tracking results of subsequent multiple frames of the current frame image to perform road operation area recognition includes: Grid the current frame image; According to the target tracking result of the current frame image, perform binary initialization on the values in the grid; wherein, set the grid where the moving target is located to the first value; set the grid where the stationary target is located to the second value; Determine the road operation area according to the values in the grid; The determining the road operation area according to the values in the grid includes: Update the values in the grid according to the target tracking results of subsequent multiple frames of the current frame image; Use the updated values in the grid to determine the road operation area.

2. The method according to claim 1, characterized in that, Using the updated values in the grid to determine the road operation area specifically includes: Determine the connected components according to the updated values in the grid and number the connected components; wherein, the values of the grids in any connected component are all the second value; Use the connected components that meet the preset conditions to determine the road operation area.

3. The method according to claim 2, characterized in that, Using the connected components that meet the preset conditions to determine the road operation area specifically includes: Map the grid coordinates in the connected components that meet the preset conditions to the current frame image to obtain the coordinates of the area on the current frame image corresponding to the connected component; According to the coordinates of the area, determine the polygonal area on the current frame image, and use this polygonal area as the road operation area, and the polygonal area covers the area on the current frame image corresponding to the connected component.

4. The method according to claim 3, characterized in that, This method further includes: Use the lane lines in the current frame image to correct the road operation area.

5. The method according to claim 4, characterized in that, Using the lane lines in the current frame image to correct the road operation area specifically includes: According to the position information of the road operation markers in the target tracking result and the marked positions of the lane lines, determine the lane where the road operation markers are located and its adjacent lanes; Merge the areas where the lane where the road operation markers are located and its adjacent lanes are located to form a candidate operation lane area; Use the area jointly covered by the road operation area and the candidate operation lane area as the corrected road operation area.

6. An image processing apparatus, characterized in that, comprising: a memory for storing program instructions; a processor for calling the program instructions stored in the memory and executing the method according to any one of claims 1 to 5 according to the obtained program.

7. A computer program product for a computer, characterized in that, comprising a software code portion that, when the product runs on the computer, is configured to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores computer-executable instructions for causing the computer to execute the method according to any one of claims 1 to 5.

Images