Line segment extraction method and device, storage medium, and electronic device

Abstract

An embodiment of the present invention provides a line segment extraction method and device, a storage medium, and an electronic device. The line segment extraction method includes: inputting a first target image acquired by a mobile robot into a visual detection model to obtain a target area corresponding to the first target image, wherein the target area includes a target line segment to be extracted; increasing the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by an image enhancement operator to obtain a first image; extracting the target line segment from the first image. The above technical solution is used to solve the problems in the related art that, for images with unclear line features, the line segment detection rate is low and the line segment extraction effect is not ideal.

Description

[Technical field]

[0001] The present invention relates to the field of communications, and in particular to a line segment extraction method and device, a storage medium, and an electronic device. [Background technology]

[0002] In the process of studying images, users are often only interested in certain parts of the image. Therefore, part of the image needs to be extracted. Lines are the basic features of two-dimensional objects. In the process of image extraction, line segments need to be extracted first. Line extraction is widely used in positioning, feature extraction, and object detection in image processing.

[0003] In the related line extraction technology, the acquired image is first filtered and denoised, and then the line extraction is performed using the Hough line extraction and LCD line extraction algorithms. However, for images with unclear line features, this method has a low line segment detection rate and an unsatisfactory line segment extraction effect.

[0004] In the related technology, for images with unclear line features, the line segment detection rate is low and the line segment extraction effect is not ideal, and no effective technical solution has been proposed.

[0005] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. [Summary of the invention]

[0006] The embodiments of the present invention provide a line segment extraction method and device, a storage medium, and an electronic device to at least solve the problems of low line segment detection rate and unsatisfactory line segment extraction effect for images with unclear line features.

[0007] According to one embodiment of the present invention, a method for extracting line segments is provided, comprising: inputting a first target image acquired by a mobile robot into a visual detection model to obtain a target area corresponding to the first target image, wherein the target area includes a target line segment to be extracted; increasing the contrast between a first pixel value of the target line segment in the target area and a second pixel value of a background of the target area by an image enhancement operator to obtain a first image; and extracting the target line segment from the first image.

[0008] In an exemplary embodiment, a first target image acquired by a mobile robot is input into a visual detection model to obtain a target area corresponding to the first target image, including: inputting the first target image acquired by the mobile robot into the visual detection model to obtain first target image information corresponding to the first target image, wherein the first target image information includes: a first image label, a first image confidence, the first image label is used to indicate the type of a line segment in the first target image, and the first image confidence is used to indicate the score of the line segment being the target line segment; determining whether a target area exists in the first target image according to the first image label and the first image confidence, and if the target area exists, obtaining the target area from the first target image.

[0009] In an exemplary embodiment, determining whether a target area exists in the first target image based on the image label and the image confidence includes: matching the first image label with the label of the target line segment to be extracted, and determining whether the first image confidence is greater than a preset threshold; when the first image label is the same as the label of the target line segment to be extracted and the first image confidence is greater than a preset threshold, determining that a target area exists in the first target image.

[0010] In an exemplary embodiment, determining whether a target area exists in the first target image based on the first image label and the first image confidence includes: if there is no target area in the first target image, acquiring a second target image by the mobile robot, and inputting the second target image into the visual detection model to obtain second image information corresponding to the second target image, wherein the second image information includes: a second image label, a second image confidence; determining whether a target area exists in the second target image based on the second image label and the second image confidence, and acquiring the target area from the second target image if the target area exists.

[0011] In an exemplary embodiment, increasing the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by using an image enhancement operator includes: acquiring all pixel values ​​of the target area; mapping the pixel values ​​by using the image enhancement operator to increase the contrast between the first pixel value and the second pixel value, wherein the image enhancement operator includes at least one of the following: Laplace operator, histogram equalization.

[0012] In an exemplary embodiment, extracting the target line segment from the first image includes: extracting the target line segment from the first image according to a line extraction algorithm, wherein the line extraction algorithm includes at least one of the following: Hough line extraction and LCD line extraction.

[0013] In an exemplary embodiment, after extracting the target line segment from the first image, the method further includes: acquiring a first position of the target line segment, wherein the first position is used to indicate the position of the target line segment in the first target image; determining a distance of the target line segment from the mobile robot based on the first position of the target line segment and an imaging principle; and determining a second position of the target line segment in the working area of ​​the mobile robot based on the distance.

[0014] According to another embodiment of the present invention, a line segment extraction device is provided, the device comprising: an input module, used to input a first target image acquired by a mobile robot into a visual detection model to obtain a target area corresponding to the first target image, wherein the target area includes a target line segment to be extracted; an adjustment module, used to increase the contrast between a first pixel value of the target line segment in the target area and a second pixel value of a background of the target area through an image enhancement operator to obtain a first image; and an extraction module, used to extract the target line segment from the first image.

[0015] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned line segment extraction method when running.

[0016] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0017] Through the present invention, a first target image acquired by a mobile robot is input into a visual detection model to obtain a target area corresponding to the first target image, wherein the target area includes a target line segment to be extracted; an image enhancement operator is used to increase the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area to obtain a first image; the target line segment is extracted from the first image, i.e., an image enhancement operator is used to increase the contrast between a first pixel value of the target line segment and a second pixel value of the background of the target area, and then the target line segment is extracted from the first image after adjusting the pixel value. The above technical solution is adopted to solve the problems in the related art that, for images with unclear line features, the line segment detection rate is low and the line segment extraction effect is not ideal, and then the target line segment can be effectively extracted.

[0018] The present invention has the following beneficial effects: using a visual detection model to obtain a target area on a first target image, thereby narrowing the search range of a target line segment; adjusting a first pixel value of a target line segment and a second pixel value of a background of the target area through an image enhancement operator, thereby increasing the contrast of the target area; performing line extraction on the adjusted first image, and effectively proposing a target line segment.

Brief Description of the Drawings

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 It is a hardware structure block diagram of a line segment extraction method according to an embodiment of the present invention;

[0021] Figure 2 is a flow chart of a method for extracting line segments according to an embodiment of the present invention (I);

[0022] Figure 3 is a flow chart of a method for extracting line segments according to an embodiment of the present invention (II);

[0023] Figure 4 is a schematic diagram showing the effect of the method for extracting line segments according to an embodiment of the present invention;

[0024] Figure 5 4 is a structural block diagram of a line segment extraction device according to an embodiment of the present invention. [Specific implementation method]

[0025] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a computer device or a similar computing device. Taking running on a computer device as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a computer device for a line segment extraction method according to an embodiment of the present invention. Figure 1 As shown, the computer device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. In an exemplary embodiment, the above-mentioned computer device may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer device. Figure 1 More or fewer components as shown, or with Figure 1 Equivalent functions or comparisons shown Figure 1 A different configuration with more features is shown.

[0028] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the line segment extraction method of the computer device in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer device via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0029] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a line segment extraction method is provided, which is applied in a computer device. Figure 2 : is a flow chart of a method for extracting line segments according to an embodiment of the present invention (I), the flow chart comprising the following steps:

[0031] Step S202: inputting the first target image acquired by the mobile robot into the visual detection model to acquire a target area corresponding to the first target image, wherein the target area includes the target line segment to be extracted;

[0032] It should be noted that when the mobile robot detects a target obstacle at a distance, the mobile robot acquires a first target image through a camera or other equipment on the mobile robot. Furthermore, the mobile robot can also acquire the first target image in real time through a camera or other equipment on the mobile robot. The embodiment of the present invention does not limit the device for acquiring the first target image and the prerequisites for acquiring the first target image.

[0033] Step S204: increasing the contrast between the first pixel value of the target line segment in the target area and the second pixel value of the background of the target area by using an image enhancement operator to obtain a first image;

[0034] Step S206: extracting the target line segment from the first image.

[0035] Through the embodiment of the present invention, the first target image acquired by the mobile robot is input into the visual detection model to obtain the target area corresponding to the first target image; the contrast between the first pixel value of the target line segment in the target area and the second pixel value of the background of the target area is increased by the image enhancement operator to obtain the first image; the target line segment is extracted from the first image, that is, the contrast between the first pixel value of the target line segment and the second pixel value of the background of the target area is increased by the image enhancement operator, and then the target line segment is extracted from the first image after the pixel value is adjusted. The above technical solution is adopted to solve the problems in the related art that for images with unclear line features, the line segment detection rate is not high and the line segment extraction effect is not ideal, and the target line segment can be effectively extracted.

[0036] There are many ways to implement the above-mentioned step S202. An embodiment of the present invention provides an implementation method, specifically: inputting the first target image acquired by the mobile robot into the visual detection model to obtain the first target image information corresponding to the first target image, wherein the first target image information includes: a first image label, a first image confidence, the first image label is used to indicate the type of the line segment in the first target image, and the first image confidence is used to indicate the score of the line segment being the target line segment; determining whether there is a target area in the first target image according to the first image label and the first image confidence, and if the target area exists, obtaining the target area from the first target image.

[0037] It can be understood that the first target image is input into the trained visual detection model to obtain the first image label and the first image confidence of the first target image. When the first image label is the same as the label of the target line segment to be extracted and the first image confidence is greater than a preset threshold, it can be determined that there is a target area in the first target image. It should be noted that the visual detection model is a deep learning detection model trained based on sample data. The deep learning detection model can be a YOLO series model or an RCN series model, etc. The samples of the visual detection model can be images collected on the network and images acquired by a mobile robot, and the embodiments of the present invention are not limited to this.

[0038] Determining whether a target area exists in the first target image according to the image label and the image confidence, the embodiment of the present invention mainly provides two implementation methods, which are as follows:

[0039] Method 1:

[0040] Step 1: Match the first image label with the label of the target line segment to be extracted;

[0041] Step 2: Determine whether the confidence of the first image is greater than a preset threshold.

[0042] When the first image label is the same as the label of the target line segment to be extracted and the first image confidence is greater than a preset threshold, it is determined that a target area exists in the first target image.

[0043] It can be understood that, first, it is determined based on the first image label whether it is the same as the label of the target line segment to be extracted. When the first image label is determined to be the same as the label of the target line segment to be extracted, it is judged whether the first image confidence is greater than a preset threshold. When it is determined that the first image confidence is greater than the preset threshold, there is a target area in the first target image. It should be noted that when the first image label is determined to be the same as the label of the target area, it can be understood that the object category corresponding to the first image is the same as the object category of the target area. The first image confidence can be understood as the credibility score of the object in the first image. For example, the first target image is input into the visual detection model, and the credibility of the object in the first target image being a door lintel is 90 points, which can also be understood as the possibility that the object in the first target image is a door lintel is 90%.

[0044] Method 2:

[0045] Step 3: Determine whether the confidence of the first image is greater than a preset threshold;

[0046] Step 4: Match the first image label with the label of the target line segment to be extracted.

[0047] When the first image label is the same as the label of the target line segment to be extracted and the first image confidence is greater than a preset threshold, it is determined that a target area exists in the first target image.

[0048] It can be understood that first, it is determined whether the confidence of the first image is greater than a preset threshold. When it is determined that the confidence of the first image is greater than the preset threshold, it is determined whether the first image label is the same as the label of the target line segment to be extracted. When the first image label is determined to be the same as the label of the target line segment to be extracted, it is determined whether the confidence of the first image is greater than the preset threshold, and there is a target area in the first target image. If the confidence of the first image is less than the preset threshold, and or, the first image label is different from the label of the target line segment to be extracted, it is directly determined that there is no target area in the first target image.

[0049] It can be seen from the above-mentioned method 1 and method 2 that it is possible to first determine whether the confidence of the first image is greater than a preset threshold; or to first match the first image label with the label of the target line segment to be extracted. The embodiments of the present invention do not limit the above-mentioned execution steps.

[0050] In an exemplary embodiment, when there is no target area in the first target image, a second target image is acquired by the mobile robot, and the second target image is input into the visual detection model to obtain second image information corresponding to the second target image, wherein the second image information includes: a second image label, a second image confidence; determining whether there is a target area in the second target image based on the second image label and the second image confidence, and acquiring the target area from the second target image if the target area exists.

[0051] In other words, when the confidence of the first image is less than a preset threshold, and or, when the first image label is different from the label of the target line segment to be extracted, that is, when there is no target area in the first target image, the mobile robot is instructed to obtain a second target image, wherein the second target image is an image corresponding to other areas different from the first target image, and the second target image is input into the trained visual detection model to obtain the second image label and the second image confidence of the second target image. When the second image label is the same as the label of the target line segment to be extracted and the confidence of the second image is greater than a preset threshold, it can be determined that there is a target area in the second target image. Otherwise, there is no target area in the second target image. When there is no target area in the second target image, it is necessary to continue to obtain a third target image, wherein the third target image is an image corresponding to other areas different from the first target image and the second image, until there is a target area in the target image.

[0052] It should be noted that, when the target area does not exist in the first target image, another image (ie, the second target image) is acquired, and the first target image and the second target image are two images acquired by the mobile robot.

[0053] In an exemplary embodiment, the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area is increased by an image enhancement operator. Specifically, all pixel values ​​of the target area are obtained; the pixel values ​​are mapped by the image enhancement operator to increase the contrast between the first pixel value and the second pixel value, wherein the image enhancement operator includes at least one of the following: Laplace operator, histogram equalization.

[0054] It should be noted that all pixel values ​​in the target area are mapped through the Laplace operator and histogram equalization to increase the contrast between the first pixel value and the second pixel value, so as to increase the contrast between the target line segment and the background. Furthermore, the contrast between the first pixel value and the second pixel value can be increased by multiplication, convolution, etc. Taking the door lintel as an example, the pixel value of the target area is remapped to between 0 and 255 through the image enhancement operator. In the second image after pixel adjustment, the pixel value contrast between the door lintel and the background is increased, thereby solving the problem of being unable to extract the target line segment due to the small contrast between the background and target line segment pixel values.

[0055] In an exemplary embodiment, the first image after adjusting the pixel values ​​is acquired; and the target line segment is extracted from the first image after adjusting the pixel values ​​by using a line extraction algorithm, wherein the line extraction algorithm includes at least one of the following: Hough line extraction and LCD line extraction.

[0056] That is, a line extraction algorithm is applied to the first image to extract the target line segment. The line extraction algorithm may be a Hough line extraction algorithm and an LCD line extraction algorithm, which is not limited in the embodiment of the present invention.

[0057] In an exemplary embodiment, after extracting the target line segment from the first image, the first position of the target line segment is obtained, wherein the first position is used to indicate the position of the target line segment in the first target image; based on the first position of the target line segment and the imaging principle, the distance of the target line segment from the mobile robot is determined; and based on the distance, the second position of the target line segment in the working area of ​​the mobile robot is determined.

[0058] It can be understood that the first position of the target line segment in the first target image is determined, and based on the imaging principle and the first position, the distance of the target line segment from the mobile robot is determined, and then the second position of the target line segment in the working area of ​​the mobile robot is determined based on the distance and the position of the mobile robot, thereby accurately determining the position of the target line segment in the working area of ​​the mobile robot.

[0059] In order to better understand the above-mentioned line segment extraction method, the above-mentioned technical solution is explained below in combination with optional embodiments, but it is not used to limit the technical solution of the embodiment of the present invention.

[0060] Figure 3 Flowchart (II) of the method for extracting line segments according to an embodiment of the present invention, Figure 3 As shown, the specific steps are as follows:

[0061] Step S301: acquiring an original image (equivalent to the first target image in the above embodiment);

[0062] Step S302: inputting the original image into the visual detection model, and outputting a local image (equivalent to the target area in the above embodiment);

[0063] It should be noted that the visual detection model is a deep learning detection model trained based on sample data. The deep learning detection model can be a YOLO series model or an RCN series model, which is not limited in the embodiment of the present invention.

[0064] Step S303: enhancing the local image by using an image enhancement operator;

[0065] The image enhancement operator is used to increase the pixel value contrast of the pixels in the target area within the range of 0-255 pixel values. The image enhancement operator can be a histogram equalization operator. The image after local image enhancement increases the pixel value contrast of the target line segment and the background. For example, after the white wall and the white door lintel are enhanced by the image enhancement operator, the door lintel line can be clearly seen, thereby solving the problem that the target line segment cannot be extracted due to the small contrast between the background and the target line segment pixel values.

[0066] Step S304: extracting lines in the adjusted local image;

[0067] The line extraction algorithm may be Hough line extraction or LCD line extraction, which is not limited in the embodiment of the present invention.

[0068] Step S305: Determine the exact position of the target line segment through the imaging principle.

[0069] For example, an application scenario involved in this embodiment is: a sweeping robot determines the exact position of the front door by detecting the front door lintel. The specific method is as follows: a camera is located directly in front of the sweeping robot, and the original image is obtained through the camera of the sweeping robot. The original image is input into a visual detection model. The trained visual detection model gives the door lintel line in the image in real time. At this time, the detection frame includes the entire door lintel line, and the detection frame is the target area in the image that needs to be enhanced.

[0070] An image enhancement operator is designed to remap the pixel values ​​of the target frame to between 0 and 255. For example, histogram equalization can be performed. After local image enhancement, the pixel value contrast between the lintel and the background is increased, such as Figure 4 As shown, Figure 4is a schematic diagram of the effect of the line segment extraction method according to an embodiment of the present invention. A line extraction algorithm is applied to the enhanced target area to extract the door lintel line. The line extraction algorithm may be Hough line extraction and LCD line extraction, which is not limited in the embodiment of the present invention. After the position of the door lintel line in the original image is obtained, the distance between the door lintel line and the sweeping robot is calculated through the imaging principle, and then the position of the door lintel line in the working area of ​​the sweeping robot is accurately determined according to the position of the sweeping robot and the distance between the door lintel line and the sweeping robot.

[0071] Through the embodiment of the present invention, the first target image acquired by the mobile robot is input into the visual detection model to obtain the target area corresponding to the first target image; the contrast between the first pixel value of the target line segment in the target area and the second pixel value of the background of the target area is increased by the image enhancement operator to obtain the first image; the target line segment is extracted from the first image; the target line segment is extracted from the first image, that is, the contrast between the first pixel value of the target line segment and the second pixel value of the background of the target area is increased by the image enhancement operator, and then the target line segment is extracted from the first image. The above technical scheme is adopted to solve the problems in the related art that for images with unclear line features, the line segment detection rate is not high and the line segment extraction effect is not ideal, and the target line segment can be effectively extracted.

[0072] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0073] In the present embodiment, an extraction device for line segment is also provided, which is used to realize the above-mentioned embodiment and preferred implementation mode, and the description thereof will not be repeated. As used below, the term "module" can realize the combination of software and / or hardware of the predetermined function. Although the device described in the following embodiments is preferably realized with software, the realization of hardware, or the combination of software and hardware is also possible and conceived.

[0074] Figure 5 is a structural block diagram of a line segment extraction device according to an embodiment of the present invention, such as Figure 3 As shown, including:

[0075] An input module 52, used to input a first target image acquired by the mobile robot into a visual detection model to acquire a target area corresponding to the first target image, wherein the target area includes a target line segment to be extracted;

[0076] It should be noted that the mobile robot can acquire the first target image through a camera or other device on the mobile robot, and the embodiment of the present invention does not limit the device for acquiring the first target image.

[0077] An adjustment module 54 is used to increase the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by using an image enhancement operator to obtain a first image;

[0078] The extraction module 56 is used to extract the target line segment from the first image.

[0079] Through the embodiment of the present invention, the first target image acquired by the mobile robot is input into the visual detection model to obtain the target area corresponding to the first target image; the contrast between the first pixel value of the target line segment in the target area and the second pixel value of the background of the target area is increased by the image enhancement operator to obtain the first image; the target line segment is extracted from the first image, that is, the contrast between the first pixel value of the target line segment and the second pixel value of the background of the target area is increased by the image enhancement operator, and then the target line segment is extracted from the first image. The above technical solution is adopted to solve the problems in the related technology that for images with unclear line features, the line segment detection rate is not high and the line segment extraction effect is not ideal, and the target line segment can be effectively extracted.

[0080] In an exemplary embodiment, the input module is also used to input the first target image acquired by the mobile robot into the visual detection model to obtain first target image information corresponding to the first target image, wherein the first target image information includes: a first image label, a first image confidence; determining whether there is a target area in the first target image according to the first image label and the first image confidence, and if the target area exists, acquiring the target area from the first target image.

[0081] It can be understood that the first target image is input into the trained visual detection model to obtain the first image label and the first image confidence of the first target image. When the first image label is the same as the label of the target line segment to be extracted and the first image confidence is greater than a preset threshold, it can be determined that there is a target area in the first target image. It should be noted that the visual detection model is a deep learning detection model trained based on sample data. The deep learning detection model can be a YOLO series model or an RCN series model, etc. The samples of the visual detection model can be images collected on the network and images acquired by a mobile robot, and the embodiments of the present invention are not limited to this.

[0082] In an exemplary embodiment, the above-mentioned device also includes: a determination module, used to match the first image label with the label of the target line segment to be extracted; determine whether the first image confidence is greater than a preset threshold; when the first image label is the same as the label of the target line segment to be extracted and the first image confidence is greater than a preset threshold, determine that there is a target area in the first target image.

[0083] It can be understood that, first, it is determined based on the first image label whether it is the same as the label of the target line segment to be extracted. When the first image label is determined to be the same as the label of the target line segment to be extracted, it is judged whether the first image confidence is greater than a preset threshold. When it is determined that the first image confidence is greater than the preset threshold, there is a target area in the first target image. It should be noted that when the first image label is determined to be the same as the label of the target area, it can be understood that the object category corresponding to the first image is the same as the object category of the target area. The first image confidence can be understood as the credibility score of the object in the first image. For example, the first target image is input into the visual detection model, and the credibility of the object in the first target image being a door lintel is 90 points, which can also be understood as the possibility that the object in the first target image is a door lintel is 90%.

[0084] In an exemplary embodiment, the above-mentioned device determination module is also used to determine whether the confidence of the first image is greater than a preset threshold; match the first image label with the label of the target line segment to be extracted; when the first image label is the same as the label of the target line segment to be extracted and the confidence of the first image is greater than a preset threshold, determine that there is a target area in the first target image.

[0085] It can be understood that first, it is determined whether the confidence of the first image is greater than a preset threshold. When it is determined that the confidence of the first image is greater than the preset threshold, it is determined whether the first image label is the same as the label of the target line segment to be extracted. When the first image label is determined to be the same as the label of the target line segment to be extracted, it is determined whether the confidence of the first image is greater than the preset threshold, and there is a target area in the first target image. If the confidence of the first image is less than the preset threshold, and or, the first image label is different from the label of the target line segment to be extracted, it is directly determined that there is no target area in the first target image.

[0086] It can be seen from the above embodiments that it is possible to first determine whether the confidence of the first image is greater than a preset threshold; or to first match the first image label with the label of the target line segment to be extracted. The embodiment of the present invention does not limit the order of the above execution steps.

[0087] In an exemplary embodiment, the input module is also used to obtain a second target image through the mobile robot when there is no target area in the first target image, and input the second target image into the visual detection model to obtain second image information corresponding to the second target image, wherein the second image information includes: a second image label and a second image confidence; determine whether there is a target area in the second target image according to the second image label and the second image confidence, and obtain the target area from the second target image when the target area exists.

[0088] In other words, when the confidence of the first image is less than a preset threshold, and or, when the first image label is different from the label of the target line segment to be extracted, that is, when there is no target area in the first target image, the mobile robot is instructed to obtain a second target image, wherein the second target image is an image corresponding to other areas different from the first target image, and the second target image is input into the trained visual detection model to obtain the second image label and the second image confidence of the second target image. When the second image label is the same as the label of the target line segment to be extracted and the confidence of the second image is greater than a preset threshold, it can be determined that there is a target area in the second target image. Otherwise, there is no target area in the second target image. When there is no target area in the second target image, it is necessary to continue to obtain a third target image, wherein the third target image is an image corresponding to other areas different from the first target image and the second image, until there is a target area in the target image.

[0089] It should be noted that, when the target area does not exist in the first target image, another image (ie, the second target image) is acquired, and the first target image and the second target image are two images acquired by the mobile robot.

[0090] In an exemplary embodiment, the adjustment module is also used to obtain all pixel values ​​of the target area; map the pixel values ​​through the image enhancement operator to increase the contrast between the first pixel value and the second pixel value, wherein the image enhancement operator includes at least one of the following: Laplace operator, histogram equalization.

[0091] It should be noted that all pixel values ​​in the target area are mapped through the Laplace operator and histogram equalization to increase the contrast between the first pixel value and the second pixel value, so as to increase the contrast between the target line segment and the background. Furthermore, the contrast between the first pixel value and the second pixel value can be increased by multiplication, convolution, etc. Taking the door lintel as an example, the pixel value of the target area is remapped to between 0 and 255 through the image enhancement operator. In the second image after pixel adjustment, the pixel value contrast between the door lintel and the background is increased, thereby solving the problem of being unable to extract the target line segment due to the small contrast between the background and target line segment pixel values.

[0092] In an exemplary embodiment, the extraction module is further used to acquire the first image; and extract the target line segment from the first image using a line extraction algorithm, wherein the line extraction algorithm includes at least one of the following: Hough line extraction and LCD line extraction.

[0093] That is, a line extraction algorithm is applied to the first image to extract the target line segment. The line extraction algorithm may be a Hough line extraction algorithm and an LCD line extraction algorithm, which is not limited in the embodiment of the present invention.

[0094] In an exemplary embodiment, the determination module is also used to obtain the first position of the target line segment, wherein the first position is used to indicate the position of the target line segment in the first target image; determine the distance of the target line segment from the mobile robot based on the first position of the target line segment and the imaging principle; and determine the second position of the target line segment in the working area of ​​the mobile robot based on the distance.

[0095] It can be understood that the first position of the target line segment in the first target image is determined, and based on the imaging principle and the first position, the distance of the target line segment from the mobile robot is determined, and then the second position of the target line segment in the working area of ​​the mobile robot is determined based on the distance and the position of the mobile robot, thereby accurately determining the position of the target line segment in the working area of ​​the mobile robot.

[0096] An embodiment of the present invention further provides a storage medium, which includes a stored program, wherein the program executes any of the above methods when it is run.

[0097] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following steps:

[0098] S1, inputting a first target image acquired by the mobile robot into a visual detection model to acquire a target area corresponding to the first target image, wherein the target area includes a target line segment to be extracted;

[0099] S2, increasing the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by using an image enhancement operator to obtain a first image;

[0100] S3: extract the target line segment from the first image.

[0101] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0102] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0103] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0104] S1, inputting a first target image acquired by the mobile robot into a visual detection model to acquire a target area corresponding to the first target image, wherein the target area includes a target line segment to be extracted;

[0105] S2, increasing the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by using an image enhancement operator to obtain a first image;

[0106] S3: extract the target line segment from the first image.

[0107] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0108] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0109] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0110] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for extracting line segments, characterized in that: include: Inputting a first target image acquired by the mobile robot into a visual detection model to acquire a target area corresponding to the first target image, wherein the target area is an area where a local image in the first target image is located, wherein the target area includes a target line segment to be extracted; Inputting a first target image acquired by the mobile robot into a visual detection model to acquire a target area corresponding to the first target image includes: Inputting a first target image acquired by the mobile robot into a visual detection model to obtain first target image information corresponding to the first target image, wherein the first target image information includes: a first image label and a first image confidence, wherein the first image label is used to indicate the type of a line segment in the first target image, and the first image confidence is used to indicate the score of the line segment being the target line segment; determining whether a target area exists in the first target image according to the first image label and the first image confidence, and obtaining the target area from the first target image if the target area exists; increasing the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by using an image enhancement operator to obtain a first image; The target line segment is extracted from the first image.

2. The line segment extraction method according to claim 1, characterized in that: Determining whether a target area exists in the first target image according to the first image label and the first image confidence includes: Matching the first image label with the label of the target line segment to be extracted, and determining whether the first image confidence is greater than a preset threshold; When the first image label is the same as the label of the target line segment to be extracted and the first image confidence is greater than a preset threshold, it is determined that a target area exists in the first target image.

3. The line segment extraction method according to claim 1, characterized in that: Determining whether a target area exists in the first target image according to the first image label and the first image confidence includes: In the case where there is no target area in the first target image, a second target image is acquired by the mobile robot, and the second target image is input into the visual detection model to obtain second image information corresponding to the second target image, wherein the second image information includes: a second image label and a second image confidence; Determine whether a target region exists in the second target image according to the second image label and the second image confidence, and acquire the target region from the second target image if the target region exists.

4. The line segment extraction method according to claim 1, characterized in that: Increasing the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by using an image enhancement operator includes: Obtaining pixel values ​​of the target area; The pixel values ​​are mapped by the image enhancement operator to increase the contrast between the first pixel value and the second pixel value, wherein the image enhancement operator includes at least one of the following: a Laplacian operator and a histogram equalization.

5. The line segment extraction method according to claim 1, characterized in that: Extracting the target line segment from the first image includes: The target line segment is extracted from the first image according to a line extraction algorithm, wherein the line extraction algorithm includes at least one of the following: Hough line extraction and LCD line extraction.

6. The line segment extraction method according to claim 1, characterized in that: After extracting the target line segment from the first image, the method further includes: Acquire a first position of the target line segment, wherein the first position is used to indicate a position of the target line segment in the first target image; Determine the distance between the target line segment and the mobile robot according to the first position of the target line segment and the imaging principle; A second position of the target line segment in the working area of ​​the mobile robot is determined according to the distance.

7. A line segment extraction device, characterized in that: The device comprises: An input module, used to input a first target image acquired by a mobile robot into a visual detection model to obtain a target area corresponding to the first target image, wherein the target area is an area in the first target image where a local image is located, wherein the target area includes a target line segment to be extracted; inputting the first target image acquired by the mobile robot into the visual detection model to obtain the target area corresponding to the first target image, comprising: inputting the first target image acquired by the mobile robot into the visual detection model to obtain first target image information corresponding to the first target image, wherein the first target image information comprises: a first image label, a first image confidence, wherein the first image label is used to indicate the type of line segment in the first target image, and the first image confidence is used to indicate the score of the line segment being the target line segment; determining whether there is a target area in the first target image according to the first image label and the first image confidence, and if the target area exists, obtaining the target area from the first target image; an adjustment module, configured to increase the contrast between a first pixel value of the target line segment in the target area and a second pixel value of the background of the target area by using an image enhancement operator to obtain a first image; An extraction module is used to extract the target line segment from the first image.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

Images