Flexible wire harness detection method and device, readable storage medium and electronic equipment

Abstract

The invention belongs to the technical field of object detection, and particularly relates to a flexible wire harness detection method and device, a computer readable storage medium and electronic equipment. The method comprises the following steps: acquiring event stream data of a flexible wire harness collected by an event camera; converting the event stream data into an event image; performing flexible wire harness detection in the event image to obtain pixel position coordinates of the flexible wire harness in a pixel plane coordinate system of the event camera; and mapping the pixel position coordinates to an actual space plane from the pixel plane coordinate system to obtain actual position coordinates of the flexible wire harness in the actual space plane. Due to the fact that the dynamic range of the event camera is wide, the influence of external factors such as light source brightness is effectively avoided, the event camera is sensitive to an object moving rapidly, obvious motion blur cannot be generated, the data size of the event stream data is small, redundant information of the background is little, and the detection efficiency and accuracy can be effectively improved.

Description

Technical Field

[0001] The present application belongs to the field of object detection technology, and in particular relates to a flexible wire harness detection method, device, computer-readable storage medium, and electronic device. Background Art

[0002] Flexible wiring harnesses often require inspection in industrial production. For example, when using a robotic arm to install or remove circuit boards, the position of the flexible wiring harnesses needs to be detected in real time to replan the robotic arm's movement path and prevent wiring harness entanglement, which could lead to safety accidents.

[0003] In the existing technology, flexible harness detection is generally based on images captured by traditional RGB cameras. However, this method has high requirements on lighting conditions and is easily affected by external factors such as light source brightness. When the flexible harness shakes rapidly, the captured image will produce obvious motion blur. In addition, the image data volume is large and mixed with a lot of redundant information about the background, resulting in low detection efficiency and accuracy. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a flexible wire harness detection method, device, computer-readable storage medium, and electronic device to solve the problems of low efficiency and accuracy of existing flexible wire harness detection methods.

[0005] A first aspect of an embodiment of the present application provides a flexible wiring harness detection method, which may include:

[0006] Obtain event stream data of the flexible harness collected by the event camera;

[0007] Converting the event stream data into an event image;

[0008] Performing flexible wire harness detection in the event image to obtain pixel position coordinates of the flexible wire harness in a pixel plane coordinate system of the event camera;

[0009] The pixel position coordinates are mapped from the pixel plane coordinate system to the actual space plane to obtain the actual position coordinates of the flexible harness in the actual space plane.

[0010] Through the above method, the event stream data of the flexible wiring harness can be collected based on the event camera, and the flexible wiring harness detection can be performed based on the event image converted from the event stream data. Since the event camera has a wide dynamic range and can work stably in both strong and weak light, it effectively avoids the influence of external factors such as the brightness of the light source. The event camera is more sensitive to fast-moving objects and will not produce obvious motion blur. Moreover, the data volume of the event stream data is small, and the background redundant information is less, which can effectively improve the efficiency and accuracy of detection.

[0011] In a specific implementation of the first aspect, detecting the flexible harness in the event image and obtaining the pixel position coordinates of the flexible harness in the pixel plane coordinate system of the event camera may include:

[0012] Performing image expansion processing on the event image to obtain an expanded image;

[0013] performing image erosion processing on the dilation-processed image to obtain an erosion-processed image;

[0014] Performing skeleton extraction processing on the erosion-processed image to obtain a skeleton image;

[0015] The pixel position coordinates are determined according to the skeleton image.

[0016] Through the above method, a clearer flexible harness shape can be obtained based on a series of image processing processes such as expansion, erosion, and skeleton extraction, thereby obtaining more accurate pixel position coordinates.

[0017] In a specific implementation of the first aspect, the event image may include an event frame image;

[0018] Accordingly, converting the event stream data into an event image may include:

[0019] Converting the event stream data into the event frame image;

[0020] Detecting the flexible wire harness in the event image and obtaining pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera may include:

[0021] Flexible wire harness detection is performed in the event frame image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0022] Through the above method, event stream data can be converted into event frame images, which facilitates the use of various image processing methods for flexible wire harness detection and effectively improves detection efficiency.

[0023] In a specific implementation of the first aspect, the event image may include a time plane image;

[0024] Accordingly, converting the event stream data into an event image may include:

[0025] Converting the event stream data into the time plane image;

[0026] Detecting the flexible wire harness in the event image and obtaining pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera may include:

[0027] Flexible wire bundle detection is performed in the time plane image to obtain pixel position coordinates of the flexible wire bundle in the pixel plane coordinate system of the event camera.

[0028] Through the above method, event stream data can be converted into time plane images, which makes it easier to use various image processing methods to perform flexible wire harness detection, effectively improving the detection efficiency.

[0029] In a specific implementation of the first aspect, the event image may include an event counting image;

[0030] Accordingly, converting the event stream data into an event image may include:

[0031] converting the event stream data into the event count image;

[0032] Detecting the flexible wire harness in the event image and obtaining pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera may include:

[0033] Flexible wire harness detection is performed in the event counting image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0034] Through the above method, event stream data can be converted into event count images, which facilitates the use of various image processing methods for flexible wire harness detection and effectively improves detection efficiency.

[0035] In a specific implementation of the first aspect, after acquiring the event stream data of the flexible harness collected by the event camera, the method may further include:

[0036] Performing filtering on the event stream data to obtain event stream filtered data;

[0037] Accordingly, converting the event stream data into an event image may include:

[0038] The event stream filtered data is converted into the event image.

[0039] In a specific implementation of the first aspect, filtering the event stream data to obtain event stream filtered data may include:

[0040] Gaussian filtering is performed on the event stream data to obtain the event stream filtered data.

[0041] In a specific implementation of the first aspect, filtering the event stream data to obtain event stream filtered data may include:

[0042] Perform mean filtering on the event stream data to obtain the event stream filtered data.

[0043] In a specific implementation of the first aspect, filtering the event stream data to obtain event stream filtered data may include:

[0044] Perform median filtering on the event stream data to obtain the event stream filtered data.

[0045] Through the above method, possible noise interference in the event stream data can be filtered out, thereby obtaining purer data and further improving the accuracy of detection.

[0046] In a specific implementation of the first aspect, obtaining the event stream data of the flexible harness collected by the event camera may include:

[0047] Sending a data acquisition trigger instruction to the control device of the event camera; wherein the data acquisition trigger instruction is used to instruct the control device to trigger the event camera to collect the event stream data;

[0048] The event stream data sent by the event camera is received.

[0049] Through the above method, the timing of event camera data collection can be accurately controlled through data collection trigger instructions.

[0050] In a specific implementation of the first aspect, after mapping the pixel position coordinates from the pixel plane coordinate system to the real space plane to obtain the actual position coordinates of the flexible harness in the real space plane, the method may further include:

[0051] Adjacent actual position coordinates are connected to obtain a continuous image of the flexible harness in the actual space plane.

[0052] By using the above method, discontinuous actual position coordinates can be connected to obtain a complete continuous image of the flexible harness.

[0053] A second aspect of an embodiment of the present application provides a flexible wire harness detection device, which may include:

[0054] A data acquisition module, used to acquire event stream data of the flexible harness collected by the event camera;

[0055] A data conversion module, configured to convert the event stream data into an event image;

[0056] A flexible wire harness detection module, configured to detect the flexible wire harness in the event image and obtain pixel position coordinates of the flexible wire harness in a pixel plane coordinate system of the event camera;

[0057] A coordinate mapping module is used to map the pixel position coordinates from the pixel plane coordinate system to the actual space plane to obtain the actual position coordinates of the flexible harness in the actual space plane.

[0058] In a specific implementation of the second aspect, the flexible harness detection module may include:

[0059] An image expansion processing unit, configured to perform image expansion processing on the event image to obtain an expanded image;

[0060] An image corrosion processing unit, configured to perform image corrosion processing on the dilation-processed image to obtain a corrosion-processed image;

[0061] a skeleton extraction processing unit, configured to perform skeleton extraction processing on the erosion processed image to obtain a skeleton image;

[0062] A pixel position coordinate determining unit is configured to determine the pixel position coordinates according to the skeleton image.

[0063] In a specific implementation of the second aspect, the event image may include an event frame image;

[0064] Accordingly, the data conversion module may be specifically configured to: convert the event stream data into the event frame image;

[0065] The flexible wire harness detection module may be specifically configured to perform flexible wire harness detection in the event frame image, and obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0066] In a specific implementation of the second aspect, the event image may include a time plane image;

[0067] Accordingly, the data conversion module may be specifically configured to: convert the event stream data into the time plane image;

[0068] The flexible wire harness detection module may be specifically configured to perform flexible wire harness detection in the time plane image, and obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0069] In a specific implementation of the second aspect, the event image may include an event counting image;

[0070] Accordingly, the data conversion module may be specifically configured to: convert the event stream data into the event count image;

[0071] The flexible wire harness detection module may be specifically configured to perform flexible wire harness detection in the event counting image, and obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0072] In a specific implementation of the second aspect, the flexible wire harness detection device may further include:

[0073] A data filtering module, configured to filter the event stream data to obtain event stream filtered data;

[0074] Accordingly, the data conversion module may be specifically configured to convert the event stream filtered data into the event image.

[0075] In a specific implementation of the second aspect, the data filtering module may include:

[0076] The Gaussian filtering unit is used to perform Gaussian filtering on the event stream data to obtain the event stream filtered data.

[0077] In a specific implementation of the second aspect, the data filtering module may include:

[0078] The mean filtering unit is used to perform mean filtering on the event stream data to obtain the event stream filtered data.

[0079] In a specific implementation of the second aspect, the data filtering module may include:

[0080] The median filtering unit is used to perform median filtering on the event stream data to obtain the event stream filtered data.

[0081] In a specific implementation of the second aspect, the data acquisition module may include:

[0082] A trigger instruction sending unit, configured to send a data acquisition trigger instruction to the control device of the event camera; wherein the data acquisition trigger instruction is used to instruct the control device to trigger the event camera to acquire the event stream data;

[0083] An event stream data receiving unit is configured to receive the event stream data sent by the event camera.

[0084] In a specific implementation of the second aspect, the flexible wire harness detection device may further include:

[0085] An image connection module is used to connect adjacent actual position coordinates to obtain a continuous image of the flexible harness in the actual space plane.

[0086] A third aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned flexible wire harness detection methods are implemented.

[0087] A fourth aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned flexible harness detection methods when executing the computer program.

[0088] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps of any one of the above-mentioned flexible wire harness detection methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0090] Figure 1 This is a schematic diagram of a specific practical application scenario in the embodiment of the present application;

[0091] Figure 2 This is a flow chart of an embodiment of a flexible wire harness detection method in an embodiment of the present application;

[0092] Figure 3 A schematic flow chart for obtaining pixel position coordinates of the flexible wire bundle in the pixel plane coordinate system of the event camera for flexible wire bundle detection in an event image;

[0093] Figure 4 This is a comparison diagram of the image before and after expansion processing;

[0094] Figure 5 This is a comparison diagram of the image before and after corrosion processing;

[0095] Figure 6 This is a comparison diagram before and after skeleton extraction;

[0096] Figure 7 A schematic diagram of mapping pixel position coordinates from the pixel plane coordinate system to the real space plane;

[0097] Figure 8 This is a structural diagram of an embodiment of a flexible wire harness detection device in an embodiment of the present application;

[0098] Figure 9 This is a schematic block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0099] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0100] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0101] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0102] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0103] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0104] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0105] Flexible wiring harnesses often require inspection in industrial production. For example, when using a robotic arm to install or remove circuit boards, the position of the flexible wiring harnesses needs to be detected in real time to replan the robotic arm's movement path and prevent wiring harness entanglement, which could lead to safety accidents.

[0106] In the existing technology, flexible harness detection is generally based on images captured by traditional RGB cameras. However, this method has high requirements on lighting conditions and is easily affected by external factors such as light source brightness. When the flexible harness shakes rapidly, the captured image will produce obvious motion blur. In addition, the image data volume is large and mixed with a lot of redundant information about the background, resulting in low detection efficiency and accuracy.

[0107] In view of this, embodiments of the present application provide a flexible wire harness detection method, device, computer-readable storage medium, and electronic device to solve the problems of low efficiency and accuracy of existing flexible wire harness detection methods.

[0108] In an embodiment of the present application, event stream data of a flexible wiring harness can be collected based on an event camera, and flexible wiring harness detection can be performed based on an event image converted from the event stream data. Since the event camera has a wide dynamic range and can work stably in both strong and weak light conditions, it effectively avoids the influence of external factors such as light source brightness. The event camera is more sensitive to fast-moving objects and will not produce obvious motion blur. Moreover, the amount of event stream data is small, and the background has less redundant information, which can effectively improve the efficiency and accuracy of detection.

[0109] The execution subject of the method of the present application may be an electronic device, which may include but is not limited to desktop computers, notebooks, PDAs, servers and other computing devices.

[0110] Figure 1 A schematic diagram of a specific practical application scenario in an embodiment of the present application is shown. As shown in the figure, the flexible wire harness is located in a certain actual space plane and shakes in the actual space plane. The dotted line shows the flexible wire harness before shaking, and the solid line shows the flexible wire harness after shaking. The event camera faces the actual space plane and is used to collect event stream data of the flexible wire harness. A communication connection is established between the electronic device and the event camera for executing the flexible wire harness detection method in the embodiment of the present application.

[0111] See also Figure 2 In an embodiment of the present application, a flexible wire harness detection method may include:

[0112] Step S201: Acquire event stream data of the flexible harness collected by an event camera.

[0113] When flexible harness testing is required, the electronic device can send a data acquisition trigger instruction to the control device of the event camera. The data acquisition trigger instruction is used to instruct the control device to trigger the event camera to collect event stream data. The control device may include but is not limited to a programmable logic controller (PLC) or other device capable of controlling the event camera.

[0114] Once triggered, the event camera begins collecting event stream data from the flexible harness and sends it to the electronic device. The event camera only records information about changes in motion within its field of view. Over a period of time, the event camera records the changes in light intensity at each pixel in the image at every moment. If the light intensity change exceeds a certain threshold, the pixel is assigned a polarity (usually positive 1 indicates an increase in brightness, negative 1 indicates a decrease in brightness). This approach, compared to traditional cameras, which produce an "average photo" within a certain exposure time and synchronously record the average light intensity of each pixel, asynchronously outputs event stream data consisting of a series of four-tuples, each of which includes the pixel coordinates of the event, a timestamp, and a polarity.

[0115] After receiving the event stream data sent by the event camera, the electronic device starts to execute the subsequent flexible harness detection process.

[0116] Step S202: Convert the event stream data into an event image.

[0117] Because event stream data is composed of a series of quads, it is not convenient to directly analyze and process it. Therefore, in the embodiments of the present application, it can first be converted into an image format that is easier to process. The event image is the image of the event stream data, which can include but is not limited to: event frame image, time surface image, or event count image.

[0118] For event frame images, we can select events within a time period to form an event package. We then count the number of events that occurred at each pixel in the event package, ultimately outputting an event frame where each pixel has a value that represents some information about the event. For example, we can superimpose the polarity of all timestamp events at each pixel to create the simplest event frame. This event frame can be considered the aggregate of events within the event package time period.

[0119] For time-plane images, each pixel records a timestamp, representing the timestamp of the event at that pixel. In the time plane, the intensity of each pixel represents the "motion trace" at that location. Because the brightness and darkness changes that produce an event are often associated with motion, this event representation format is particularly effective for problems involving motion.

[0120] For event counting images, in order to calculate the density of motion compensation, the image plane can be discretized, each coordinate in the motion compensation will be mapped to a pixel, and then the number falling into each pixel is recorded as the value of the pixel, thereby obtaining an event counting image.

[0121] Through the above method, event stream data can be converted into event images such as event frame images, time plane images or event count images, which facilitates the use of various image processing methods for flexible wire harness detection and effectively improves detection efficiency.

[0122] In a specific implementation of an embodiment of the present application, after receiving event stream data sent by an event camera, the electronic device may first filter the event stream data to obtain filtered event stream data, and then convert the filtered event stream data into an event image. The filtering process may include, but is not limited to, Gaussian filtering, mean filtering, and / or median filtering.

[0123] Among them, the core of Gaussian filtering is the Gaussian function, which is a continuous two-dimensional normal distribution function. In image processing, a discretized Gaussian kernel (also called a Gaussian template) is usually used to filter the image. The size and standard deviation of the Gaussian kernel determine the smoothness and range of influence of the filter. For each pixel in the image, a neighborhood (i.e., the size of the Gaussian kernel) is selected with it as the center, and then the weight of each pixel in the neighborhood is calculated according to the Gaussian function. Finally, the values ​​of all pixels in the neighborhood are multiplied by their corresponding weights and summed to obtain the filtered pixel value. This process traverses every pixel in the image and completes the Gaussian filtering of the entire image.

[0124] Mean filtering, also known as average filtering, is a linear filtering method. Its basic concept is to replace the pixel value of each pixel in an image with the average of the pixel values ​​of its neighboring pixels. This method effectively eliminates noise in an image, but it also blurs the image. The mean filter is based on the neighborhood averaging method. Specifically, for any pixel in a given image, the average value of all the pixels in its neighborhood is calculated and assigned to the current pixel to obtain the filtered pixel value. This process iterates through every pixel in the image, completing the mean filtering process for the entire image.

[0125] Median filtering is a nonlinear filtering technique based on statistical sorting theory. The basic idea behind median filtering is to replace the value of a point in a digital image or sequence with the median of all the points in its neighborhood. This filtering method effectively eliminates salt-and-pepper noise while preserving image edge information, unlike mean filtering, which can cause image blur. Median filtering works by sorting the pixel values ​​in a neighborhood and selecting the median value as the output. For two-dimensional median filtering, a 3×3 or 5×5 window is typically selected as the neighborhood. All pixel values ​​within the window are sorted, the median value is found, and then assigned to the pixel at the center of the window. This process effectively eliminates isolated noise points while preserving image edges and detail.

[0126] Through the above method, possible noise interference in the event stream data can be filtered out, and only the events triggered by the moving flexible harness are retained, thereby obtaining purer data and further improving the detection accuracy.

[0127] Step S203 : performing flexible wire harness detection in the event image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0128] In a specific implementation of the embodiment of the present application, step S203 may specifically include the following: Figure 3 The process shown:

[0129] Step S2031: Perform image expansion processing on the event image to obtain an expanded image.

[0130] The primary purpose of image dilation is to enlarge the boundaries of foreground objects in an image, fill small holes within them, and connect adjacent objects, thereby enhancing the features of the foreground objects. Image dilation performs a neighborhood analysis on each pixel in the image based on a structuring element. A structuring element is a predefined shape that determines the scope and method of the dilation process. Common structuring elements include rectangles, ellipses, and crosses. During dilation, for each foreground pixel in the image, all pixels within its neighborhood are examined. If any pixel in the neighborhood corresponds to a position in the structuring element, the foreground pixel is retained or expanded. This results in the foreground object's boundaries expanding outward, creating a dilation effect.

[0131] Figure 4The figure below shows a comparison of image dilation before and after processing. The left image shows the event image before dilation, and the right image shows the event image after dilation, also known as the dilated image. As shown in the figure, because the events triggered by the event camera are asynchronous and discrete, the pixels of the converted event image may be discontinuous. Image dilation can fill these discontinuities to form a complete, continuous connected domain.

[0132] Step S2032: Perform image corrosion processing on the dilation-processed image to obtain a corrosion-processed image.

[0133] The main purpose of image erosion is to smooth the boundaries of foreground objects, remove small bumps and burrs, and shrink the foreground objects in the image so that their boundaries shrink inward. Image erosion performs a neighborhood analysis on each pixel in the image based on the structuring element. Only when the structuring element is completely within the foreground object is the pixel corresponding to the center of the structuring element considered a foreground pixel; otherwise, it is considered a background pixel. Specifically, image erosion traverses each pixel in the image and places the structuring element at its center. If all pixels within the structuring element belong to the foreground object, the center pixel is retained as a foreground pixel; otherwise, it is set to a background pixel. In this way, the edges of the foreground objects in the image gradually shrink inward, achieving an erosion effect.

[0134] Figure 5 The figure below shows a comparison of an image before and after erosion. The left image shows the event image before erosion, and the right image shows the event image after erosion, also known as the eroded image. As shown in the figure, image erosion removes burrs and noise, smoothing connected domains and eliminating some boundary values, resulting in an overall reduction in image size.

[0135] Step S2033: Perform skeleton extraction processing on the eroded image to obtain a skeleton image.

[0136] Skeleton extraction involves using an image thinning algorithm to thin foreground objects in an image down to a single-pixel width, thereby extracting the object's skeleton or centerline. The core idea is to iteratively remove edge pixels of the foreground object until only a single-pixel skeleton remains. The underlying principle is to define a series of rules or conditions to determine which pixels should be removed and in what order. Image thinning algorithms include, but are not limited to, the Hilditch algorithm, the Rosenfeld algorithm, the Pavlidis algorithm, the Zhang-Suen algorithm, and index table-based thinning algorithms.

[0137] Figure 6The figure below shows a comparison of the image before and after skeleton extraction. The left image shows the event image before skeleton extraction, and the right image shows the event image after skeleton extraction, also known as the skeleton image. As shown in the figure, skeleton extraction can remove some points from an image while still preserving the original shape of the remaining points, which is the image skeleton.

[0138] Step S2034: Determine pixel position coordinates based on the skeleton image.

[0139] After a series of image processing processes such as expansion, corrosion, and skeleton extraction, a clearer shape of the flexible wire harness can be obtained, so that the pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera can be determined more accurately.

[0140] Step S204 : Mapping the pixel position coordinates from the pixel plane coordinate system to the actual space plane to obtain the actual position coordinates of the flexible harness in the actual space plane.

[0141] In the embodiments of this application, Figure 7 As shown in the figure, the pixel position coordinates can be mapped from the pixel plane coordinate system to the real space plane according to the mapping relationship between the pixel plane coordinate system and the world coordinate system where the real space plane is located, thereby obtaining the actual position coordinates of the flexible harness in the real space plane. Taking the imaging point A (a0, b0) of the flexible harness in the pixel plane coordinate system as an example, after the coordinate system mapping, its actual position coordinates A in the real space plane can be obtained. ′ Point (x0, y0), in this way, traverses the coordinates of each pixel position of the flexible harness in the pixel plane coordinate system, and performs coordinate system mapping on them respectively to obtain the corresponding actual position coordinates.

[0142] Since the actual position coordinates of the flexible harness obtained after coordinate mapping in the actual space plane may be discontinuous, after obtaining the actual position coordinates of the flexible harness in the actual space plane, adjacent actual position coordinates can be connected to obtain a continuous image of the flexible harness in the actual space plane.

[0143] To sum up, in the embodiments of the present application, the event stream data of the flexible wiring harness can be collected based on the event camera, and the flexible wiring harness detection can be performed based on the event image converted from the event stream data. Since the event camera has a wide dynamic range and can work stably in both strong and weak light, it effectively avoids the influence of external factors such as the brightness of the light source. The event camera is more sensitive to fast-moving objects and will not produce obvious motion blur. Moreover, the amount of event stream data is small, and the background redundant information is less, which can effectively improve the efficiency and accuracy of detection.

[0144] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0145] Corresponding to the flexible wire harness detection method described in the above embodiment, Figure 8 A structural diagram of an embodiment of a flexible wire harness detection device provided in an embodiment of the present application is shown.

[0146] In this embodiment, a flexible wire harness detection device may include:

[0147] The data acquisition module 801 is used to acquire the event stream data of the flexible harness collected by the event camera;

[0148] A data conversion module 802 is used to convert the event stream data into an event image;

[0149] A flexible wire harness detection module 803 is configured to detect a flexible wire harness in the event image and obtain pixel position coordinates of the flexible wire harness in a pixel plane coordinate system of the event camera;

[0150] The coordinate mapping module 804 is configured to map the pixel position coordinates from the pixel plane coordinate system to a real space plane to obtain the real position coordinates of the flexible harness in the real space plane.

[0151] In a specific implementation of the present application, the flexible harness detection module may include:

[0152] An image expansion processing unit, configured to perform image expansion processing on the event image to obtain an expanded image;

[0153] An image corrosion processing unit, configured to perform image corrosion processing on the dilation-processed image to obtain a corrosion-processed image;

[0154] a skeleton extraction processing unit, configured to perform skeleton extraction processing on the erosion processed image to obtain a skeleton image;

[0155] A pixel position coordinate determining unit is configured to determine the pixel position coordinates according to the skeleton image.

[0156] In a specific implementation of the present application, the event image may include an event frame image;

[0157] Accordingly, the data conversion module may be specifically configured to: convert the event stream data into the event frame image;

[0158] The flexible wire harness detection module may be specifically configured to perform flexible wire harness detection in the event frame image, and obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0159] In a specific implementation of the present application, the event image may include a time plane image;

[0160] Accordingly, the data conversion module may be specifically configured to: convert the event stream data into the time plane image;

[0161] The flexible wire harness detection module may be specifically configured to perform flexible wire harness detection in the time plane image, and obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0162] In a specific implementation of the present application, the event image may include an event counting image;

[0163] Accordingly, the data conversion module may be specifically configured to: convert the event stream data into the event count image;

[0164] The flexible wire harness detection module may be specifically configured to perform flexible wire harness detection in the event counting image, and obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

[0165] In a specific implementation of the present application, the flexible wire harness detection device may further include:

[0166] A data filtering module, configured to filter the event stream data to obtain event stream filtered data;

[0167] Accordingly, the data conversion module may be specifically configured to convert the event stream filtered data into the event image.

[0168] In a specific implementation of the present application, the data filtering module may include:

[0169] The Gaussian filtering unit is used to perform Gaussian filtering on the event stream data to obtain the event stream filtered data.

[0170] In a specific implementation of the present application, the data filtering module may include:

[0171] The mean filtering unit is used to perform mean filtering on the event stream data to obtain the event stream filtered data.

[0172] In a specific implementation of the present application, the data filtering module may include:

[0173] The median filtering unit is used to perform median filtering on the event stream data to obtain the event stream filtered data.

[0174] In a specific implementation of the present application, the data acquisition module may include:

[0175] A trigger instruction sending unit, configured to send a data acquisition trigger instruction to the control device of the event camera; wherein the data acquisition trigger instruction is used to instruct the control device to trigger the event camera to acquire the event stream data;

[0176] An event stream data receiving unit is configured to receive the event stream data sent by the event camera.

[0177] In a specific implementation of the present application, the flexible wire harness detection device may further include:

[0178] An image connection module is used to connect adjacent actual position coordinates to obtain a continuous image of the flexible harness in the actual space plane.

[0179] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0180] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0181] Figure 9 A schematic block diagram of an electronic device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0182] like Figure 9 As shown, the electronic device 9 of this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90. When the processor 90 executes the computer program 92, the steps in the above-mentioned flexible harness detection method embodiments are implemented, such as Figure 2 Alternatively, when the processor 90 executes the computer program 92, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 8 The functions of the data acquisition module 801 to the coordinate mapping module 804 are shown.

[0183] Exemplarily, the computer program 92 may be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 92 in the electronic device 9.

[0184] The electronic device 9 can be a computing device such as a desktop computer, a notebook, a palmtop computer, a server, etc. It will be understood by those skilled in the art that Figure 9 It is only an example of the electronic device 9 and does not constitute a limitation of the electronic device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 9 may also include input and output devices, network access devices, buses, etc.

[0185] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0186] The memory 91 may be an internal storage unit of the electronic device 9, such as a hard disk or memory of the electronic device 9. The memory 91 may also be an external storage device of the electronic device 9, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 9. Furthermore, the memory 91 may include both an internal storage unit of the electronic device 9 and an external storage device. The memory 91 is used to store the computer program and other programs and data required by the electronic device 9. The memory 91 may also be used to temporarily store data that has been output or is about to be output.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0188] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0189] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0190] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0191] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0192] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0193] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0194] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A flexible wiring harness detection method, characterized in that: include: Obtain event stream data of the flexible harness collected by the event camera; Converting the event stream data into an event image; Performing flexible wire harness detection in the event image to obtain pixel position coordinates of the flexible wire harness in a pixel plane coordinate system of the event camera; The pixel position coordinates are mapped from the pixel plane coordinate system to the actual space plane to obtain the actual position coordinates of the flexible harness in the actual space plane.

2. The flexible wire harness detection method according to claim 1, characterized in that: The detecting of the flexible wire harness in the event image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera includes: Performing image expansion processing on the event image to obtain an expanded image; performing image erosion processing on the dilation-processed image to obtain an erosion-processed image; Performing skeleton extraction processing on the erosion-processed image to obtain a skeleton image; The pixel position coordinates are determined according to the skeleton image.

3. The flexible wire harness detection method according to claim 1 or 2, characterized in that: The event image includes an event frame image; Accordingly, converting the event stream data into an event image includes: Converting the event stream data into the event frame image; The detecting of the flexible wire harness in the event image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera includes: Flexible wire harness detection is performed in the event frame image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

4. The flexible wire harness detection method according to claim 1 or 2, characterized in that: The event image includes a time plane image; Accordingly, converting the event stream data into an event image includes: Converting the event stream data into the time plane image; The detecting of the flexible wire harness in the event image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera includes: Flexible wire bundle detection is performed in the time plane image to obtain pixel position coordinates of the flexible wire bundle in the pixel plane coordinate system of the event camera.

5. The flexible wire harness detection method according to claim 1 or 2, characterized in that: The event image includes an event count image; Accordingly, converting the event stream data into an event image includes: converting the event stream data into the event count image; The detecting of the flexible wire harness in the event image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera includes: Flexible wire harness detection is performed in the event counting image to obtain pixel position coordinates of the flexible wire harness in the pixel plane coordinate system of the event camera.

6. The flexible wire harness detection method according to any one of claims 1 to 5, characterized in that: After obtaining the event stream data of the flexible harness collected by the event camera, it also includes: Performing filtering on the event stream data to obtain event stream filtered data; Accordingly, converting the event stream data into an event image includes: The event stream filtered data is converted into the event image.

7. The flexible wire harness detection method according to claim 6, characterized in that: The filtering process on the event stream data to obtain event stream filtered data includes: Gaussian filtering is performed on the event stream data to obtain the event stream filtered data.

8. The flexible wire harness detection method according to claim 6, characterized in that: The filtering process on the event stream data to obtain event stream filtered data includes: Perform mean filtering on the event stream data to obtain the event stream filtered data.

9. The flexible wire harness detection method according to claim 6, characterized in that: The filtering process on the event stream data to obtain event stream filtered data includes: Perform median filtering on the event stream data to obtain the event stream filtered data.

10. The flexible wire harness detection method according to any one of claims 1 to 9, characterized in that: The method of obtaining the event stream data of the flexible wiring harness collected by the event camera includes: Sending a data acquisition trigger instruction to the control device of the event camera; wherein the data acquisition trigger instruction is used to instruct the control device to trigger the event camera to collect the event stream data; The event stream data sent by the event camera is received.

11. The flexible wire harness detection method according to any one of claims 1 to 10, characterized in that: After mapping the pixel position coordinates from the pixel plane coordinate system to the real space plane to obtain the actual position coordinates of the flexible harness in the real space plane, the method further includes: Adjacent actual position coordinates are connected to obtain a continuous image of the flexible harness in the actual space plane.

12. A flexible wire harness detection device, characterized in that: include: A data acquisition module, used to acquire event stream data of the flexible harness collected by the event camera; A data conversion module, configured to convert the event stream data into an event image; A flexible wire harness detection module, configured to detect the flexible wire harness in the event image and obtain pixel position coordinates of the flexible wire harness in a pixel plane coordinate system of the event camera; A coordinate mapping module is used to map the pixel position coordinates from the pixel plane coordinate system to the actual space plane to obtain the actual position coordinates of the flexible harness in the actual space plane.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the flexible wire harness detection method according to any one of claims 1 to 11 are implemented.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the flexible wire harness detection method according to any one of claims 1 to 11 are implemented.

Images