Cleaning robot offline detection method, device and system

Abstract

The present invention provides a cleaning robot offline detection method, device and system, which belongs to the field of signal processing technology, including: obtaining the audio characteristics of the test sound signal of the cleaning robot to be tested within a sampling period; determining the abnormal sound detection result of the cleaning robot according to the audio characteristics of the test sound signal within each sampling period within the detection cycle; the test sound signal within any sampling period is collected based on the following steps: determining its position information according to the initial sound signal within the sampling period; re-collecting the sound signal in the direction of the position information as the test sound signal. The cleaning robot offline detection method provided by the present invention does not need to work in a silent box environment, and only needs to use a microphone array arranged on a test table to quickly and accurately locate the vibration fault position, can effectively suppress environmental noise and other test station noise interference, and can accurately realize the location and characterization of abnormal sound faults.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a cleaning robot offline detection method, device and system. Background Art

[0002] Before the cleaning robot is assembled and manufactured and rolled off the production line, it needs to undergo a relatively complete comprehensive function and performance test, mainly including cleaning route planning function test, actual operation trajectory test, edge obstacle avoidance function test and other test items. Abnormal noise detection (abnormal sound detection for short) is also required, mainly including abnormal noise detection of the robot's driving wheels, motors and other moving mechanisms, to ensure that the function and performance of the cleaning robot meet the quality requirements before it rolls off the production line and that there are no manufacturing defects in the core moving parts.

[0003] The current method of detecting abnormal noise on the production line is mainly to manually place the cleaning robot to be tested on the test table, and then use human ears to roughly judge whether each cleaning robot is normal before it goes offline by listening to the running sound of the cleaning robot. The current manual detection solution has the following problems:

[0004] On the one hand, abnormal sound detection currently relies mainly on testers to judge whether the noise is abnormal or not. The judgment criteria are relatively subjective and rely too much on the experience of testers, and cannot be quantified well. On the other hand, the production line rhythm requires that multiple sweeping robot products be tested at the same time. There are multiple test stations. Due to the interference of factory environmental noise and the mutual interference of noise between test stations, the accuracy of abnormal sound detection is seriously affected. Furthermore, abnormal sound detection by human ears can only detect whether there is a fault or not, and cannot locate the position of the abnormal sound, let alone accurately classify the fault mode of the abnormal sound.

[0005] In view of this, there is an urgent need to provide an abnormal sound detection method to overcome or improve to a certain extent the many disadvantages of the existing abnormal sound detection method. Summary of the invention

[0006] The present invention provides a cleaning robot offline detection method, device and system, which are used to solve the defects of low detection accuracy, large workload and inability to locate and characterize faults in the current abnormal sound detection method using human ears.

[0007] The present invention provides a cleaning robot offline detection method, comprising the following steps:

[0008] Acquire the audio characteristics of the test sound signal of the cleaning robot to be tested in any sampling period; the test sound signal is collected by a microphone array arranged on the testing table;

[0009] Determine the abnormal sound detection result of the cleaning robot to be tested according to the audio features of the test sound signal in each sampling period within the detection cycle;

[0010] The test sound signal within any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested within any sampling period to determine the first position information of the cleaning robot to be tested; and re-collecting the sound signal in the direction of the first position information as the test sound signal.

[0011] According to a cleaning robot offline detection method provided by the present invention, after determining the abnormal sound detection result of the cleaning robot to be tested, the method further includes:

[0012] Obtaining the actual running trajectory of the cleaning robot to be tested during the testing period;

[0013] According to the comparison result between the actual running trajectory and the running trajectory template, a preliminary detection result of the trajectory of the cleaning robot to be tested is determined.

[0014] According to a cleaning robot offline detection method provided by the present invention, after determining a preliminary detection result of the trajectory of the cleaning robot to be tested, the method further includes:

[0015] Acquire an operation video of the cleaning robot to be tested on the test table within any sampling period; the operation video is collected synchronously when the test sound signal within any sampling period is collected;

[0016] Determine the operation scene information of the cleaning robot to be tested within any sampling period according to the operation video, wherein the operation scene information is identification information of a simulated operation scene set on the test table according to the actual operation scene simulation of the cleaning robot;

[0017] Determine a segmented trajectory detection result of the cleaning robot to be tested according to the operation scene information and the actual operation trajectory in each sampling period;

[0018] Based on the preliminary trajectory detection result and the segmented trajectory detection result, a path planning detection result of the cleaning robot to be tested is determined.

[0019] According to a cleaning robot offline detection method provided by the present invention, the actual running trajectory is determined according to the first position information of the cleaning robot to be tested in each sampling period;

[0020] Alternatively, the cleaning robot to be tested is positioned and determined based on the operation video in each sampling period.

[0021] According to a cleaning robot offline detection method provided by the present invention, the cleaning robot to be tested is positioned according to the operation video in each sampling period to obtain the actual operation trajectory, specifically comprising:

[0022] According to the position of the cleaning robot to be tested in each frame of the video image determined by performing frame-by-frame analysis on the running video, a local moving path of the cleaning robot to be tested in each sampling period is obtained;

[0023] Preliminarily splicing all the local movement paths within the sampling period in chronological order to form a preliminary trajectory of the robot within the entire detection cycle;

[0024] According to the position coordinates and orientation of the microphone array in the global coordinate system, the preliminary trajectory is projected into the global coordinate system to obtain the actual running trajectory.

[0025] According to a cleaning robot offline detection method provided by the present invention, after determining the operation scene information of the cleaning robot to be tested within any sampling period according to the operation video, the method further includes:

[0026] Determine the action execution information of the cleaning robot to be tested within any sampling period according to the operation video;

[0027] Determining the action execution requirements of the cleaning robot to be tested within any sampling period according to the operation scene information of the cleaning robot to be tested within any sampling period;

[0028] The action execution information is compared with the action execution requirement to determine a performance test result of the cleaning robot to be tested.

[0029] According to a cleaning robot offline detection method provided by the present invention, the step of determining the action execution information of the cleaning robot to be tested within any sampling period according to the operation video includes:

[0030] Extracting a plurality of key video frames containing the cleaning robot to be tested from the running video;

[0031] Extracting the motion posture features of the cleaning robot to be tested contained in each of the key video frames, and constructing a motion posture feature sequence; the motion posture features include at least one of position information, motion speed information, and angular velocity information of each active part;

[0032] Inputting the motion posture feature sequence into a time series model to obtain the action execution information output by the time series model;

[0033] The time series model is obtained by training with training data having action execution information labels.

[0034] According to a cleaning robot offline detection method provided by the present invention, determining the abnormal sound detection result of the cleaning robot to be tested according to the audio features of the test sound signal in each sampling period within a detection cycle specifically includes:

[0035] All extracted audio features are normalized and fused to obtain a multi-audio feature vector;

[0036] Inputting the multi-audio feature vector into an abnormal sound detection model to obtain the abnormal sound detection result output by the abnormal sound detection model;

[0037] The abnormal sound detection model is obtained after training based on the frequency feature sample vector corresponding to each sound signal sample and the fault label of each sound signal sample.

[0038] According to a cleaning robot offline detection method provided by the present invention, the audio feature includes one or more of a time domain signal feature, a frequency domain signal feature, and a time-frequency domain signal feature;

[0039] The time domain signal feature includes at least one of a time domain signal mean, a time domain signal variance, a time domain signal peak-to-peak value, and a time domain signal kurtosis;

[0040] The frequency domain signal characteristics include at least one of frequency domain energy, frequency center, and frequency domain autocorrelation spectrum in the frequency domain;

[0041] The time-frequency domain signal characteristics include at least one image feature of a short-time Fourier transform image.

[0042] The present invention also provides a cleaning robot offline detection device, comprising:

[0043] A sound receiving unit is used to obtain the audio characteristics of the test sound signal of the cleaning robot to be tested within any sampling period; the test sound signal is collected by a microphone array arranged on the test table;

[0044] An abnormal sound detection unit, used to determine an abnormal sound detection result of the cleaning robot to be tested according to the audio characteristics of the test sound signal in each sampling period within a detection cycle;

[0045] Among them, the test sound signal within any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested within any sampling period to determine the first position information of the cleaning robot to be tested; re-collecting the sound signal in the direction of the first position information as the test sound signal.

[0046] The present invention also provides a cleaning robot offline detection system, comprising: a detection platform, an industrial computer, and a microphone array arranged on the detection platform;

[0047] Wherein, the detection platform is a test platform set up according to the simulation of the actual operation scene of the cleaning robot, and the cleaning robot to be tested runs on the detection platform during the abnormality detection stage;

[0048] The industrial computer executes the cleaning robot offline detection method provided in any one of the above items during the abnormality detection stage.

[0049] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the cleaning robot offline detection method described in any one of the above methods is implemented.

[0050] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the cleaning robot offline detection method as described in any one of the above is implemented.

[0051] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the cleaning robot offline detection method as described in any one of the above is implemented.

[0052] The cleaning robot offline detection method, device and system provided by the present invention do not need to work in a silent box environment. They only need to use a microphone array arranged on the detection table to quickly and accurately locate the vibration fault position, can effectively suppress environmental noise and other test station noise interference, and can accurately locate and characterize abnormal sound faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 This is one of the flow charts of the cleaning robot offline detection method provided by the present invention.

[0055] Figure 2 This is the second flow chart of the cleaning robot offline detection method provided by the present invention.

[0056] Figure 3It is a structural schematic diagram of the cleaning robot offline detection device provided by the present invention.

[0057] Figure 4 It is a structural schematic diagram of the cleaning robot offline detection system provided by the present invention.

[0058] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] It should be noted that in the description of the present invention, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The orientation or position relationship indicated by the terms "upper", "lower" and the like is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0062] Combine the following Figure 1-Figure 5 The present invention describes a cleaning robot offline detection method, device and system.

[0063] Figure 1 This is one of the flow charts of the offline detection method of the cleaning robot provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps:

[0064] Step 101, obtaining audio features of a test sound signal of the cleaning robot to be tested within any sampling period; the test sound signal is collected by a microphone array arranged on a testing platform.

[0065] Among them, the test sound signal within any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested within any sampling period to determine the first position information of the cleaning robot to be tested; re-collecting the sound signal in the direction of the first position information as the test sound signal.

[0066] Step 102: Determine an abnormal sound detection result of the cleaning robot to be tested according to the audio features of the test sound signal in each sampling period within a detection cycle.

[0067] After the cleaning robot is manufactured, abnormal sound detection is required to ensure that its functions and performance meet quality requirements. Traditional manual detection methods have problems such as strong subjectivity, inability to quantify, and susceptibility to environmental interference. Therefore, the present invention provides an automatic detection system that combines a microphone array and a data processing algorithm to detect abnormal sounds before the cleaning robot goes offline.

[0068] The present invention performs offline testing of cleaning robots by controlling each cleaning robot to simulate the operating requirements of various scenarios in reality on the testing table to perform testing operations. Among them, the testing table can be set to a box shape, mainly including components such as a treadmill bottom plate, side walls and a top plate. The treadmill bottom plate can simulate materials in different scenarios such as house floors, which are used for simulated path planning and cleaning of the cleaning robot to be tested. The side walls simulate the walls of the house and prevent the cleaning robot from falling, and the top plate is mainly used to fix test components such as microphone array sensors and lights.

[0069] The microphone array is composed of multiple microphone sensors, and the shape can be a linear array, a circular array, etc. The design frequency response range of the microphone array at least includes the sound frequency range that the cleaning robot may generate, for example, including the sound frequency range of 10HZ-10KHz, and the spatial resolution is not less than 2cm. Among them, the microphone can be a MEMS microphone or an analog microphone, etc., which can capture the sound signal during the operation of the cleaning robot and convert it into an electrical signal for subsequent processing.

[0070] During the operation of the cleaning robot, the sampling frequency of the microphone array will be pre-set to collect sound signals in multiple sampling periods. For example, the sound signal is collected at a sampling frequency of 1 time / second in each detection cycle to ensure that the sound signals of the cleaning robot under test at different operation stages can be captured. The detection cycle refers to the cycle of abnormal sound detection of the cleaning robot under test. Different detection cycles can be set according to the production plan and quality control requirements to ensure product quality.

[0071] The abnormal sound detection process of the cleaning robot to be tested within a detection cycle is described below in conjunction with a specific embodiment:

[0072] In step 101, the robot to be tested is placed at a preset position on a testing platform and the testing process is started, allowing it to run autonomously along a preset trajectory.

[0073] The microphone array is controlled to start collecting the initial sound signal of the cleaning robot under test in any sampling period. The initial sound signal generally includes various sounds of the cleaning robot under test during operation, including normal working sounds and possible abnormal sounds, and may also include other noises in the environment.

[0074] According to the structure of the microphone array itself, the installation position on the test table and the orientation, the collected initial sound signal is processed using a sound source localization algorithm (such as a beamforming algorithm) to determine the position of the cleaning robot to be tested, which is recorded as the first position information.

[0075] The beamforming algorithm uses the phase difference and time difference of the sound signal received by each microphone in the microphone array to calculate the specific position of the sound source (i.e., the cleaning robot to be tested). The beamforming algorithm can be used to determine the first position information, specifically including but not limited to the following steps:

[0076] Assume that the microphone array includes multiple microphones (for example, 8 microphones) for synchronously collecting the initial sound signal of the cleaning robot to be tested in any sampling period.

[0077] When the cleaning robot to be tested generates an initial sound signal, the initial sound signal will propagate to each microphone in the microphone array in the form of a wave.

[0078] Since the distances between each microphone in the microphone array and the cleaning robot to be tested are different, there will be certain phase and time differences between the sound signals received by different microphones. By comparing the phase and time differences of the initial sound signals received by different microphones, the specific time when the initial sound signal reaches each microphone can be calculated.

[0079] Assume that the time when the first microphone and the second microphone receive the initial sound signal is t1 and t2 respectively, and the distance between the first microphone and the second microphone is d, and the speed of sound propagation in the air is v, then the formula d=v (t2-t1) / 2 is used to calculate the distance of the initial sound signal transmitted from the cleaning robot to the midpoint of the first microphone and the second microphone. Similarly, the distance from the initial sound signal to the midpoint of any other two microphones can be calculated.

[0080] Furthermore, the triangulation positioning principle or other positioning algorithms can be used to calculate the three-dimensional coordinates of the cleaning robot to be tested, that is, the first position information, according to the specific time (or distance) when the initial sound signal reaches each microphone.

[0081] For example, on a two-dimensional plane, assuming that the time it is known that the sound signal reaches the three microphones (the first microphone, the second microphone, and the third microphone) are t1, t2, and t3 respectively, and the position coordinates of the three microphones are known to be (x1, y1), (x2, y2), and (x3, y3), respectively, the position coordinates (x, y) of the cleaning robot to be tested can be obtained by solving a set of three linear equations.

[0082] It should be noted that, if in three-dimensional space, at least four microphones are required to solve the position coordinates (x, y, z) of the cleaning robot to be tested.

[0083] Of course, in order to improve the accuracy and stability of the positioning of the beamforming algorithm, the layout, collection direction, weight, etc. of the microphone array can be optimized and adjusted. For example, the accuracy and stability of positioning can be improved by increasing the number of microphones, optimizing the shape and layout of the microphone array, and adopting more advanced signal processing technology. At the same time, according to the needs of the actual application scenario, the collection direction, weight, etc. of the microphone array can be dynamically adjusted to adapt to different sound source locations and environmental noise conditions.

[0084] In summary, by comparing the phase difference and time difference of the initial sound signals received by different microphones, the specific time when the sound signal arrives at each microphone can be calculated, and then the first position information of the sound source (i.e., the cleaning robot to be tested) can be determined.

[0085] After accurately locating the first position information of the robot to be tested, the collection direction or weight of each microphone in the microphone array can be adjusted to more accurately collect the sound signal in the direction of the first position information. The sound signal collected at this time is called a test sound signal for subsequent analysis and processing.

[0086] As another optional embodiment, after the cleaning robot to be tested is positioned according to the initial sound signal, the test sound signal that is of most concern for offline detection can be determined in the following manner:

[0087] First, an initial audio signal (referred to as a first audio signal) collected by the microphone array is received to determine a first angle range of the microphone array according to the audio characteristics of the first audio signal. The microphones constituting each microphone array can be specifically arranged in one of the array arrangement forms such as a linear array, a planar array, a circular array, a spherical array, a spiral array, a multi-mode array, a compact array, etc.

[0088] Furthermore, after acquiring the first audio signal collected by the microphone array, the first angle range where the leakage sound may exist can be roughly determined by analyzing the audio features (such as frequency, intensity, etc.) of the first audio signal and combining some prior knowledge.

[0089] Specifically, the first angle range can be understood as the horizontal angle area and the elevation angle area between the center of the microphone array and the audio signal source that may generate the first audio signal. The horizontal angle area and the elevation angle area refer to the azimuth angle range of the audio signal source relative to the center of the microphone array.

[0090] The horizontal angle usually refers to the angle between the audio signal source on the horizontal plane and a reference direction (such as the center line of the microphone array). The pitch angle refers to the angle between the audio signal source and the horizontal plane where the microphone array is located, that is, the up and down tilt angle of the audio signal source relative to the horizontal plane.

[0091] Therefore, the three-dimensional spatial position of the audio signal source that may generate the first audio signal is jointly determined by the two angle regions, the horizontal angle region and the pitch angle region, which is figuratively referred to as a first angle range in the present invention.

[0092] Then, the microphone array is controlled to collect multiple channels of second audio signals within the first angle range.

[0093] The difference is that the second audio signal is obtained by scanning according to different frequency range and intensity range parameter combinations. The purpose of this step is to capture the audio signal that may be generated by leakage by changing the acquisition parameters such as frequency range and dynamic range, and to provide a data basis for the subsequent beamforming algorithm.

[0094] Dynamic range refers to the signal strength of an audio signal, indicating the range between the weakest and strongest sounds that an audio signal can produce. It can be specifically expressed as the ratio between the minimum audible level (usually the noise level) and the maximum undistorted level, usually in decibels (dB). Therefore, dynamic range describes the range of variation in signal strength of an audio signal.

[0095] In the collected audio signal, sound components of different intensities may contain different information. By adjusting the dynamic range, sound components of specific intensities can be highlighted or suppressed, thereby enhancing the recognition of leakage sounds, especially improving the ability to capture weak leakage sounds, so as to more accurately determine whether there is a leak in a closed container.

[0096] Assuming that the determined first angle range is an angle range of 30° to 60° and the ambient noise is approximately 40dB, then in step 102, the microphone array will be controlled to scan within the angle range of 30° to 60°, first collecting in a low frequency band (such as 20Hz-200Hz) and a high intensity range (such as 100dB), and then gradually adjusting to a high frequency band (such as 2kHz-20kHz) and a low intensity range (such as 30dB), and so on, until all possible frequency range and intensity range parameter combinations are covered.

[0097] Furthermore, the method provided in the above embodiment is used to perform a beamforming process on the collected multiple channels of second audio signals to determine the third audio signal with the strongest signal.

[0098] When the present invention applies beamforming technology to a microphone array, it can focus on a specific angle range and enhance the signal strength within this range. If the audio signal generated by a fault happens to be within this focusing range, its intensity will be significantly enhanced. During the scanning process, by recording the signal strengths under different parameter combinations and comparing these signal strengths, the strongest audio signal can be selected. The third audio signal detected at this time is determined after excluding environmental noise and some other interfering sounds, and is most likely the audio signal generated by the cleaning robot to be tested when it fails, because it is most strongly enhanced within a specific angle and frequency range.

[0099] Finally, the third audio signal is used as the final collected test sound signal.

[0100] Further, the test sound signal can be preprocessed, such as filtering, noise reduction, etc., to improve its signal quality. Then, some audio features of the test sound signal obtained after preprocessing are extracted, and these features can be frequency domain features, such as frequency center, autocorrelation spectrum, etc., and can also include time domain features, such as signal mean, variance, peak-to-peak value, etc., and can also include STFT image features in the time-frequency domain, etc., which are not specifically limited in the present invention.

[0101] After obtaining the audio features of the test sound signal, the audio features of the test sound signal can be distinguished based on the abnormal sound records generated by different types of faults in various components of the cleaning robot (such as driving wheels, motors, sweeping brushes and other active components) recorded in historical data, so as to determine whether the cleaning robot under test makes abnormal sounds during the test. If so, the locations of the different components that make the abnormal sounds and the causes of the faults can be accurately located. A feasible implementation method is given below:

[0102] Before the actual test, a sound feature library containing various known sound features is pre-built. The sound feature library can be obtained by analyzing and extracting a large number of known sound samples.

[0103] The audio features of the extracted test sound signal are compared with each audio feature sample in the abnormal sound feature library (for example, similarity matching is performed). If the audio features match an audio feature sample in the abnormal sound feature library (the similarity matching value is greater than a preset threshold), or exceed the normal sound range, it is determined that there is an abnormal sound in the operating sound of the cleaning robot to be tested collected during this sampling period.

[0104] Furthermore, during the entire detection cycle, the above analysis and comparison are performed on the test sound signal in each sampling period to obtain an analysis result.

[0105] Finally, the analysis results of all sampling periods can be combined to determine the abnormal sound detection results of the cleaning robot to be tested. If an abnormal sound is detected, the fault type, fault component, fault time and other information related to the abnormal sound can be output for further processing by the user or the system.

[0106] The cleaning robot offline detection method provided by the present invention does not need to work in a silent box environment. It only needs to use the microphone array arranged on the detection table to quickly and accurately locate the position of the faulty component, can effectively suppress the interference of environmental noise and other test station noise, and can accurately realize the positioning and qualitative analysis of abnormal sound faults.

[0107] Based on the content of the above embodiment, as an optional embodiment, after determining the abnormal sound detection result of the cleaning robot to be tested, the present invention further includes:

[0108] Obtaining the actual running trajectory of the cleaning robot to be tested during the testing period;

[0109] According to the comparison result between the actual running trajectory and the running trajectory template, a preliminary detection result of the trajectory of the cleaning robot to be tested is determined.

[0110] Optionally, while performing abnormal sound detection, the present invention will determine in real time the position information, average speed, steering angle and other operating parameters of the cleaning robot to be tested based on the first position information of the cleaning robot to be tested obtained in each sampling period, so as to use these operating parameters to generate in real time the actual operating trajectory of the cleaning robot to be tested during the entire detection cycle.

[0111] Specifically, the first position information of each sampling period is determined according to the method provided in the above embodiment, and the first position information may be the coordinates of the robot in a two-dimensional or three-dimensional space.

[0112] The beamforming algorithm uses the phase difference and time difference of the sound signals received by each microphone in the microphone array to calculate the specific location of the sound source (i.e. the cleaning robot to be tested).

[0113] A beamforming algorithm may be used to determine the first position information, including but not limited to the following steps:

[0114] Assume that a microphone array including multiple microphones (for example, 8 microphones) is provided on the testing platform to collect the initial sound signal of the cleaning robot to be tested within any sampling period.

[0115] When the cleaning robot to be tested generates a sound signal, the sound signal will propagate to each microphone in the microphone array in the form of a wave.

[0116] Since the distances between each microphone in the microphone array and the cleaning robot to be tested are different, there will be certain phase and time differences between the sound signals received by different microphones. By comparing the phase and time differences of the sound signals received by different microphones, the specific time when the sound signal reaches each microphone can be calculated.

[0117] Assume that the time when the first microphone and the second microphone receive the sound signal are t1 and t2 respectively, and the distance between the first microphone and the second microphone is d, and the speed of sound propagation in the air is v, then the formula d=v The distance of the sound signal from the cleaning robot to the midpoint of the first microphone and the second microphone can be calculated by (t2-t1) / 2. Similarly, the distance of the sound signal to the midpoint of any other two microphones can be calculated.

[0118] Furthermore, the triangulation positioning principle or other positioning algorithms can be used to calculate the three-dimensional coordinates of the cleaning robot to be tested, that is, the first position information, according to the specific time (or distance) when the sound signal reaches each microphone.

[0119] For example, on a two-dimensional plane, assuming that the time it is known that the sound signal reaches the three microphones (the first microphone, the second microphone, and the third microphone) are t1, t2, and t3 respectively, and the position coordinates of the three microphones are known to be (x1, y1), (x2, y2), and (x3, y3), respectively, the position coordinates (x, y) of the cleaning robot to be tested can be obtained by solving a set of three linear equations.

[0120] It should be noted that, if in three-dimensional space, at least four microphones are required to solve the position coordinates (x, y, z) of the cleaning robot to be tested.

[0121] Of course, in order to improve the accuracy and stability of the positioning of the beamforming algorithm, the layout, collection direction, weight, etc. of the microphone array can be optimized and adjusted. For example, the accuracy and stability of positioning can be improved by increasing the number of microphones, optimizing the shape and layout of the microphone array, and adopting more advanced signal processing technology. At the same time, according to the needs of the actual application scenario, the collection direction, weight, etc. of the microphone array can be dynamically adjusted to adapt to different sound source locations and environmental noise conditions.

[0122] Furthermore, by comparing the first position information of adjacent sampling periods, the displacement of the cleaning robot to be tested between the two periods is calculated. The displacement is divided by the time interval of the sampling period to obtain the average speed of the cleaning robot to be tested in the sampling period.

[0123] By comparing the first position information of adjacent sampling periods, the steering angle of the cleaning robot to be tested between adjacent sampling periods is calculated using the difference in position information. Of course, for the cleaning robot to be tested that is equipped with a steering sensor (such as a gyroscope, accelerometer, etc.), the steering angle can be directly determined based on the steering sensor data.

[0124] Furthermore, the position information, speed, steering angle and other operating parameters of each sampling period can be used as trajectory points, so that each trajectory point contains the complete motion state of the cleaning robot under test in the sampling period. All trajectory points can be smoothed, for example, using Kalman filtering, particle filtering, etc., to obtain real-time trajectories.

[0125] During the entire detection cycle, position information is continuously collected, operating parameters are calculated, and trajectory points are generated. The real-time trajectory of the cleaning robot to be tested is updated and displayed in real time until the end of the entire detection cycle, and the actual operation trajectory during the detection cycle can be obtained.

[0126] Of course, the above determination of the actual running trajectory is only an example and can also be performed in other ways, which is not regarded as the only limitation to the implementation manner of the present invention.

[0127] Furthermore, the actual running trajectory of the cleaning robot to be tested is compared with the defined trajectory template to obtain a comparison result. The comparison method adopted may be to calculate the similarity between the two trajectories (such as using a dynamic time warping algorithm, etc.), or to compare the position deviation and speed deviation of some key points.

[0128] If the deviation between the actual running trajectory and the running trajectory template is within the set tolerance range, the preliminary trajectory detection result is considered qualified; otherwise, it is unqualified.

[0129] The comparison results and judgment basis can also be used as a test report of the preliminary trajectory test results, and the test report can record detailed information such as a comparison chart of the actual running trajectory and the running template trajectory, key deviation values, and test results.

[0130] The cleaning robot offline detection method provided by the present invention collects sound signals through a microphone array, and uses a sound source localization algorithm to determine the first position information of the cleaning robot to be tested in each sampling period, so as to generate the actual operation trajectory in real time, and can make the positioning accurate to the centimeter level. By comparing the actual operation trajectory with the preset operation trajectory template, the comparison result can be used as an important basis for the offline detection of the cleaning robot, thereby improving the accuracy and reliability of the detection. By comparing the actual operation trajectory with the trajectory template, the adaptability of the robot in different environments can be analyzed. And according to the analysis results, the motion parameters and path planning strategies of the cleaning robot can be adjusted to make it better adapt to the complex and changeable working environment.

[0131] Based on the content of the above embodiment, as an optional embodiment, after determining the preliminary detection result of the trajectory of the cleaning robot to be tested, the present invention further includes:

[0132] Acquire an operation video of the cleaning robot to be tested on the test table within any sampling period; the operation video is collected synchronously when the test sound signal within any sampling period is collected;

[0133] Determine the operation scene information of the cleaning robot to be tested within any sampling period according to the operation video, wherein the operation scene information is identification information of a simulated operation scene set on the test table according to the actual operation scene simulation of the cleaning robot;

[0134] Determine a segmented trajectory detection result of the cleaning robot to be tested according to the operation scene information and the actual operation trajectory in each sampling period;

[0135] Based on the preliminary trajectory detection result and the segmented trajectory detection result, a path planning detection result of the cleaning robot to be tested is determined.

[0136] In order to more accurately locate the real-time position of the cleaning robot, the present invention can integrate the microphone array with the camera to collect the sound information and the running video of the cleaning robot to be tested on the test table during each sampling period. The camera can be a monocular camera or a multi-camera combination camera.

[0137] Figure 2 This is the second flow chart of the cleaning robot offline detection method provided by the present invention. Figure 2 The content shown provides a more fine-grained trajectory detection method for the cleaning robot to be tested during offline detection.

[0138] On the test bench, when the cleaning robot to be tested starts running, the video acquisition device (such as an integrated camera) is started synchronously to ensure that the running video of the cleaning robot can be collected in each sampling period. The collection of the running video is synchronized with the collection of the sound signal, that is, the sound signal and the running video in each sampling period are synchronized in time.

[0139] On the test bench, multiple simulated operation scenarios have been set up according to the actual operation scenarios of the cleaning robot, and unique identification information is set for each scenario. Through video analysis technology, such as image recognition, machine learning algorithms, etc., the identification information of the simulated operation scenario in which the cleaning robot to be tested is located during the current sampling period can be identified through the key video frames of the running video.

[0140] The actual running trajectory can be divided into different sections according to the running scenario information in each sampling period, and each section corresponds to a specific simulated running scenario. The actual running trajectory of each section is tested and analyzed in detail, including evaluation of path planning, obstacle avoidance, speed control, etc., to obtain the segmented trajectory detection results.

[0141] Finally, based on the preliminary trajectory detection results (such as the deviation degree of the overall path, speed fluctuation, etc.) and the trajectory segmentation detection results (such as the specific performance in each scene), the path planning evaluation of the cleaning robot to be tested can be comprehensively realized. According to the results of the comprehensive evaluation, the path planning test results of the cleaning robot to be tested are determined. If the overall performance meets the preset standards, it is judged as qualified; otherwise, it is judged as unqualified, and the specific items and reasons for the unqualified are pointed out.

[0142] Suppose there is a household cleaning robot to be tested, which is designed to clean the floor in a home environment. In order to evaluate the path planning ability of the cleaning robot to be tested in different scenarios, the following common household scenarios are simulated on the test bench in advance: living room, bedroom, kitchen and corridor. Each scenario has a unique layout and obstacle settings to simulate the complexity of a real home environment.

[0143] Video analysis technology can be used to identify the simulated operating scenario in which the cleaning robot to be tested is located during each sampling period. For example, when the cleaning robot moves in a spacious and obstacle-free space, we identify it as a living room scenario; when it moves in a narrow space blocked by furniture, we identify it as a bedroom scenario. Based on the identified operating scenario information, the actual operating trajectory of the cleaning robot is further divided into different sections. Each section is then tested and analyzed in detail, including the rationality of path planning, the effectiveness of obstacle avoidance, the stability of speed control, etc. For example, in the kitchen scenario, special attention is paid to how the cleaning robot bypasses obstacles such as the kitchen island or dining table, and its path planning ability in this process is evaluated.

[0144] Assume that according to the trajectory segmentation test results, the cleaning robot performs well in the living room and corridor scenes, can accurately plan the path and move quickly, but in the kitchen scene, due to the complex layout of obstacles such as the dining table, the cleaning robot has a slight path deviation when avoiding obstacles. Therefore, it can be judged that the path planning test result of the cleaning robot is partially qualified, and it is recommended to further optimize the obstacle avoidance algorithm to improve the path planning ability in complex scenes.

[0145] The present invention provides a method for combining sound signals with video signals to achieve a subdivision of the actual operation trajectory and narrow the detection scope to each specific simulated operation scenario, so as to more finely evaluate the performance of the cleaning robot in different environments and more effectively capture local problems that may be overlooked in preliminary detection, such as obstacle avoidance capabilities in front of specific obstacles and path planning in narrow spaces. It can achieve a comprehensive evaluation of the path planning capability of the cleaning robot, focusing on both the overall and local aspects, thereby improving the accuracy of detection.

[0146] At the same time, the present invention adopts a sound and image fusion positioning method that combines a microphone array with a camera. Compared with the method of simply relying on images taken by a camera for real-time positioning, it greatly reduces the pixel requirements for the camera, making the overall cost of the detection equipment lower.

[0147] As an optional embodiment, the present invention specifically provides two implementation methods to complete the drawing of the actual running trajectory.

[0148] One implementation method is to draw the actual running trajectory according to the first position information of the cleaning robot to be tested in each sampling period, and the other implementation method is to determine the location of the cleaning robot to be tested according to the running video in each sampling period. The following will be described in detail, as follows:

[0149] The cleaning robot to be tested is usually equipped with a variety of sensors, such as gyroscopes, accelerometers, odometers, and infrared or laser sensors for obstacle avoidance. These sensors can capture the motion status and environmental information of the cleaning robot to be tested in real time.

[0150] When adopting the first implementation mode, its essence is to construct the actual operation trajectory of the robot by connecting the first position information determined in each sampling period in chronological order. This generation method has been described in detail in the above embodiment and will not be repeated here.

[0151] Based on the contents of the above embodiments, as an optional embodiment, the cleaning robot to be tested is positioned according to the operation video in each sampling period to obtain the actual operation trajectory, specifically including: obtaining the local movement path of the cleaning robot to be tested in each sampling period according to the position of the cleaning robot to be tested in each frame of the video image determined by frame-by-frame analysis of the operation video; preliminarily splicing the local movement paths in all the sampling periods in chronological order to form a preliminary trajectory of the robot in the entire detection cycle; projecting the preliminary trajectory to the global coordinate system according to the position coordinates and orientation of the microphone array in the global coordinates to obtain the actual operation trajectory.

[0152] The following is a detailed description of this part:

[0153] (1) The running video containing the motion trajectory of the cleaning robot to be tested is imported into the video processing system, and the video is preprocessed, including denoising, brightness adjustment, contrast enhancement, etc., to improve the accuracy of subsequent target detection.

[0154] (2) Use a target detection algorithm (such as a deep learning model) to analyze the running video frame by frame and detect the position of the cleaning robot to be tested in each frame of the video image. For continuous frames, a target tracking algorithm (such as KLT tracker, MeanShift, CamShift, etc.) can be used to track the movement of the cleaning robot to be tested to ensure that the cleaning robot to be tested is continuously located in the video sequence. Record the position information of the cleaning robot to be tested detected in each frame of the video image, including coordinates (x, y) and possible orientation information.

[0155] (3) The video is divided into multiple sampling periods according to preset time intervals or specific events (such as the robot completing a cleaning task). For each sampling period, the robot position information recorded in all frames within the period is connected to form the local movement path within the period.

[0156] (4) All local movement paths are preliminarily spliced ​​according to the time sequence of the sampling period to form the preliminary trajectory of the cleaning robot to be tested during the entire detection period. The preliminary trajectory is smoothed to remove abnormal points caused by noise or detection errors, making the trajectory more continuous and smooth.

[0157] Finally, assuming that the microphone array has known position coordinates and orientation in the global coordinate system, the preliminary trajectory can be projected from the video coordinate system to the global coordinate system based on the position and orientation information of the microphone array. This coordinate conversion process mainly involves coordinate transformation (such as translation, rotation) and scale adjustment to ensure the accuracy of the trajectory in the global coordinate system. After projection and conversion, the actual running trajectory of the robot in the global coordinate system is obtained. This method combines video analysis technology and global coordinate system projection to provide high-precision and continuous trajectory information.

[0158] The present invention provides a variety of trajectory tracking methods, and in practical applications, appropriate solutions can be selected according to specific needs and scenarios. For example, when real-time performance and low cost are required, a method of determining the trajectory based on the first position information of each sampling period can be selected, while when high accuracy and environmental perception capabilities are required, a method of extracting motion trajectories based on video analysis technology can be selected.

[0159] Based on the contents of the above embodiments, as an optional embodiment, after determining the operation scene information of the cleaning robot to be tested within any sampling period according to the operation video, the present invention may also include: determining the action execution information of the cleaning robot to be tested within any sampling period according to the operation video; determining the action execution requirements of the cleaning robot to be tested within any sampling period according to the operation scene information of the cleaning robot to be tested within any sampling period; comparing the action execution information and the action execution requirements to determine the performance test result of the cleaning robot to be tested.

[0160] The purpose of offline testing of cleaning robots is to evaluate whether they can adapt to a variety of household environments and efficiently complete cleaning tasks. In order to comprehensively evaluate the performance of the cleaning robot, the present invention uses the operation video during the detection process to perform a detailed analysis. Through the operation video of each sampling period, the operation scene information of the cleaning robot is first analyzed, and different operation scene information is pre-configured with different action execution requirements. In this way, by analyzing the operation video in each sampling period, the actual action execution information is determined, and the performance test of each cleaning robot before it goes offline can be completed by comparing whether the action execution information matches the action execution requirements. The following will be explained with specific examples:

[0161] On the one hand, different sampling periods can be pre-set, such as early morning, afternoon, after dinner and other time periods to set different cleaning needs, corresponding to different action execution requirements; on the other hand, different action execution requirements can be set in combination with the activity habits of family members and home layout, such as daily cleaning (removing dust and hair), deep cleaning (dealing with stubborn stains) and emergency cleaning (such as accidentally spilled liquids); furthermore, a variety of ground materials and obstacles are taken into consideration, such as wooden floors, tiles, carpets, furniture, pets and other possible obstacles, and different action execution requirements are set.

[0162] In this way, in the actual testing process, the performance test results of the cleaning robot to be tested are determined by comparing the action execution information of the cleaning robot to be tested in each sampling period and each operating scenario (such as cleaning demand scenario and ground scenario) with the action execution requirements. If the cleaning robot to be tested can accurately navigate, clean efficiently and adapt to different cleaning needs in all scenarios, its performance can be considered qualified. If the cleaning robot to be tested does not complete the operation in accordance with the action execution requirements in certain specific operating scenarios (such as encountering complex obstacles or special ground materials), its performance is considered unqualified.

[0163] By combining multi-dimensional operation scenario information, action execution information and action execution requirements, the present invention provides a comprehensive and detailed performance detection method for the cleaning robot before it goes offline, which not only helps to ensure that the cleaning robot can perform well in various scenarios, but also provides valuable data support for the subsequent optimization of the cleaning robot.

[0164] As an optional embodiment, determining the action execution information of the cleaning robot to be tested within any sampling period according to the operation video includes:

[0165] Extracting a plurality of key video frames containing the cleaning robot to be tested from the running video;

[0166] Extracting the motion posture features of the cleaning robot to be tested contained in each of the key video frames, and constructing a motion posture feature sequence; the motion posture features include at least one of position information, motion speed information, and angular velocity information of each active part;

[0167] Inputting the motion posture feature sequence into a time series model to obtain the action execution information output by the time series model;

[0168] The time series model is obtained by training with training data having action execution information labels.

[0169] During the offline inspection of cleaning robots, in order to fully evaluate their performance, it is necessary not only to detect their abnormal sounds and trajectory planning capabilities, but also to conduct a detailed analysis of their action execution in specific scenarios. By running the video, the motion status of the cleaning robot at each sampling period can be captured, and then key information can be extracted for performance testing.

[0170] The key video frames containing the cleaning robot's motion can be automatically detected and extracted using video processing techniques, such as frame difference method, background subtraction method, etc. At the same time, key video frames can also be manually selected based on a preset sampling frequency or a specific event (such as the robot completing a certain action).

[0171] For each key video frame, the motion posture features of the cleaning robot are extracted. These motion posture features should be able to fully describe the motion state of the robot in the frame, mainly including the position information of each active part (such as active parts, body posture, etc.), motion speed information and at least one of angular velocity information. These motion posture information can be extracted through image processing technology (such as edge detection, contour extraction, etc.) and computer vision algorithms (such as posture estimation, motion tracking, etc.).

[0172] The motion posture features can be stored and managed in an orderly manner using data structures and algorithms (such as linked lists, arrays, etc.). At the same time, the motion posture features extracted from each key video frame are connected in chronological order to construct a motion posture feature sequence.

[0173] The constructed motion posture feature sequence is input into the time series model to obtain the action execution information output by the model. This information should be able to accurately describe the specific action execution of the cleaning robot under test in each sampling period.

[0174] The time series model can be constructed using machine learning algorithms (such as support vector machines, random forests, etc.) or deep learning models (such as recurrent neural networks, long short-term memory networks, etc.), and then trained using training data with action execution information labels. These training data can come from previous detection processes or simulation experiments, and include the action execution status and corresponding label information of the cleaning robot in different operating scenarios.

[0175] The cleaning robot offline detection method provided by the present invention can make full use of the information in the running video to comprehensively evaluate the action execution of the cleaning robot to be tested in each sampling period. This not only helps to discover potential problems of the cleaning robot in motion control, path planning, etc., but also can provide valuable data support for the subsequent optimization and improvement of the cleaning robot.

[0176] Based on the content of the above embodiment, as an optional embodiment, the present invention provides a method for determining the abnormal sound detection result of the cleaning robot to be tested according to the audio characteristics of the test sound signal in each sampling period within the detection cycle, which specifically includes but is not limited to the following steps:

[0177] First, the detection sound signal of the cleaning robot to be tested in each sampling period is collected on the test table through the microphone array. The detection sound signal can be preprocessed to extract time domain signal features (such as time domain signal mean, time domain signal variance, time domain signal peak-to-peak value, time domain signal kurtosis, etc.), frequency domain signal features (such as frequency domain energy, frequency center, frequency domain autocorrelation spectrum, etc.) and time-frequency domain signal features (such as image features of short-time Fourier transform images, etc.). These audio features can fully reflect the various properties of the sound signal and provide strong data support for subsequent abnormal sound detection.

[0178] After all audio features are extracted, they need to be normalized to eliminate the dimension differences and value range differences between different features. Normalization can make all feature values ​​fall within the same value range, which facilitates subsequent data processing and model training.

[0179] All normalized audio features are fused to obtain a multi-audio feature vector containing multiple feature dimensions. This multi-audio feature vector can fully and accurately represent the sound signal characteristics of the cleaning robot under test in each sampling period.

[0180] Furthermore, the known sound signal samples and their corresponding fault labels are used to train the abnormal sound detection model. These sound signal samples can come from historical detection data, which contain normal sounds and various abnormal sound samples. Each sound signal sample corresponds to an audio feature sample vector and a fault label indicating whether the sample has an abnormal sound. During the training process, the audio feature sample vector and the fault label can be learned through machine learning algorithms (such as support vector machines, neural networks, etc.) so that the model can learn the distinguishing boundary between normal sounds and abnormal sounds. After the training is completed, the obtained abnormal sound detection model can predict whether the cleaning robot to be tested has an abnormal sound based on the input audio feature vector, and output the corresponding abnormal sound detection result.

[0181] Optionally, the audio features include one or more of time domain signal features, frequency domain signal features, and time-frequency domain signal features;

[0182] The time domain signal feature includes at least one of a time domain signal mean, a time domain signal variance, a time domain signal peak-to-peak value, and a time domain signal kurtosis;

[0183] The frequency domain signal characteristics include at least one of frequency domain energy, frequency center, and frequency domain autocorrelation spectrum in the frequency domain;

[0184] The time-frequency domain signal characteristics include at least one image feature of a short-time Fourier transform image.

[0185] It should be noted that the audio features of the test sound signal extracted by the present invention during abnormal sound detection can simultaneously include time domain signal features, frequency domain signal features, and time-frequency domain signal features. This is because the time domain signal features can truly reflect the characteristics of the sound signal changing over time, such as the mean, variance, peak-to-peak value, and kurtosis of the time domain signal. These audio features can capture information such as the amplitude and fluctuation of the sound, which is very helpful for detecting the intensity and stability of the abnormal sound. At the same time, the frequency domain signal features mainly reflect the distribution characteristics of the test sound signal in the frequency domain, such as frequency domain energy, frequency center, and frequency domain autocorrelation spectrum. These frequency domain features can reveal the frequency composition of the sound, which is of great significance for distinguishing different types of sounds (such as normal sounds, different abnormal sounds generated by different components). The time-frequency domain signal features combine the audio information of the time domain and the frequency domain, such as the image features of the short-time Fourier transform (STFT) image. These time-frequency domain features can capture the frequency characteristics of the sound signal changing over time, which is very useful for analyzing the dynamic change process of the sound.

[0186] The present invention takes into account that different types of audio features have different sensitivities to noise. In a noisy environment, some audio features may be greatly affected, while other audio features may remain relatively stable. Therefore, by setting multiple audio features, a richer and more comprehensive sound signal feature vector can be constructed. In the process of abnormal sound detection, the advantages of these feature vectors are comprehensively utilized to improve the accuracy and robustness of detection.

[0187] Figure 3 Schematic diagram of the structure of the cleaning robot offline detection device provided by the present invention. Figure 3 As shown, mainly including but not limited to:

[0188] The sound receiving unit 31 is used to obtain the audio characteristics of the test sound signal of the cleaning robot to be tested in any sampling period; the test sound signal is collected by a microphone array arranged on the test table. The test sound signal in any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested in any sampling period to determine the first position information of the cleaning robot to be tested; and re-collecting the sound signal in the direction of the first position information as the test sound signal.

[0189] The abnormal sound detection unit 32 is used to determine the abnormal sound detection result of the cleaning robot to be tested according to the audio characteristics of the test sound signal in each sampling period within a detection cycle.

[0190] It should be noted that, during operation, the cleaning robot offline detection device provided by the present invention is used to execute the relevant steps of the cleaning robot offline detection method provided by any of the above embodiments.

[0191] The cleaning robot offline detection device provided by the present invention does not need to work in a silent box environment. It can quickly and accurately locate the vibration fault position by using only the microphone array arranged on the detection table. It can effectively suppress the interference of environmental noise and other test station noise, and can accurately realize the positioning and characterization of abnormal sound faults.

[0192] Figure 4 Schematic diagram of the structure of the cleaning robot offline detection system provided by the present invention. Figure 4 As shown, it mainly includes but is not limited to a testing platform 1, an industrial computer 4 and a microphone array 2 arranged on the testing platform;

[0193] The detection platform 1 is a test platform set up according to the simulation of the actual operation scene of the cleaning robot 3, and the cleaning robot 3 runs on the detection platform 1 during the abnormality detection stage;

[0194] The industrial computer 4 executes the cleaning robot offline detection method provided by any of the above embodiments during the abnormality detection stage.

[0195] Wherein, the test bench 1 can be set in a box shape, mainly including a treadmill bottom plate, side walls and a top plate. The treadmill bottom plate is a material that simulates the floor of a house and is used for the cleaning robot to simulate path planning and cleaning. The side walls are mainly used to simulate walls and prevent the cleaning robot from falling. The top plate is mainly used to fix the microphone array to measure the trajectory and noise of the cleaning robot in real time. In particular, the test bench 1 can also simulate some special scenes in the room as needed to better test the working performance of the cleaning robot.

[0196] The microphone array 2 is composed of a plurality of microphone sensor sets. The shape of the microphone unit can be a linear array, a circular array, a rectangular array, a spherical array or a cycloid array, etc. The microphone can be a MEMS microphone or an analog microphone. The design frequency response range of the microphone array 2 needs to include the range of 10HZ-10KHz, and the spatial resolution is not less than 2cm.

[0197] In particular, in order to more accurately locate the real-time position of the cleaning robot 3, the microphone array can be directly integrated with the camera, which can be a monocular camera or a multi-camera combination camera. The microphone array is mainly used to locate the current position of the cleaning robot and to collect sound in a directional manner in the current position direction of the cleaning robot, so as to reduce interference from noise sources in other areas other than the non-target cleaning robot.

[0198] The industrial computer 4 is mainly used to communicate with the microphone array and the camera, receive sound signals and visual operation videos, and is also used to deploy data processing algorithms and detection software to quickly give the function and performance test results of the cleaning robot. Among them, the detection software mainly includes the function of scanning the code or ID recognition of the cleaning robot under test, the visualization function of the cleaning robot based on the real image background of the detection platform (such as image fusion technology based on the video frames captured in the operation video of the detection platform and the IMU or visual detection trajectory of the cleaning robot), the visualization and fault diagnosis function of the noise time-frequency analysis of the cleaning robot, the function of excluding and storing the defective products separately after uploading the result data to the MES system or directly linking with the automatic actuator, the statistical analysis function of test data, the automatic report generation function, the historical data review function, the authority management function, etc. Figure 4 The display device 5 in the figure mainly provides an interface for human-machine interaction and does not affect the normal operation of the core cleaning robot detection system.

[0199] The cleaning robot offline detection system provided by the present invention does not need to work in a silent box environment. It can quickly and accurately locate the vibration fault position by using only the microphone array arranged on the detection table. It can effectively suppress the interference of environmental noise and noise from other test stations, and can accurately locate and characterize abnormal sound faults.

[0200] Figure 5 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the offline detection method of the cleaning robot, the method comprising: obtaining the audio characteristics of the test sound signal of the cleaning robot to be tested in any sampling period; the test sound signal is collected by a microphone array arranged on the detection table; according to the audio characteristics of the test sound signal in each sampling period in the detection cycle, determining the abnormal sound detection result of the cleaning robot to be tested; the test sound signal in any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested in any sampling period to determine the first position information of the cleaning robot to be tested; re-collecting the sound signal in the direction of the first position information as the test sound signal.

[0201] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0202] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the cleaning robot offline detection method provided in the above-mentioned embodiments, the method including: obtaining the audio characteristics of a test sound signal of the cleaning robot to be tested within any sampling period; the test sound signal is collected by a microphone array arranged on a detection table; according to the audio characteristics of the test sound signal within each sampling period within the detection cycle, determining the abnormal sound detection result of the cleaning robot to be tested; the test sound signal within any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested within the any sampling period to determine the first position information of the cleaning robot to be tested; and re-collecting the sound signal in the direction of the first position information as the test sound signal.

[0203] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the cleaning robot offline detection method provided in the above-mentioned embodiments, the method comprising: obtaining the audio characteristics of a test sound signal of the cleaning robot to be tested within any sampling period; the test sound signal is collected by a microphone array arranged on a detection table; according to the audio characteristics of the test sound signal within each sampling period within the detection cycle, determining the abnormal sound detection result of the cleaning robot to be tested; the test sound signal within any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested within the any sampling period to determine the first position information of the cleaning robot to be tested; and re-collecting the sound signal in the direction of the first position information as the test sound signal.

[0204] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0205] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cleaning robot offline detection method, characterized in that: include: Acquire the audio characteristics of the test sound signal of the cleaning robot to be tested within any sampling period; The test sound signal is collected by a microphone array arranged on the test platform; Determine the abnormal sound detection result of the cleaning robot to be tested according to the audio features of the test sound signal in each sampling period within the detection cycle; The test sound signal in any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested in any sampling period to determine the first position information of the cleaning robot to be tested; re-collecting the sound signal in the direction of the first position information as the test sound signal; After determining the abnormal sound detection result of the cleaning robot to be tested, the method further includes: Obtaining the actual running trajectory of the cleaning robot to be tested during the testing period; According to the comparison result between the actual running trajectory and the running trajectory template, a preliminary detection result of the trajectory of the cleaning robot to be tested is determined.

2. The cleaning robot offline detection method according to claim 1, characterized in that: After determining the preliminary detection result of the trajectory of the cleaning robot to be tested, the method further includes: Acquire an operation video of the cleaning robot to be tested on the test table within any sampling period; the operation video is collected synchronously when the test sound signal within any sampling period is collected; Determine the operation scene information of the cleaning robot to be tested within any sampling period according to the operation video, wherein the operation scene information is identification information of a simulated operation scene set on the test table according to the actual operation scene simulation of the cleaning robot; Determine a segmented trajectory detection result of the cleaning robot to be tested according to the operation scene information and the actual operation trajectory in each sampling period; Based on the preliminary trajectory detection result and the segmented trajectory detection result, a path planning detection result of the cleaning robot to be tested is determined.

3. The cleaning robot offline detection method according to claim 2, characterized in that: The actual running trajectory is determined according to the first position information of the cleaning robot to be tested in each sampling period; Alternatively, the cleaning robot to be tested is positioned and determined based on the operation video in each sampling period.

4. The cleaning robot offline detection method according to claim 3, characterized in that: Positioning the cleaning robot to be tested according to the operation video in each sampling period to obtain the actual operation trajectory specifically includes: According to the position of the cleaning robot to be tested in each frame of the video image determined by performing frame-by-frame analysis on the running video, a local moving path of the cleaning robot to be tested in each sampling period is obtained; Preliminarily splicing all the local movement paths within the sampling period in chronological order to form a preliminary trajectory of the robot within the entire detection cycle; According to the position coordinates and orientation of the microphone array in the global coordinate system, the preliminary trajectory is projected into the global coordinate system to obtain the actual running trajectory.

5. The cleaning robot offline detection method according to claim 2, characterized in that: After determining the operation scene information of the cleaning robot to be tested within any sampling period according to the operation video, the method further includes: Determine the action execution information of the cleaning robot to be tested within any sampling period according to the operation video; Determining the action execution requirements of the cleaning robot to be tested within any sampling period according to the operation scene information of the cleaning robot to be tested within any sampling period; The action execution information is compared with the action execution requirement to determine a performance test result of the cleaning robot to be tested.

6. The cleaning robot offline detection method according to claim 5, characterized in that: The determining, according to the operation video, the action execution information of the cleaning robot to be tested within any sampling period includes: Extracting a plurality of key video frames containing the cleaning robot to be tested from the running video; Extracting the motion posture features of the cleaning robot to be tested contained in each of the key video frames, and constructing a motion posture feature sequence; the motion posture features include at least one of position information, motion speed information, and angular velocity information of each active part; Inputting the motion posture feature sequence into a time series model to obtain the action execution information output by the time series model; The time series model is obtained by training with training data having action execution information labels.

7. The cleaning robot offline detection method according to claim 1, characterized in that: Determining the abnormal sound detection result of the cleaning robot to be tested according to the audio feature of the test sound signal in each sampling period within the detection cycle specifically includes: All extracted audio features are normalized and fused to obtain a multi-audio feature vector; Inputting the multi-audio feature vector into an abnormal sound detection model to obtain the abnormal sound detection result output by the abnormal sound detection model; The abnormal sound detection model is obtained after training based on the frequency feature sample vector corresponding to each sound signal sample and the fault label of each sound signal sample.

8. The cleaning robot offline detection method according to claim 7, characterized in that: The audio features include one or more of time domain signal features, frequency domain signal features, and time-frequency domain signal features; The time domain signal feature includes at least one of a time domain signal mean, a time domain signal variance, a time domain signal peak-to-peak value, and a time domain signal kurtosis; The frequency domain signal characteristics include at least one of frequency domain energy, frequency center, and frequency domain autocorrelation spectrum in the frequency domain; The time-frequency domain signal characteristics include at least one image feature of a short-time Fourier transform image.

9. A cleaning robot offline detection device, characterized in that: include: A sound receiving unit, used to obtain the audio characteristics of the test sound signal of the cleaning robot to be tested within any sampling period; The test sound signal is collected by a microphone array arranged on the test platform; An abnormal sound detection unit, used to determine an abnormal sound detection result of the cleaning robot to be tested according to the audio characteristics of the test sound signal in each sampling period within a detection cycle; After determining the abnormal sound detection result of the cleaning robot to be tested, the method further includes: Obtaining the actual running trajectory of the cleaning robot to be tested during the testing period; Determining a preliminary detection result of the trajectory of the cleaning robot to be tested according to a comparison result between the actual running trajectory and the running trajectory template; Among them, the test sound signal within any sampling period is collected based on the following steps: collecting the initial sound signal of the cleaning robot to be tested within any sampling period to determine the first position information of the cleaning robot to be tested; re-collecting the sound signal in the direction of the first position information as the test sound signal.

10. A cleaning robot offline detection system, characterized in that: include: A testing platform, an industrial computer, and a microphone array arranged on the testing platform; Wherein, the detection platform is a test platform set up according to the simulation of the actual operation scene of the cleaning robot, and the cleaning robot to be tested runs on the detection platform during the abnormality detection stage; The industrial computer executes the cleaning robot offline detection method as described in any one of claims 1 to 8 during the abnormality detection stage.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the cleaning robot offline detection method according to any one of claims 1 to 8 is implemented.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the cleaning robot offline detection method according to any one of claims 1 to 8 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the cleaning robot offline detection method according to any one of claims 1 to 8 is implemented.

Images