Quality Detection Method and Device for Image Acquisition of Humanoid Robots

Abstract

An embodiment of the present application provides a quality detection method and device for image acquisition of a humanoid robot. The method includes: obtaining image acquisition data of the humanoid robot, where the image acquisition data includes a plurality of consecutive image frames; determining time differences corresponding to multiple pairs of adjacent frames respectively according to the timestamps of the multiple image frames; if the multiple time differences are all less than a preset duration, determining similarity values corresponding to the multiple pairs of adjacent frames respectively; and determining a quality inspection result of the image acquisition data according to the multiple similarity values and a preset condition, where the preset condition is related to a preset initial threshold and incremental thresholds corresponding to the multiple pairs of adjacent frames respectively. The method provided in this embodiment can reduce costs, improve efficiency, and ensure accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of humanoid robots, and in particular, to a method and device for quality detection of image acquisition for humanoid robots. Background Art

[0002] After image data of a humanoid robot is collected, it is necessary to perform quality detection on the collected image data to ensure that the collected image data is all qualified and available, and does not affect subsequent operations such as model training.

[0003] In the related art, the quality of the collected image data can be detected by manual quality inspection. However, by manually checking the pictures, it requires a large amount of time cost and labor cost, with low efficiency and low accuracy. Summary of the Invention

[0004] The embodiments of the present application provide a method and device for quality detection of image acquisition for humanoid robots to reduce costs and improve the efficiency and accuracy of quality inspection.

[0005] In a first aspect, the embodiments of the present application provide a method for quality detection of image acquisition for humanoid robots, including:

[0006] Obtain the image acquisition data of the humanoid robot; the image acquisition data includes a plurality of consecutive image frames;

[0007] Determine the time differences respectively corresponding to multiple pairs of adjacent frames according to the timestamps of the multiple image frames;

[0008] If the multiple time differences are all less than a preset duration, determine the similarity values respectively corresponding to the multiple pairs of adjacent frames;

[0009] Determine the quality inspection result of the image acquisition data according to the multiple similarity values and a preset condition; the preset condition is related to a preset initial threshold and an increment threshold respectively corresponding to the multiple pairs of adjacent frames.

[0010] In a possible design, the determining the similarity values respectively corresponding to the multiple pairs of adjacent frames includes:

[0011] Calculate the similarity values respectively corresponding to the multiple pairs of adjacent frames in sequence according to the chronological order of the timestamps of the multiple image frames;

[0012] The determining the quality inspection result of the image acquisition data according to the multiple similarity values and a preset condition includes:

[0013] Compare the similarity value of the current adjacent frame with a first threshold; the first threshold is determined based on the preset initial threshold and the increment threshold corresponding to the previous adjacent frame;

[0014] If the similarity value of the current adjacent frame is less than the first threshold, compare the similarity value of the current adjacent frame with the preset initial threshold;

[0015] If the similarity of the current adjacent frame is less than the preset initial threshold, determine the current adjacent frame as an abnormal frame, increment the total number of current abnormal frames by one to obtain a new total number of abnormal frames, and add the increment threshold corresponding to the current adjacent frame to the current total amount of abnormal increment thresholds to obtain a new total amount of abnormal increment thresholds;

[0016] Determine the quality inspection result of the image acquisition data according to the new total number of abnormal frames and the new total amount of abnormal increment thresholds.

[0017] In a possible design, the determining the quality inspection result of the image acquisition data according to the new total number of abnormal frames and the new total amount of abnormal increment thresholds includes:

[0018] If the total number of abnormal frames is less than a first preset value and the total amount of abnormal increment thresholds is less than a second preset value, determine the quality inspection result of the image acquisition data as qualified.

[0019] In a possible design, the method further includes:

[0020] If the similarity value of the current adjacent frame is greater than or equal to the first threshold, determine a new first threshold based on the preset initial threshold and the increment threshold corresponding to the current adjacent frame;

[0021] Determine the next pair of adjacent frames of the current adjacent frame as the new current adjacent frame;

[0022] Compare the similarity value of the new current adjacent frame with the new first threshold.

[0023] In a possible design, the determining the quality inspection result of the image acquisition data according to multiple similarity values and preset conditions includes:

[0024] For each of the multiple similarity values, determine a second threshold corresponding to the similarity value according to the preset initial threshold and the increment threshold corresponding to the similarity value;

[0025] If each similarity value is greater than the corresponding second threshold, determine the quality inspection result of the image acquisition data as qualified.

[0026] In a possible design, construct a time series from multiple pairs of adjacent frames in the chronological order of the adjacent frames; wherein, the increment thresholds corresponding to multiple pairs of adjacent frames in the time series show an increasing change.

[0027] In a possible design, the time series is sequentially divided into an initial segment, an intermediate segment, and an end segment in chronological order; wherein, the first change rate of the increment thresholds respectively corresponding to multiple pairs of adjacent frames in the initial segment and the third change rate of the increment thresholds respectively corresponding to multiple pairs of adjacent frames in the end segment are both smaller than the second change rate of the increment thresholds respectively corresponding to multiple pairs of adjacent frames in the intermediate segment.

[0028] In a possible design, the expression of the increment threshold is:

[0029]

[0030] Wherein, is the increment threshold, is the preset initial threshold, is the maximum threshold, is the adjustment factor constant, T is the total number of image frames, and n is the position of the previous frame in multiple adjacent frames in the order of time stamps.

[0031] In a possible design, the determination of the similarity values respectively corresponding to multiple pairs of adjacent frames includes:

[0032] Based on the cosine similarity algorithm, determine the similarity values respectively corresponding to multiple pairs of adjacent frames.

[0033] In a second aspect, an embodiment of the present application provides a quality detection device for image acquisition of a humanoid robot, including:

[0034] An acquisition module, configured to acquire image acquisition data of the humanoid robot; the image acquisition data includes multiple consecutive image frames;

[0035] A time difference determination module, configured to determine the time differences respectively corresponding to multiple pairs of adjacent frames according to the timestamps of multiple said image frames;

[0036] A similarity determination module, configured to determine the similarity values respectively corresponding to multiple pairs of adjacent frames if multiple said time differences are all smaller than a preset duration;

[0037] A quality inspection module, configured to determine the quality inspection result of the image acquisition data according to multiple said similarity values and a preset condition; the preset condition is related to the preset initial threshold and the increment thresholds respectively corresponding to multiple pairs of said adjacent frames.

[0038] In a third aspect, an embodiment of the present application provides a quality detection device for image acquisition of a humanoid robot, including: at least one processor and a memory;

[0039] The memory stores computer execution instructions;

[0040] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method described in the first aspect above and various possible designs of the first aspect.

[0041] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method described in the first aspect above and various possible designs of the first aspect is implemented.

[0042] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect above and various possible designs of the first aspect.

[0043] The quality detection method and device for image acquisition of a humanoid robot provided in this embodiment include obtaining image acquisition data of the humanoid robot. The image acquisition data includes multiple consecutive image frames. According to the timestamps of the multiple image frames, the time differences corresponding to multiple pairs of adjacent frames are determined. If all the multiple time differences are less than a preset duration, the similarity values corresponding to the multiple pairs of adjacent frames are determined. According to the multiple similarity values and a preset condition, the quality inspection result of the image acquisition data is determined. The preset condition is related to a preset initial threshold and an increment threshold corresponding to multiple pairs of adjacent frames. The method provided in this embodiment first determines whether the time difference between adjacent frames is small enough based on the timestamps of the image frames to ensure that there is no frame loss in the image acquisition data. Furthermore, the similarity values of adjacent frames are calculated, and the quality inspection result of the image acquisition data is automatically determined based on the similarity values, the preset initial threshold, and the increment threshold, which can reduce costs, improve efficiency, and ensure accuracy. Description of the Drawings

[0044] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments that conform to the present application, and are used together with the specification to explain the principles of the present application.

[0045] Figure 1 It is a schematic diagram of the scenario of the quality detection method for image acquisition of a humanoid robot provided by an embodiment of the present application;

[0046] Figure 2 It is a flowchart of the quality detection method for image acquisition of a humanoid robot provided by an embodiment of the present application Figure 1 ;

[0047] Figure 3 It is a schematic diagram of the change curve of the increment threshold provided by an embodiment of the present application;

[0048] Figure 4Flow schematic of the quality detection method for humanoid robot image acquisition provided by the embodiments of the present application Figure 2 ;

[0049] Figure 5 Structural schematic diagram of the quality detection device for humanoid robot image acquisition provided by the embodiments of the present application;

[0050] Figure 6 Hardware structural schematic diagram of the quality detection device for humanoid robot image acquisition provided by the embodiments of the present application.

[0051] Through the above drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Specific embodiments

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0053] It should be noted that the data quality detection methods and devices provided by the present application can be used in the technical field of humanoid robots, and can also be used in any field other than the technical field of humanoid robots. The application fields of the data quality detection methods and devices provided by the present application are not limited.

[0054] In order to enable humanoid robots to achieve more efficient, intelligent, and safe operations and lay a technical foundation for future wide applications, it is necessary to collect data for humanoid robots. Taking the data collection when a humanoid robot performs tasks through its upper limbs (such as grasping an object on a table, washing dishes, etc.) as an example, image data of the humanoid robot during the task execution can be collected through a vision sensor (such as a camera dedicated to data collection). During the data collection process by the collector, image data may be lost or damaged due to improper operation, or frames may be lost when the image data is saved due to network communication delay problems, or frames may be lost or the image may be damaged due to problems with the internal sensor of the camera. These problems are very likely to occur in daily image data collection. Once the above problems occur, the collected image data becomes unusable, thus affecting subsequent work such as model training. Therefore, after the image data collection is completed, it is necessary to perform quality detection on the collected image data to ensure that the collected image data of the humanoid robot is all qualified and usable.

[0055] In the related art, the quality of the collected image data can be detected by means of manual quality inspection. However, by manually checking the pictures, it is often necessary for the operator to highly concentrate for a long time to distinguish whether adjacent frame pictures are continuous and to judge whether there is a problem of frame loss in the camera images, resulting in a large amount of time cost and labor cost, and the quality of the inspection is not easy to guarantee.

[0056] To solve the above technical problems, the inventors of the present application have found through research that by analyzing the similarity of adjacent image frames in a sequence of continuously captured image frames and analyzing the time difference between the timestamps of adjacent frames, it is possible to automatically detect problems such as frame loss or damage based on the similarity results and time difference results, which can not only save labor costs but also improve the efficiency and quality of quality inspection. Based on this, an embodiment of the present application provides a quality detection method for humanoid robot image acquisition.

[0057] Figure 1 The scene schematic diagram of the quality detection method for humanoid robot image acquisition provided by the embodiment of the present application. As Figure 1 shown, the vision sensor 101 is connected to the quality inspection device 103. Among them, the vision sensor 101 is used to capture the actions, task objects, and surrounding environment of the humanoid robot 102 during the task execution of the humanoid robot 102 to generate image acquisition data, and transmit the image acquisition data to the quality inspection device 103. The quality inspection device 103 is used to automatically perform quality inspection on the image acquisition data. The quality inspection device 103 can be a terminal device or a server. The vision sensor 101 can be a device specifically used for image acquisition. When collecting data, the vision sensor 101 can be installed on the humanoid robot 102, for example, installed on the head of the humanoid robot 102 to facilitate the shooting of upper limb actions. The acquisition frequency of the vision sensor 101 can be set artificially. Image acquisition is performed at an acquisition frequency of 30 frames per second. Assuming that it takes 20 seconds to complete a specific task, 600 frames of image acquisition data can be obtained for this task.

[0058] In the specific implementation process, when the humanoid robot 102 performs a specific task (such as putting the apple on the table into the plate, or pouring a glass of water into the water cup, etc.), the visual sensor 101 captures the actions of the humanoid robot 102, the task object, and the surrounding environment to generate image acquisition data, and transmits the image acquisition data to the quality inspection device 103. The quality inspection device 103 obtains the image acquisition data of the humanoid robot. The image acquisition data includes multiple consecutive image frames. According to the timestamps of the multiple image frames, the time differences corresponding to multiple pairs of adjacent frames are determined respectively. If multiple time differences are all less than the preset duration, the similarity values corresponding to multiple pairs of adjacent frames are determined respectively. According to the multiple similarity values and the preset conditions, the quality inspection result of the image acquisition data is determined. The preset conditions are related to the preset initial threshold and the incremental thresholds corresponding to multiple pairs of adjacent frames respectively. The method provided in this embodiment first determines whether the time difference between adjacent frames is small enough based on the timestamps of the image frames to ensure that there is no frame loss in the image acquisition data. Furthermore, the similarity values of adjacent frames are calculated, and the quality inspection result of the image acquisition data is automatically determined based on the similarity values, the preset initial threshold, and the incremental thresholds, which can reduce costs, improve efficiency, and ensure accuracy.

[0059] The technical solution of the present application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0060] Figure 2 It is a schematic flow chart of the quality inspection method for image acquisition of a humanoid robot provided by an embodiment of the present application Figure 1 As Figure 2 shown, the method includes:

[0061] 201. Obtain the image acquisition data of the humanoid robot; the image acquisition data includes multiple consecutive image frames.

[0062] The execution subject of this embodiment can be a terminal device or a server, such as Figure 1 the quality inspection device 103 shown.

[0063] In this embodiment, the image acquisition data can be obtained by a visual sensor dedicated to image acquisition of a humanoid robot, such as a camera. During the image acquisition process, the visual sensor can be installed on the humanoid robot.

[0064] Specifically, during the process of the humanoid robot performing a specific task (such as a pouring task, an apple plate loading task, etc.), the camera dedicated to data acquisition installed on the head of the humanoid robot takes pictures based on a preset frequency and sends the obtained image acquisition data (multiple consecutive image frames) to the quality inspection device.

[0065] 202. Determine the time differences corresponding to multiple pairs of adjacent frames according to the timestamps of the multiple image frames.

[0066] Specifically, when each image frame is captured, a timestamp will be recorded for each image frame. Based on the timestamps corresponding to two adjacent image frames respectively, the time difference between adjacent frames can be calculated. Considering the convenience and efficiency of time difference calculation, the condition that the time difference meets the requirement is used as the primary condition for determining whether the image acquisition data is qualified. If the time differences do not meet the conditions, it can be directly determined that the image acquisition data is unqualified, and subsequent similarity calculations do not need to be performed to avoid wasting computing power resources. Among them, the multiple pairs of adjacent frames are consecutive adjacent frames.

[0067] Exemplarily, assuming there are 5 consecutive image frames, then image frame 1 and 2 are a pair of adjacent frames, image frame 2 and 3 are a pair of adjacent frames, image frame 3 and 4 are a pair of adjacent frames, and image frame 4 and 5 are a pair of adjacent frames. The difference between the timestamp of image frame 2 and the timestamp of image frame 1 can be used as the time difference corresponding to the pair of adjacent frames of image frame 1 and 2.

[0068] 203. If multiple time differences are all less than a preset duration, determine the similarity values corresponding to multiple pairs of adjacent frames respectively.

[0069] Specifically, the magnitude of the time difference can characterize the continuity of the image frames. If the time difference is too large, it indicates that there may be a frame loss phenomenon. Therefore, a frame rate threshold can be set as the preset duration. If the time differences corresponding to multiple pairs of adjacent frames are all less than the preset duration, it indicates that there is no frame loss phenomenon. In the case of determining no frame loss, the similarity between adjacent frames can be further determined to further determine the image quality.

[0070] In some embodiments, various algorithms can be used to calculate the similarity of adjacent frames, such as the cosine similarity algorithm, the histogram comparison algorithm, the structural similarity algorithm, etc.

[0071] In one implementable manner, determining the similarity values corresponding to multiple pairs of adjacent frames respectively may include: determining the similarity values corresponding to multiple pairs of adjacent frames based on the cosine similarity algorithm. The method of this embodiment, by determining the similarity of multiple pairs of adjacent frames based on the cosine similarity algorithm, is simple and efficient in calculation, can measure the similarity of directions between frames from the perspective of the vector space, is not affected by the vector length, and can effectively reflect the similarity degree of the content features of adjacent frames, providing an accurate reference for subsequent processing.

[0072] 204. Determine the quality inspection result of the image acquisition data according to multiple similarity values and preset conditions; the preset conditions are related to a preset initial threshold and incremental thresholds corresponding to multiple pairs of adjacent frames.

[0073] Specifically, after determining the similarity values of multiple pairs of adjacent frames, the similarity values can be analyzed based on a preset condition. If the condition is met, it can be determined that the quality of the image acquisition data is qualified and can be normally applied, such as being used as training data for a humanoid robot model. If the condition is not met, it can be determined that the quality of the image acquisition data is unqualified and can be directly discarded.

[0074] In some embodiments, there are multiple ways to determine the quality inspection result. In one implementable way, the similarity value can be directly compared with a preset initial threshold. If the similarity values of all adjacent frames are greater than the preset initial threshold, it can be determined that the quality of the image acquisition data is qualified. Otherwise, it is determined that the quality of the image acquisition data is unqualified.

[0075] In another implementable way, in order to more strictly control the quality of the image acquisition data, different incremental thresholds can be preset for different image frames based on the characteristics of the image acquisition for a specific task. Then, based on the preset initial threshold and the incremental threshold, the second threshold corresponding to different pairs of adjacent frames is determined. If the similarity values of all adjacent frames meet the corresponding second threshold, it can be determined that the image acquisition quality is qualified. Among them, the above-mentioned second threshold is the sum of the preset initial threshold and the incremental threshold.

[0076] Specifically, according to the multiple similarity values and the preset condition, determining the quality inspection result of the image acquisition data may include: for each similarity value among the multiple similarity values, determining the second threshold corresponding to the similarity value according to the preset initial threshold and the incremental threshold corresponding to the similarity value; if all similarity values are greater than the corresponding second threshold, determining the quality inspection result of the image acquisition data as qualified. The method of this embodiment can flexibly control the quality standard according to the image features by setting incremental thresholds for different image frames based on the characteristics of a specific task, combining the preset initial threshold to determine the second threshold to inspect the image acquisition data, and more strictly and accurately evaluate the data quality to ensure that the image acquisition quality meets the requirements of a specific task.

[0077] Exemplarily, assume that the incremental thresholds corresponding to consecutive image frames 1, 2, and 3 are δ1, δ2, and δ3 respectively. The preset initial threshold is δ0. Then the second threshold for the adjacent frames (image frames 1 and 2) is δ0 + δ1, and the second threshold for the adjacent frames (image frames 2 and 3) is δ0 + δ2.

[0078] In some embodiments, considering that in the initial stage of a humanoid robot performing various tasks, the number of valid images is usually small. For example, when grasping an apple, among the 600-frame image acquisition data, perhaps the first 50 frames have not even touched the apple yet. Therefore, in terms of importance, the importance of the images in the initial stage is lower than that of the 550 frames acquired in the subsequent stage. Thus, the threshold value of the similarity value can be set lower than that in the subsequent stage. Therefore, the incremental threshold can be set to increase incrementally. Specifically, multiple pairs of adjacent frames can be constructed into a time series in the chronological order of adjacent frames; among them, the incremental thresholds corresponding to multiple pairs of adjacent frames in the time series show an increasing change. The method provided in this embodiment can dynamically adjust the threshold according to the importance of the images in different task stages by constructing adjacent frames into a time series with an increasing incremental threshold. The low threshold in the initial stage can avoid misjudgment, and the subsequent stage is strictly controlled, improving the rationality and accuracy of data quality evaluation.

[0079] In some embodiments, considering that if the incremental threshold keeps increasing, the threshold for comparison with the similarity value will gradually become larger, even approaching or exceeding 1, which is unreasonable and will also lead to misjudgment because the similarity value fails to reach the threshold and the data is judged as unqualified. Therefore, the time series can be sequentially divided into an initial segment, a middle segment, and an end segment according to the time order; among them, the first change rate of the incremental thresholds corresponding to multiple pairs of adjacent frames in the initial segment and the third change rate of the incremental thresholds corresponding to multiple pairs of adjacent frames in the end segment are both smaller than the second change rate of the incremental thresholds corresponding to multiple pairs of adjacent frames in the middle segment. Thus, the effect of slow increase first, then rapid increase, and then slow increase is achieved. The method provided in this embodiment can avoid unreasonable increase of the threshold, reduce misjudgment, and more scientifically evaluate the quality of image acquisition data by dividing the time series into an initial segment, a middle segment, and an end segment and setting incremental thresholds with different change rates to achieve the effect of slow-fast-slow threshold increase. Optionally, the average value of the first change rate of the incremental thresholds corresponding to multiple pairs of adjacent frames in the initial segment can be greater than the average value of the third change rate of the incremental thresholds corresponding to multiple pairs of adjacent frames in the end segment. This embodiment can increase the threshold at a slightly faster speed in the initial stage of the task by setting the average value of the incremental threshold change rate in the initial segment to be greater than that in the end segment, preparing for the threshold increase in the middle segment; the slow change in the end segment can stabilize the threshold control and prevent it from increasing too much to avoid misjudgment, making the data quality evaluation more in line with the task rhythm and ensuring the stability and reliability of data quality.

[0080] In some embodiments, for the convenience of calculation, the inventor found that the incremental threshold can be defined through the following function expression related to the position of the image frame to achieve the effect of slow increase first, then rapid increase, and then slow increase. The expression of the incremental threshold is:

[0081] (1)

[0082] Among them, is the incremental threshold, is the preset initial threshold, is the maximum threshold, is the adjustment factor constant, T is the total number of image frames, and n is the position of the previous frame in adjacent frames among multiple image frames in the order of time stamps.

[0083] Exemplarily, as Figure 3 shown, the acquisition process is divided into three stages: an initial stage a, an intermediate stage b, and an end stage c. Assume that T is 50 frames. Before the 15th image frame is the starting stage a, and the incremental threshold increases slowly. Between the 15th image frame and the 35th image frame is the intermediate stage b, and the incremental threshold increases rapidly. After the 35th image frame is the end stage c, and the incremental threshold increases slowly.

[0084] The quality detection method for humanoid robot image acquisition provided in this embodiment first determines whether the time difference between adjacent frames is small enough based on the time stamps of the image frames to ensure that no frame is lost in the image acquisition data. Furthermore, it calculates the similarity value between adjacent frames and automatically determines the quality inspection result of the image acquisition data based on the similarity value, the preset initial threshold, and the incremental threshold, which can reduce costs, improve efficiency, and ensure accuracy.

[0085] Figure 4 is the flowchart of the quality detection method for humanoid robot image acquisition provided in the embodiments of this application Figure 2 . As Figure 4 shown, based on the above embodiment, this embodiment uses a loop comparison method to process each image frame in a series of consecutive image frames in turn, which is convenient to end the calculation in time when the unqualified standard is reached, save resources, and improve efficiency. Specifically, the method includes:

[0086] 401. Obtain the image acquisition data of the humanoid robot; the image acquisition data includes a plurality of consecutive image frames.

[0087] 402. Determine the time differences corresponding to multiple pairs of adjacent frames according to the time stamps of the multiple image frames.

[0088] 403. Determine whether all the time differences are less than a preset duration. If so, execute step 404; otherwise, end.

[0089] Steps 401 to 403 in this embodiment are similar to steps 201 to 202 in the above embodiment, and will not be elaborated here.

[0090] 404. Calculate the similarity values corresponding to multiple pairs of adjacent frames in sequence according to the chronological order of the timestamps of the multiple image frames.

[0091] Specifically, the multiple image frames can be arranged in chronological order of timestamps to obtain an image frame sequence. For each image frame in the image frame sequence in turn, perform the similarity calculation for the corresponding adjacent frame and the comparison operation with the preset conditions. Each time the loop ends, the quality inspection result is updated. If the quality inspection result reaches the unqualified standard, the loop can be stopped, thereby timely determining that the quality inspection result of the image acquisition data is unqualified.

[0092] 405. Compare the similarity value of the current adjacent frame with the first threshold; determine whether the similarity value of the current adjacent frame is less than the first threshold. If so, execute step 406; if not, execute step 409. The first threshold is determined based on the preset initial threshold and the incremental threshold corresponding to the previous adjacent frame.

[0093] Specifically, for the first adjacent frame in the sequence, that is, image frames 1 and 2, the corresponding similarity value can be compared with the sum of the incremental threshold corresponding to image frame 1 and the preset initial threshold. For the second adjacent frame, that is, image frames 2 and 3, the corresponding similarity value can be compared with the sum of the incremental threshold corresponding to image frame 2 (i.e., the incremental threshold of the latter image frame in the previous adjacent frame) and the preset initial threshold.

[0094] 406. Determine whether the similarity value of the current adjacent frame is less than the preset initial threshold. If so, execute step 407; if not, execute step 409.

[0095] In this embodiment, the first threshold is the sum of the preset initial threshold and the incremental threshold corresponding to the previous adjacent frame, and the incremental threshold is greater than 0. Therefore, the first threshold is greater than the preset initial threshold.

[0096] 407. Determine the current adjacent frame as an abnormal frame, increment the current total number of abnormal frames by one to obtain a new total number of abnormal frames, and add the incremental threshold corresponding to the current adjacent frame to the current total amount of abnormal incremental thresholds to obtain a new total amount of abnormal incremental thresholds.

[0097] In this embodiment, the total amount of abnormal incremental thresholds is the sum of the incremental thresholds of each abnormal frame. Exemplarily, assuming that when processing the adjacent frames from the first one to the 20th one in sequence, a total of 3 abnormal frames appear, namely abnormal frames 1, 2, and 3, then the current total amount of abnormal incremental thresholds is the sum of the incremental thresholds corresponding to abnormal frames 1, 2, and 3 respectively.

[0098] 408. Determine the quality inspection result of the image acquisition data according to the new total number of abnormal frames and the new total amount of abnormal incremental thresholds.

[0099] Specifically, corresponding thresholds can be set for the total amount of new abnormal frames and the total amount of the new abnormal increment threshold respectively. In the case of loose quality control, it can be set that if any total amount is less than the corresponding threshold, it can be determined as qualified. In the case of strict quality control, it can be set that both total amounts are less than the corresponding thresholds to be determined as qualified.

[0100] In some embodiments, in order to strictly control the quality of image acquisition data, strict quality inspection standards can be set. Specifically, according to the total amount of new abnormal frames and the total amount of the new abnormal increment threshold, the quality inspection result of the image acquisition data can be determined, which may include: if the total amount of abnormal frames is less than a first preset value and the total amount of the abnormal increment threshold is less than a second preset value, the quality inspection result of the image acquisition data is determined as qualified. The method provided in this embodiment can strictly screen data from multiple dimensions by setting double strict quality inspection standards for the total amount of abnormal frames and the total amount of the abnormal increment threshold, avoid the one-sidedness of single-index evaluation, and effectively ensure that the quality of image acquisition data meets the task requirements.

[0101] 409. Based on the preset initial threshold and the increment threshold corresponding to the current adjacent frame, determine a new first threshold, determine the next pair of adjacent frames of the current adjacent frame as the new current adjacent frame, and return to execute step 405 until all adjacent frames are processed or the quality inspection result is determined as unqualified.

[0102] Specifically, after comparing the similarity value of the current adjacent frame with the corresponding first threshold, the first threshold can be updated, and the next adjacent frame can be determined as the current adjacent frame, and then enter the next loop. The end condition of the loop is that the quality inspection result is determined as unqualified, then there is no need to judge the subsequent adjacent frames, or the quality inspection result is always qualified, then it proceeds to complete the loop of the last pair of adjacent frames. The end condition can be determined according to the threshold corresponding to the total amount of abnormal frames and the threshold corresponding to the total amount of the abnormal increment threshold.

[0103] The quality inspection method for humanoid robot image acquisition provided in this embodiment can strictly evaluate the quality of image acquisition data from two dimensions of time continuity and data similarity by checking the time difference between adjacent frames, dynamically calculating similarity and combining double strict standards of the total amount of abnormal frames and the total amount of the abnormal increment threshold, screen abnormal frames in time, avoid the one-sidedness of single-index, efficiently ensure that the data meets the task requirements, and at the same time improve the quality inspection efficiency through the loop processing mechanism.

[0104] Figure 5 It is a schematic structural diagram of the quality inspection device for humanoid robot image acquisition provided by the embodiments of the present application. As Figure 5As shown, the quality inspection device 50 for image acquisition of a humanoid robot includes: an acquisition module 501, a time difference determination module 502, a similarity determination module 503, and a quality inspection module 504.

[0105] The acquisition module 501 is configured to acquire image acquisition data of the humanoid robot; the image acquisition data includes a plurality of consecutive image frames;

[0106] The time difference determination module 502 is configured to determine the time differences corresponding to multiple pairs of adjacent frames respectively according to the timestamps of the plurality of image frames;

[0107] The similarity determination module 503 is configured to determine the similarity values corresponding to multiple pairs of adjacent frames respectively if the multiple time differences are all less than a preset duration;

[0108] The quality inspection module 504 is configured to determine the quality inspection result of the image acquisition data according to the multiple similarity values and a preset condition; the preset condition is related to a preset initial threshold and the increment thresholds corresponding to multiple pairs of the adjacent frames respectively.

[0109] The quality inspection device for image acquisition of a humanoid robot provided by an embodiment of the present application first determines whether the time difference between adjacent frames is small enough based on the timestamps of the image frames to ensure that there is no frame loss in the image acquisition data. Furthermore, the similarity values of adjacent frames are calculated, and the quality inspection result of the image acquisition data is automatically determined based on the similarity values, the preset initial threshold, and the increment thresholds, which can reduce costs, improve efficiency, and ensure accuracy.

[0110] In some embodiments, the similarity determination module 503 is specifically configured to: calculate the similarity values corresponding to multiple pairs of adjacent frames in sequence according to the sequence of the timestamps of the plurality of image frames; the quality inspection module 504 is specifically configured to: compare the similarity value of the current adjacent frame with a first threshold; the first threshold is determined based on the preset initial threshold and the increment threshold corresponding to the previous adjacent frame; if the similarity value of the current adjacent frame is less than the first threshold, then compare the similarity value of the current adjacent frame with the preset initial threshold; if the similarity of the current adjacent frame is less than the preset initial threshold, then determine the current adjacent frame as an abnormal frame, increment the total number of current abnormal frames by one to obtain a new total number of abnormal frames, and add the increment threshold corresponding to the current adjacent frame to the current total amount of abnormal increment thresholds to obtain a new total amount of abnormal increment thresholds; determine the quality inspection result of the image acquisition data according to the new total number of abnormal frames and the new total amount of abnormal increment thresholds.

[0111] In some embodiments, the quality inspection module 504 is specifically configured to: if the total amount of abnormal frames is less than a first preset value, and the total amount of abnormal increment thresholds is less than a second preset value, determine the quality inspection result of the image acquisition data as qualified.

[0112] In some embodiments, the quality inspection module 504 is further configured to: if the similarity value of the current adjacent frames is greater than or equal to the first threshold, determine a new first threshold based on a preset initial threshold and the increment threshold corresponding to the current adjacent frames; determine the next pair of adjacent frames of the current adjacent frames as the new current adjacent frames; compare the similarity value of the new current adjacent frames with the new first threshold.

[0113] In some embodiments, the quality inspection module 504 is specifically configured to: for each of the multiple similarity values, determine a second threshold corresponding to the similarity value according to the preset initial threshold and the increment threshold corresponding to the similarity value; if each similarity value is greater than the corresponding second threshold, determine the quality inspection result of the image acquisition data as qualified.

[0114] In some embodiments, a plurality of pairs of adjacent frames are constructed into a time series in the chronological order of the adjacent frames; wherein, the increment thresholds corresponding to the multiple pairs of adjacent frames in the time series change in an increasing manner.

[0115] In some embodiments, the time series is sequentially divided into an initial segment, an intermediate segment, and an end segment in the time order; wherein, the first change rate of the increment thresholds corresponding to the multiple pairs of adjacent frames in the initial segment and the third change rate of the increment thresholds corresponding to the multiple pairs of adjacent frames in the end segment are both less than the second change rate of the increment thresholds corresponding to the multiple pairs of adjacent frames in the intermediate segment.

[0116] In some embodiments, the expression of the increment threshold is:

[0117]

[0118] wherein, is the increment threshold, is the preset initial threshold, is the maximum threshold, is the adjustment factor constant, T is the total number of image frames, and n is the position of the previous frame in the adjacent frames in the multiple image frames in the chronological order of the timestamps.

[0119] In some embodiments, the similarity determination module 503 is specifically configured to: determine the similarity values corresponding to the multiple pairs of adjacent frames based on the cosine similarity algorithm.

[0120] The quality detection device for humanoid robot image acquisition provided by the embodiments of the present application can be used to execute the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0121] Figure 6 It is a schematic diagram of the hardware structure of the quality detection device for humanoid robot image acquisition provided by the present application. As Figure 6 shown, the electronic device 60 provided in this embodiment includes: at least one processor 601 and a memory 602. Optionally, the device 60 further includes a communication component 603. Among them, the processor 601, the memory 602, and the communication component 603 are connected through a bus 604.

[0122] In the specific implementation process, at least one processor 601 executes the computer execution instructions stored in the memory 602, so that at least one processor 601 executes the above method.

[0123] For the specific implementation process of the processor 601, reference can be made to the above method embodiments, and its implementation principle and technical effects are similar, which will not be elaborated here in this embodiment.

[0124] In the above embodiments, it should be understood that the processor can be a central processing unit (English: Central Processing Unit, abbreviated: CPU), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, abbreviated: DSP), application-specific integrated circuits (English: Application Specific Integrated Circuit, abbreviated: ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0125] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0126] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0127] This application also provides a computer program product, including a computer program which, when executed by a processor, implements the above-mentioned method.

[0128] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-mentioned method.

[0129] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0130] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0131] The division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in an electrical, mechanical, or other form.

[0132] The unit described as a separate component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, in each embodiment of the present invention, each functional unit may be integrated in a processing unit, may exist separately physically for each unit, or two or more units may be integrated in one unit.

[0134] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0135] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: various media such as ROM, RAM, magnetic disks, or optical discs that can store program codes.

[0136] Finally, it should be noted that: after considering the specification and practicing the invention disclosed herein, those skilled in the art will easily think of other implementation schemes of the present invention. The present invention aims to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in the present invention. It is not limited to the precise structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A quality detection method for image acquisition of humanoid robots, characterized in that, Including: Obtaining image acquisition data of a humanoid robot; The image acquisition data includes multiple consecutive image frames; According to the timestamps of multiple said image frames, determining the time differences respectively corresponding to multiple pairs of adjacent frames; If multiple said time differences are all less than a preset duration, determining the similarity values respectively corresponding to multiple pairs of adjacent frames; According to multiple said similarity values and a preset condition, determining the quality inspection result of the image acquisition data; The preset condition is related to a preset initial threshold and increment thresholds respectively corresponding to multiple pairs of said adjacent frames; The determining the similarity values respectively corresponding to multiple pairs of adjacent frames includes: Calculating the similarity values respectively corresponding to multiple pairs of adjacent frames in sequence according to the chronological order of the timestamps of multiple said image frames; The determining the quality inspection result of the image acquisition data according to multiple said similarity values and a preset condition includes: Comparing the similarity value of the current adjacent frame with a first threshold; the first threshold is determined based on the preset initial threshold and the increment threshold corresponding to the previous adjacent frame; If the similarity value of the current adjacent frame is less than the first threshold, comparing the similarity value of the current adjacent frame with the preset initial threshold; If the similarity of the current adjacent frame is less than the preset initial threshold, determining the current adjacent frame as an abnormal frame, incrementing the current total number of abnormal frames by one to obtain a new total number of abnormal frames, and adding the increment threshold corresponding to the current adjacent frame to the current total amount of abnormal increment thresholds to obtain a new total amount of abnormal increment thresholds; Determining the quality inspection result of the image acquisition data according to the new total number of abnormal frames and the new total amount of abnormal increment thresholds; The expression of the increment threshold is: Among them, is the incremental threshold, is the preset initial threshold, is the maximum threshold, is the adjustment factor constant, T is the total number of image frames, and n is the position of the previous frame in adjacent frames in multiple image frames in the order of time stamps.

2. The method according to claim 1, wherein The determining the quality inspection result of the image acquisition data according to the new total number of abnormal frames and the new total amount of abnormal increment thresholds includes: If the total number of abnormal frames is less than a first preset value and the total amount of abnormal increment thresholds is less than a second preset value, determining the quality inspection result of the image acquisition data as qualified.

3. The method according to claim 1, wherein The method further includes: If the similarity value of the current adjacent frame is greater than or equal to the first threshold, determining a new first threshold based on the preset initial threshold and the increment threshold corresponding to the current adjacent frame; Determining the next pair of adjacent frames of the current adjacent frame as the new current adjacent frame; Comparing the similarity value of the new current adjacent frame with the new first threshold.

4. The method according to claim 1, characterized in that, The determining the quality inspection result of the image acquisition data according to multiple said similarity values and a preset condition includes: For each similarity value among multiple said similarity values, determining a second threshold corresponding to the similarity value according to the preset initial threshold and the increment threshold corresponding to the similarity value; If each similarity value is greater than the corresponding second threshold, determining the quality inspection result of the image acquisition data as qualified.

5. The method according to any one of claims 1-4, characterized in that, Constructing multiple pairs of adjacent frames into a time series according to the chronological order of the adjacent frames; wherein, the increment thresholds respectively corresponding to multiple pairs of adjacent frames in the time series change in an increasing manner.

6. The method according to claim 5, characterized in that, The time series is sequentially divided into an initial segment, an intermediate segment, and an end segment in chronological order; wherein, a first change rate of incremental thresholds respectively corresponding to multiple pairs of adjacent frames in the initial segment and a third change rate of incremental thresholds respectively corresponding to multiple pairs of adjacent frames in the end segment are both smaller than a second change rate of incremental thresholds respectively corresponding to multiple pairs of adjacent frames in the intermediate segment.

7. The method according to any one of claims 1 to 4, characterized in that, The determination of similarity values respectively corresponding to multiple pairs of adjacent frames includes: Based on the cosine similarity algorithm, determining similarity values respectively corresponding to multiple pairs of adjacent frames.

8. A quality detection device for image acquisition of a humanoid robot, characterized in that, Including: At least one processor and a memory; The memory stores computer execution instructions; The at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the quality detection method for humanoid robot image acquisition according to any one of claims 1 to 7.

Images