Inspection controller, inspection control method, and inspection control program

Abstract

To set an appropriate determination condition for determining the normality of an operation sound of an inspection target.SOLUTION: An acquisition unit acquires labels and feature information of a specific number of devices, from operation sound information in which categories to which a plurality of devices belong, labels indicating whether the plurality of devices are normal, and feature information indicating the features of operation sounds of the plurality of devices are associated with each other. The specific number of devices are devices corresponding to a designated category. An output unit outputs statistical information including the labels and feature information of the specific number of devices. A setting unit sets a determination condition used in inspection to determine the normality of the operation sound of an inspection target based on inspection target feature information indicating the feature of the operation sound of the inspection target on the basis of specified conditions that are specified in response to the output of the statistical information. The inspection target feature information is extracted from the operation sound of the inspection target by using an extraction model generated by using the operation sound of at least a device having a label indicating normality, among the plurality of devices.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to inspection control technology. [Background technology]

[0002] Known techniques for detecting anomalies in audio include a technique for detecting anomalies using a neural network (see, for example, Patent Document 1) and a technique for generating an operating model of a circuit breaker (see, for example, Patent Document 2). Also known are a technique for identifying the causes and countermeasures for noise generated by a washing machine (see, for example, Patent Document 3), a technique for performing highly robust anomaly detection (see, for example, Patent Document 4), and a technique for reducing the capacity of a storage device for operating sounds (see, for example, Patent Document 5). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-181204 [Patent Document 2] Japanese Patent Application Laid-Open No. 2024-9691 [Patent Document 3] Japanese Patent Publication No. 2020-199010 [Patent Document 4] Japanese Patent Publication No. 2022-102053 [Patent Document 5] Japanese Patent Application Laid-Open No. 2016-14818 Summary of the Invention [Problem to be solved by the invention]

[0004] With the techniques of Patent Documents 1 to 5, it is difficult to set appropriate judgment conditions for judging the normality of the operating sound of a product during product inspection.

[0005] In one aspect, the present invention aims to set appropriate judgment conditions for judging the normality of the operating sound of an object to be inspected. [Means for solving the problem]

[0006] According to one embodiment, the inspection control device includes an acquisition unit, an output unit, and a setting unit. The acquisition unit acquires the labels and feature information of a specific number of devices from operation sound information in which a category to which each of the multiple devices belongs, a label indicating whether each of the multiple devices is normal, and feature information indicating the features of the operation sound of each of the multiple devices are associated with each other. The specific number of devices are devices that correspond to a specified category.

[0007] The output unit outputs statistical information including the label and feature information of each of the specific number of devices. The setting unit sets, based on specified conditions specified in response to the output of the statistical information, judgment conditions used in an inspection to judge the normality of the operation sound of the inspection target based on the inspection target feature information indicating the features of the operation sound of the inspection target.

[0008] The inspection target characteristic information is extracted from the operation sound of the inspection target using an extraction model generated using the operation sound of at least one device among the plurality of devices that has a label indicating normality. [Effects of the Invention]

[0009] According to one aspect, it is possible to set appropriate judgment conditions for judging the normality of the operating sound of the test object. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 2 is a functional configuration diagram of the inspection control device according to the embodiment. [Figure 2] 10 is a flowchart of an inspection control process. [Figure 3] FIG. 1 is a configuration diagram of an inspection system. [Figure 4] FIG. 2 is a functional configuration diagram of the operation sound determination device. [Figure 5] FIG. 10 is a diagram showing master information. [Figure 6] FIG. 10 is a diagram showing master information including feature amounts. [Figure 7]FIG. 10 is a diagram showing a UI screen for specifying the type and model of a product to be inspected. [Figure 8] FIG. 10 is a diagram showing a UI screen displaying a histogram. [Figure 9] FIG. 10 is a diagram showing inspection target information. [Figure 10] FIG. 10 is a diagram showing a UI screen that displays a determination result. [Figure 11] 10 is a flowchart of a model generation process. [Figure 12] 10 is a flowchart of a normality determination process. [Figure 13] FIG. 2 is a hardware configuration diagram of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the drawings.

[0012] Fig. 1 shows an example of the functional configuration of an inspection control device according to an embodiment. The inspection control device 101 in Fig. 1 includes an acquisition unit 111, an output unit 112, and a setting unit 113. The inspection control device 101 performs inspection control processing using operation sound information that associates a category to which each of a plurality of devices belongs, a label indicating whether each of the plurality of devices is normal, and feature information indicating the features of the operation sound of each of the plurality of devices.

[0013] Fig. 2 is a flowchart showing an example of an inspection control process performed by the inspection control device 101 of Fig. 1. First, the acquisition unit 111 acquires the label and feature information of each of a specific number of devices corresponding to a specified category from the operation sound information (step 201).

[0014] Next, the output unit outputs statistical information including the label and feature information of each of the specific number of devices (step 202).The setting unit then sets, based on the specified conditions specified in response to the output of the statistical information, judgment conditions to be used in an inspection to judge the normality of the operation sound of the inspection target based on the inspection target feature information indicating the features of the operation sound of the inspection target (step 203).

[0015] The inspection target characteristic information is extracted from the operation sound of the inspection target using an extraction model generated using the operation sound of at least one device among the plurality of devices that has a label indicating normality.

[0016] According to the test control device 101 of FIG. 1, it is possible to set appropriate determination conditions for determining whether the operating sound of the test object is normal.

[0017] Fig. 3 shows an example of the configuration of an inspection system including the inspection control device 101 of Fig. 1. The inspection system of Fig. 3 includes an operation sound determination device 301, a microphone 302, and terminal devices 303-1 to 303-N (N is an integer equal to or greater than 1), and performs a sensory inspection based on the operation sounds of a product. The operation sound determination device 301 corresponds to the inspection control device 101 of Fig. 1.

[0018] The operation sound determination device 301, microphone 302, and terminal device 303-i (i=1 to N) can communicate with each other via a communication network 304. The communication network 304 is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).

[0019] The microphone 302 and the operation sound determination device 301 are installed in the shipping inspection area of ​​a factory where products are manufactured. The microphone 302 records the operation sound emitted by the product under test for a certain period of time during the operation sound test, thereby acquiring audio data of the operation sound and transmitting it to the operation sound determination device 301. Products under test include, for example, small rotating machines, pumps, blowers, breakers, fans, bearings, etc.

[0020] Operation sound determining device 301 receives audio data from microphone 302 and determines the normality of the operation sound of the product to be inspected based on the received audio data. Operation sound determining device 301 then transmits the normality determination result to terminal devices 303-1 to 303-N.

[0021] Each terminal device 303-i is installed in the ith manufacturing process in a factory where products are manufactured, receives the judgment result from the operation sound judgment device 301, and displays it on the screen. The process manager of the ith manufacturing process checks the displayed judgment result and takes necessary measures, such as stopping work in that manufacturing process or repairing the equipment used in the work.

[0022] Fig. 4 shows an example of the functional configuration of the operation sound determination device 301 in Fig. 3. The operation sound determination device 301 in Fig. 4 includes a communication unit 411, a generation unit 412, an acquisition unit 413, a setting unit 414, a determination unit 415, a display unit 416, and a storage unit 417. The acquisition unit 413, the setting unit 414, and the display unit 416 correspond to the acquisition unit 111, the setting unit 113, and the output unit 112 in Fig. 1, respectively.

[0023] The operation sound determination device 301 performs a model generation process and a normality determination process. The communication unit 411 communicates with the microphone 302 and the terminal devices 303-1 to 303-N via the communication network 304.

[0024] In the model generation process, the microphone 302 acquires audio data of the operating sounds emitted by each of the multiple products under inspection and transmits it to the operating sound determination device 301. The multiple products under inspection include products belonging to multiple categories, and the products belonging to each category include normal products and abnormal products. Each category is identified by, for example, the type and model of the product. The multiple products under inspection are an example of multiple devices.

[0025] A user assigns the type and model of the product and a label indicating normality or abnormality to the sound data of each product to be inspected via a UI (User Interface) of the operation sound determination device 301.

[0026] The generation unit 412 receives voice data for each of the multiple products to be inspected from the microphone 302 via the communication unit 411, generates master information 421 including the type, model, and label assigned by the user, and stores it in the memory unit 417.

[0027] Fig. 5 shows an example of the master information 421. Each entry in the master information 421 in Fig. 5 includes a No., a type, a model, and a label. The No. is identification information for the entry, the type indicates the type of the product to be inspected, the model indicates the model of the product to be inspected, and the label indicates whether the product to be inspected is normal or not. A normal product indicates that the product to be inspected is normal, and an abnormal product indicates that the product to be inspected is abnormal.

[0028] In this example, there are two types: "Product A" and "Product B," the model of "Product A" is "A-1" or "A-2," and the model of "Product B" is "B-1." For example, among the inspected products belonging to "Product A" and "A-1," products with numbers 1 to 7 are "normal products," and products with numbers 8 to 12 are "defective products."

[0029] Next, the generation unit 412 generates an extraction model 422 that extracts feature quantities indicating the characteristics of the operation sound from the sound data of the operation sound of the product under inspection using the received sound data, and stores the generated model in the storage unit 417. The feature quantities are an example of feature information indicating the characteristics of the operation sound.

[0030] A machine learning model, a rule base, or the like is used as the extraction model 422. The machine learning model may be a neural network, a clustering model, or a random forest.

[0031] The generation unit 412 may generate the extraction model 422 using the voice data of a normal product among the multiple products to be inspected, or may generate the extraction model 422 using the voice data of both a normal product and an abnormal product.

[0032] An example of a neural network is an autoencoder. An autoencoder reconstructs the input data by compressing the dimensions of the input data to retain important information and then restoring it to its original dimensions. In machine learning to train an autoencoder, audio data of normal product operation sounds is used as training data to generate an autoencoder that reconstructs the audio data.

[0033] However, an autoencoder trained using normal audio data cannot effectively reconstruct the audio data of abnormal product operation sounds. Therefore, when abnormal audio data is input, the error (loss) between the input data and the output data becomes large. In this case, the error output from the autoencoder can be used as a feature that indicates the characteristics of the operation sound to determine the normality of the operation sound of the product being inspected.

[0034] An example of a clustering model is a model using the k-Nearest Neighbor (k-NN) method. The k-Nearest Neighbor method is an algorithm that estimates the degree of anomaly based on the distance between data. In the k-Nearest Neighbor method, a sphere containing k pieces of data (k is an integer greater than or equal to 1) that exist near the input data is found in a multidimensional space.

[0035] If the k pieces of data are audio data of the operating sounds of a normal product, the sphere obtained from the abnormal audio data will be larger than the sphere obtained from the normal audio data. In this case, the radius of the multidimensional sphere output from the model using the k-nearest neighbor method can be used as a feature quantity that indicates the characteristics of the operating sounds, making it possible to determine the normality of the operating sounds of the product being inspected.

[0036] Next, the generation unit 412 uses the extraction model 422 to extract features from the voice data of the product to be inspected corresponding to each entry of the master information 421, and registers the extracted features in that entry.

[0037] Fig. 6 shows an example of master information 421 including feature quantities. Each entry of the master information 421 in Fig. 6 corresponds to each entry of the master information 421 in Fig. 5, and a feature quantity has been added to each entry. In this example, the feature quantities are real numbers normalized in the range of 0.00 to 1.00.

[0038] The No., type, model, label, and feature amount are associated with one another in the master information 421. The master information 421 including the feature amount is an example of operation sound information.

[0039] For example, among the inspected products belonging to "Product A" and "A-2," products with No. 13 and 14 are "normal products," and their feature quantities are values ​​in the range of 0.20 to 0.55. On the other hand, products with No. 15 and 16 are "abnormal products," and their feature quantities are values ​​in the range of 0.60 to 0.95.

[0040] Therefore, by setting a threshold value for the feature between 0.55 and 0.60, products with feature values ​​below the threshold are classified as "normal products," and products with feature values ​​above the threshold are classified as "faulty products," making it possible to distinguish between "normal products" and "faulty products."

[0041] Furthermore, among the inspected products belonging to "Product A" and "A-1," products with Nos. 1 to 7 are "normal products," and their feature quantities are values ​​in the range of 0.20 to 0.50. On the other hand, products with Nos. 8 to 12 are "abnormal products," and their feature quantities are values ​​in the range of 0.50 to 0.85. In this case, if the threshold is set between 0.55 and 0.60, the "abnormal product" with No. 8 will be classified as a "normal product."

[0042] As described above, since the features extracted from the audio data vary depending on the type and model of the product being inspected, it is difficult to determine the normality of the operating sounds of all products being inspected using a single threshold. Therefore, it is desirable to set an appropriate threshold for each combination of type and model.

[0043] Therefore, in the normality determination process, the operation sound determination device 301 sets a threshold value according to the type and model of the product to be inspected, and performs a sensory test to determine the normality of the operation sound of the product to be inspected using the set threshold value. The threshold value for the feature amount is an example of a determination condition used in the test.

[0044] In the normality determination process, the display unit 416 displays a UI screen for specifying the type and model of the product to be inspected, and the user specifies the type and model of the product to be inspected via the UI screen.

[0045] 7 shows an example of a UI screen for specifying the type and model of the product to be inspected. The user selects "Product A" from the type pull-down menu 701, and selects "A-1" from the model pull-down menu 702. The user then clicks the enter button 703 to specify "Product A" and "A-1."

[0046] The acquisition unit 413 acquires the labels and feature amounts of each entry corresponding to the type and model specified by the user from the master information 421. Then, the acquisition unit 413 generates a histogram 423 indicating the frequency distribution of multiple feature amounts corresponding to the specified type and model from the acquired labels and feature amounts, and stores the histogram in the storage unit 417.

[0047] The setting unit 414 displays a histogram 423 on the UI screen via the display unit 416. The acquired labels and feature amounts are an example of label and feature information for each of a specific number of devices, and the histogram 423 is an example of statistical information.

[0048] 8 shows an example of a UI screen displaying the histogram 423. A feature distribution area 801 displays a histogram 802 showing the frequency distribution of multiple feature amounts corresponding to a specified type and model.

[0049] Histogram 802 shows the frequency distribution of the features of products corresponding to "Product A" and "A-1" specified on the UI screen of FIG. 7 among the products to be inspected included in the master information 421 of FIG. 6. The horizontal axis represents the feature, and the vertical axis represents the number of products corresponding to the feature. White bars represent the frequency distribution of "normal products," and black bars represent the frequency distribution of "abnormal products." In this example, the specific number is 12.

[0050] The anomaly detection threshold condition pull-down menu 803 includes a list of multiple anomaly detection threshold conditions, such as "do not overlook abnormal products," "highest detection accuracy," and "keep the rate of overlooking abnormal products below 20%." The anomaly detection threshold condition is a condition used to set a threshold for a feature amount. The anomaly detection threshold condition is an example of a condition related to the possibility that the operating sound of each of a specific number of devices will be determined to be abnormal.

[0051] The user specifies the anomaly detection threshold condition by selecting one of the anomaly detection threshold conditions from the pull-down menu 803. The setting unit 414 sets a threshold based on the specified anomaly detection threshold condition and outputs the set threshold to the determination unit 415. The anomaly detection threshold condition specified by the user is an example of a specified condition specified in response to the output of statistical information.

[0052] For example, as shown in FIG. 8, when “Do not overlook abnormal products” is selected as the anomaly detection threshold condition, the setting unit 414 selects the smallest value among the five “abnormal product” feature quantities included in the histogram 802 as the threshold.

[0053] The feature values ​​of "abnormal products" included in histogram 802 are 0.50, 0.60, 0.70, 0.80, and 0.85, so the smallest value, 0.50, is selected as the threshold. By classifying products with feature values ​​less than 0.50 as "normal products" and products with feature values ​​of 0.50 or greater as "abnormal products," it is possible to detect all "abnormal products" without omission.

[0054] When "highest detection accuracy" is selected as the anomaly detection threshold condition, the setting unit 414 selects as the threshold a numerical value that maximizes the F-measure for the 12 inspected products included in the histogram 802. The F-measure is one of the evaluation indices for binary classification tasks, and is calculated as the harmonic mean of precision and recall, as shown in the following equation.

[0055] F-measure = (2 × precision × recall) / (precision + recall) (1)

[0056] Precision represents the proportion of products labeled as "normal" among all products classified as "normal." Recall represents the proportion of products labeled as "normal" that are actually classified as "normal." The F-measure ranges from 0.00 to 1.00. By setting a threshold so that both precision and recall are as high as possible, the F-measure approaches 1.00. The closer the F-measure is to 1.00, the higher the detection accuracy.

[0057] Therefore, the setting unit 414 classifies the 12 products to be inspected into "normal" and "faulty" using multiple thresholds ranging from 0.00 to 1.00, and calculates the F-value from the classification results. The threshold that shows the largest F-value is then selected as the threshold for the feature quantity of the histogram 802. For example, if the maximum F-value is 0.93 and the corresponding threshold is 0.55, 0.55 is selected as the threshold.

[0058] If "keep the rate of overlooking abnormal products below 20%" is selected as the anomaly detection threshold condition, the setting unit 414 selects a numerical value as the threshold value that will result in a rate of overlooking "abnormal products" included in the histogram 802 of below 20%.

[0059] Histogram 802 contains five "abnormal products," so up to one overlooked product is acceptable. The feature values ​​of the five "abnormal products" are 0.50, 0.60, 0.70, 0.80, and 0.85, so 0.55 is selected as the threshold. Products with feature values ​​less than 0.55 are classified as "normal products," and products with feature values ​​of 0.55 or greater are classified as "abnormal products." This means that one out of five "abnormal products" is classified as a "normal product." Therefore, the overlooked product rate is 20%.

[0060] The rate of overlooking abnormal products included in the anomaly detection threshold condition is not limited to 20% and can be set to any value within the range of 0% to 100%.

[0061] The setting unit 414 displays the threshold selected in accordance with the anomaly detection threshold conditions in the anomaly detection threshold text box 804 via the display unit 416. The user checks the displayed threshold and clicks the OK button 805. When the OK button 805 is clicked, the setting unit 414 sets the selected threshold as the threshold for the feature quantities of "Product A" and "A-1". This automatically sets an appropriate threshold that satisfies the anomaly detection threshold conditions specified by the user.

[0062] By displaying a pull-down menu 803 containing a list of multiple anomaly detection threshold conditions, even a user who is not familiar with sensory testing can easily set a threshold by simply selecting the desired anomaly detection threshold condition.

[0063] Instead of selecting an anomaly detection threshold condition, the user can refer to the histogram 802 and directly input a desired threshold into the text box 804. For example, if the user is an expert who is well versed in sensory testing, the user determines an appropriate threshold from the histogram 802, inputs it into the text box 804, and clicks the enter button 805.

[0064] After the threshold value for the feature amount is set, the microphone 302 records the operating sound emitted by the product under inspection for a certain period of time, thereby acquiring audio data of the operating sound and transmitting it to the operating sound determination device 301 .

[0065] The determination unit 415 receives voice data of the product to be inspected from the microphone 302 via the communication unit 411, and extracts features from the received voice data using the extraction model 422. The determination unit 415 then generates inspection target information 424 including the type and model specified by the user and the extracted features, and stores the information in the storage unit 417.

[0066] Fig. 9 shows an example of the inspection target information 424. Each entry of the inspection target information 424 in Fig. 9 includes No., type, model, and feature amount. No. is identification information of the entry, type indicates the type of product to be inspected, model indicates the model of the product to be inspected, and feature amount indicates the feature amount of the product to be inspected.

[0067] The type is "Product A" specified in Fig. 7, and the model is "A-1" specified in Fig. 7. In the inspection target information 424, the No., type, model, and feature amount are associated with one another.

[0068] Next, the determination unit 415 determines the normality of the operating sound of the inspection target product having the feature amount by comparing the feature amount included in the inspection target information 424 with the threshold value set by the setting unit 414. Then, the determination unit 415 generates a determination result 425 of the normality of the operating sound, stores it in the storage unit 417, and displays the determination result 425 and the feature amount of the target to be determined on the UI screen via the display unit 416.

[0069] Fig. 10 shows an example of a UI screen that displays the judgment result 425. In this example, the judgment result 425 is displayed for the feature of a product with a No. of 1 among the feature included in the inspection target information 424 in Fig. 9. The feature of this product is 0.60.

[0070] For example, if the threshold is 0.50, the determination unit 415 determines that an operation sound having a feature value less than 0.50 is normal, and determines that an operation sound having a feature value of 0.50 or more is abnormal. Therefore, if the feature value is less than 0.50, the product under inspection is classified as a "normal product," and if the feature value is 0.50 or more, the product under inspection is classified as a "faulty product."

[0071] A diagonally shaded bar representing the inspected item has been added at the 0.60 position in the histogram 802. The inspected item corresponds to the product being judged. The anomaly detection result area 1001 displays an inference result text box 1002, a feature text box 1003, a sound waveform 1004, and a sound image (mel spectrogram) 1005.

[0072] The judgment result 425 is displayed in text box 1002, and the feature value of the product to be judged is displayed in text box 1003. The feature value displayed in text box 1003 is 0.60, and since 0.60 is greater than or equal to 0.50, "abnormal product" is displayed in text box 1002, indicating that the operating sound is abnormal. Sound waveform 1004 and sound image 1005 represent the audio data of the operating sound of the product to be judged.

[0073] Furthermore, the determination unit 415 transmits the information included in the UI screen via the communication unit 411 to the terminal devices 303-1 to 303-N.

[0074] Each terminal device 303-i displays on a UI screen the information received from the operation sound determination device 301. As a result, a UI screen similar to that shown in Fig. 10 is displayed on the terminal device 303-i. The process manager of the i-th manufacturing process checks the displayed UI screen and takes necessary measures, such as stopping work in that manufacturing process or repairing the equipment used in the work.

[0075] According to the inspection system of Fig. 3, the user can simply specify the abnormality detection threshold conditions to appropriately set the threshold for determining the normality of the operating sound depending on the type and model of the product being inspected. Therefore, the user does not need to determine the threshold based on the volume of the operating sound, etc., through trial and error, by referring to previously collected product audio data. This makes it possible for even an unskilled user to easily perform a highly accurate sensory inspection.

[0076] When the product to be inspected is a small rotating machine, the sound collection environment at the shipping inspection site is often poor, and noise is often added to the voice data. However, by making a judgment using the features extracted by the extraction model 422, it is possible to judge the normality of the operating sound without being affected by most steady noises.

[0077] For non-stationary noises such as chimes, the noise can be recorded in advance, and if the same noise overlaps with the operating sound of the product being inspected, the operating sound can be recorded again, thereby reducing the impact on the judgment result 425.

[0078] The setting unit 414 can also set a judgment condition other than the threshold value according to the anomaly detection threshold condition specified by the user. For example, when multiple extraction models 422 are generated from the master information 421, information specifying the extraction model 422 to be used in the sensory test may be used as another judgment condition.

[0079] In this case, the determination unit 415 extracts features from the audio data of the product to be inspected using the extraction model 422 set by the setting unit 414 from among the multiple extraction models 422. Then, the determination unit 415 uses the extracted features to determine the normality of the operating sound of the product to be inspected.

[0080] Fig. 11 is a flowchart showing an example of model generation processing performed by the operation sound determination device 301 in Fig. 4. First, the generation unit 412 receives audio data of each of a plurality of products to be inspected from the microphone 302 via the communication unit 411 (step 1101). Then, the generation unit 412 generates master information 421 including the type, model, and label assigned by the user (step 1102).

[0081] Next, the generation unit 412 generates an extraction model 422 using the received voice data (step 1103). Then, the generation unit 412 extracts features from the voice data of the product to be inspected using the extraction model 422, and registers the extracted features in the master information 421 (step 1104).

[0082] Fig. 12 is a flowchart showing an example of normality determination processing performed by the operation sound determination device 301 in Fig. 4. First, the user specifies the type and model of the product to be inspected via the UI screen (step 1201).

[0083] Next, the acquisition unit 413 acquires the labels and features corresponding to the type and model specified by the user from the master information 421 (step 1202), and generates a histogram 423 from the acquired labels and features (step 1203).

[0084] Next, the setting unit 414 displays the histogram 423 on the UI screen via the display unit 416 (step 1204), and checks whether the user has specified either the anomaly detection threshold condition or the threshold (step 1205).

[0085] If an anomaly detection threshold condition is specified (step 1205, YES), the setting unit 414 sets a threshold for the feature amount based on the specified anomaly detection threshold condition (step 1206).If a threshold is specified (step 1205, NO), the setting unit 414 sets the threshold specified by the user as the threshold for the feature amount (step 1213).

[0086] Next, the determination unit 415 receives voice data of the product to be inspected from the microphone 302 via the communication unit 411 (step 1207), and extracts features from the received voice data using the extraction model 422 (step 1208).The determination unit 415 then generates inspection target information 424 including the type and model specified by the user and the extracted features (step 1209).

[0087] Next, the determination unit 415 determines whether the operating sound of the product to be inspected is normal by comparing the feature amount included in the inspection target information 424 with a threshold value (step 1210). Then, the determination unit 415 displays the determination result 425 of the normality of the operating sound on the UI screen via the display unit 416 (step 1211), and transmits it to the terminal devices 303-1 to 303-N via the communication unit 411 (step 1212).

[0088] The configuration of the inspection control device 101 in FIG. 1 is merely an example, and some of the components may be omitted or changed depending on the application or conditions of the inspection control device 101.

[0089] 3 is merely an example, and some of the components may be omitted or changed depending on the application or conditions of the inspection system. For example, the operation sound determination device 301 may be installed in a location other than the shipping inspection site.

[0090] 4 is merely an example, and some of the components may be omitted or changed depending on the application or conditions of the inspection system. For example, if the master information 421 and the extraction model 422 are generated by an external device, the generation unit 412 can be omitted.

[0091] 2, 11, and 12 are merely examples, and some of the processes may be omitted or changed depending on the configuration or conditions of the test control device 101 or the test system. For example, in the normality determination process of Fig. 12, if the user does not directly specify a threshold value, the processes of steps 1205 and 1213 can be omitted.

[0092] The type, model, label, and feature amount of the product to be inspected shown in Figures 5, 6, and 9 are merely examples, and this information will change depending on the product to be inspected. The UI screens shown in Figures 7, 8, and 10 are merely examples, and some of the information may be omitted or changed depending on the application or conditions of the inspection system. For example, on the UI screen of Figure 10, if the user and the process manager do not need to check the audio data of the operation sound, the sound waveform 1004 and sound image 1005 can be omitted.

[0093] Fig. 13 shows an example of the hardware configuration of an information processing device (computer) used as the inspection control device 101 in Fig. 1 and the operation sound determination device 301 in Fig. 4. The information processing device in Fig. 13 includes a CPU (Central Processing Unit) 1301, a memory 1302, an input device 1303, an output device 1304, an auxiliary storage device 1305, a media drive device 1306, and a network connection device 1307. These components are hardware and are connected to each other by a bus 1308.

[0094] The memory 1302 is, for example, a semiconductor memory such as a read-only memory (ROM) or a random access memory (RAM), and stores programs and data used in processing. The memory 1302 may operate as the storage unit 417 in FIG.

[0095] 1 by executing a program using the memory 1302. The CPU 1301 (processor) also operates as the generation unit 412, the acquisition unit 413, the setting unit 414, and the determination unit 415 in FIG. 4 by executing a program using the memory 1302.

[0096] The input device 1303 is, for example, a keyboard, a pointing device, etc., and is used for inputting instructions or information from a user or operator. The output device 1304 is, for example, a display device, a printer, etc., and is used for outputting inquiries or instructions to a user or operator and processing results. The output device 1304 may operate as the output unit 112 in FIG. 1 or the display unit 416 in FIG. 4, and the processing results may be a histogram 423 or a determination result 425.

[0097] The auxiliary storage device 1305 is, for example, a magnetic disk device, an optical disk device, a magneto-optical disk device, a tape device, or the like. The auxiliary storage device 1305 may be a hard disk drive or a solid state drive (SSD). The information processing device stores programs and data in the auxiliary storage device 1305 and can use them by loading them into the memory 1302. The auxiliary storage device 1305 may operate as the storage unit 417 in FIG. 4.

[0098] The medium drive device 1306 drives the portable recording medium 1309 and accesses the recorded contents thereof. The portable recording medium 1309 is a memory device, a flexible disk, an optical disk, a magneto-optical disk, etc. The portable recording medium 1309 may be a CD-ROM (Compact Disk Read Only Memory), a DVD (Digital Versatile Disk), a USB (Universal Serial Bus) memory, etc. A user or operator can store programs and data in the portable recording medium 1309 and load them into the memory 1302 for use.

[0099] In this way, the computer-readable recording medium that stores the program and data used in the processing is a physical (non-transitory) recording medium such as the memory 1302, the auxiliary storage device 1305, or the portable recording medium 1309.

[0100] The network connection device 1307 is a communication device that is connected to the communication network 304 and performs data conversion associated with communication. The information processing device receives programs and data from external devices via the network connection device 1307 and can use them by loading them into the memory 1302. The network connection device 1307 may operate as the output unit 112 in FIG. 1 or the communication unit 411 in FIG. 4.

[0101] 13, some components may be omitted or changed depending on the application or conditions of the information processing device. For example, if the portable recording medium 1309 is not used, the medium drive device 1306 can be omitted.

[0102] The terminal device 303-i in FIG. 3 can be an information processing device similar to that in FIG.

[0103] While the disclosed embodiments and their advantages have been described in detail above, those skilled in the art may make various modifications, additions, and omissions without departing from the scope of the invention as clearly set forth in the claims. [Explanation of symbols]

[0104] 101 Inspection control device 111, 413 Acquisition Department 112 Output section 113, 414 Setting section 301 Operation sound detection device 302 Mike 303-1~303-N Terminal equipment 304 Communication Network 411 Communications Department 412 Generation part 415 Judgment section 416 Display section 417 Storage section 421 Master Information 422 Extraction Model 423 Histogram 424 Inspection subject information 425 Judgment result 701, 702, 803 pull-down menu 703, 805 Confirm button 801, 1001 area 802 Histogram 804, 1002, 1003 Text Boxes 1004 Sound Waveform 1005 Sound Image 1301 CPU 1302 memory 1303 Input Device 1304 Output Device 1305 Auxiliary storage device 1306 Media drive unit 1307 Network connection device 1308 Bus 1309 Portable recording media

Claims

1. an acquisition unit that acquires labels and feature information for each of a specific number of devices corresponding to a specified category from operation sound information in which a category to which each of a plurality of devices belongs, a label indicating whether each of the plurality of devices is normal or not, and feature information indicating features of the operation sound of each of the plurality of devices are associated with each other; an output unit that outputs statistical information including labels and feature information of each of the specific number of devices; a setting unit that sets a judgment condition used in an inspection to judge the normality of the operation sound of the inspection target based on inspection target characteristic information indicating the characteristics of the operation sound of the inspection target, based on a specified condition that is specified in response to the output of the statistical information; Equipped with The inspection control device is characterized in that the inspection object characteristic information is extracted from the operating sound of the inspection object using an extraction model generated using the operating sound of at least one device among the plurality of devices that has a label indicating normality.

2. 2. The inspection control device according to claim 1, wherein the specified condition is a condition regarding the possibility that the operating sound of each of the specific number of devices will be determined to be abnormal.

3. the output unit further outputs a plurality of conditions related to the possibility that the operating sound of each of the specific number of devices will be determined to be abnormal; 3. The inspection control device according to claim 2, wherein the setting unit acquires, as the specified condition, a condition selected from the plurality of conditions in response to the output of the statistical information.

4. 2. The inspection control device according to claim 1, wherein the judgment condition is a threshold value for the inspection object characteristic information.

5. 2. The inspection control apparatus according to claim 1, wherein the statistical information indicates a frequency distribution of the characteristic information of each of the specific number of devices.

6. 2. The inspection control device according to claim 1, wherein the category to which each of the plurality of devices belongs includes the type and model of each of the plurality of devices, and the specified category includes the type and model of the inspection target.

7. 7. The inspection control device according to claim 1, further comprising a judgment unit that extracts the inspection object characteristic information from the operation sound of the inspection object using the extraction model, and judges the normality of the operation sound of the inspection object based on the inspection object characteristic information and the judgment condition.

8. 8. The inspection control device according to claim 7, wherein the output unit further outputs the result of determination of the normality of the operating sound of the inspection object and the inspection object characteristic information.

9. acquiring labels and feature information for a specific number of devices corresponding to a specified category from operation sound information in which a category to which each of a plurality of devices belongs, a label indicating whether each of the plurality of devices is normal or not, and feature information indicating features of operation sounds of each of the plurality of devices are associated with each other; outputting statistical information including labels and feature information for each of the specified number of devices; determining whether the operation sound of the test object is normal based on test object characteristic information indicating the characteristics of the operation sound of the test object, based on specified conditions specified in response to the output of the statistical information; The computer executes the processing, an inspection control method, characterized in that the inspection object characteristic information is extracted from the operation sound of the inspection object using an extraction model generated using the operation sound of at least one device among the plurality of devices that has a label indicating normality.

10. acquiring labels and feature information for a specific number of devices corresponding to a specified category from operation sound information in which a category to which each of a plurality of devices belongs, a label indicating whether each of the plurality of devices is normal or not, and feature information indicating features of operation sounds of each of the plurality of devices are associated with each other; outputting statistical information including labels and feature information for each of the specified number of devices; determining whether the operation sound of the test object is normal based on test object characteristic information indicating the characteristics of the operation sound of the test object, based on specified conditions specified in response to the output of the statistical information; Have the computer execute the process, An inspection control program for extracting the inspection object characteristic information from the operating sound of the inspection object using an extraction model generated using the operating sound of at least one device among the plurality of devices that has a label indicating normality.

Images