Finished product quality inspection method, garment quality inspection method, system and equipment, storage medium and program product

Abstract

According to the finished product quality inspection scheme provided by the invention, the virtual and real equipment sends the collected finished product image of the target finished product to the processing equipment, the processing equipment performs defect identification on the received finished product image by using the built-in defect identification model to obtain the first defect information, and the first defect information is sent to the virtual and real equipment. According to the scheme, the external processing equipment (such as an intelligent small box) is used for performing real-time defect reasoning on the finished product image acquired by the virtual and real equipment, so that the accuracy, the real-time performance and the stability of a defect detection result can be ensured. Furthermore, the virtual-real equipment displays the corresponding first defect virtual image on the corresponding position of the target finished product in an overlapping manner based on the received first defect information, so that mixed reality display of the defect and the target finished product is realized, a user can see the defect of the target finished product clearly at a glance, and the working efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a finished product quality inspection method, a garment quality inspection method, a system, a device, a storage medium and a program product. Background Art

[0002] Garment quality inspection is a very important link in the clothing production process, which affects the attractiveness to consumers and whether the garment can be firmly grasped in the market. At present, garment quality inspection is mostly manual quality inspection. It is difficult for manual quality inspection to fully understand the types and weak points of defects in different categories of garments, and it is impossible to ensure that defects are found continuously and stably, without missing or misjudging defects. In the relevant intelligent garment quality inspection scheme, it is necessary to manually input the type of defects found, whether to rework, and select the rework process, etc., which is complicated to operate. Summary of the invention

[0003] In view of the above problems mentioned in the background technology, the present application provides a finished product quality inspection method, a finished garment quality inspection method, a system, a device, a storage medium and a program product to solve or at least partially solve the above problems.

[0004] In the first embodiment, the present application provides a finished product quality inspection method, which is applied to virtual and real equipment. The method includes:

[0005] Collecting a finished product image of a target finished product;

[0006] Perform defect recognition based on the finished product image to obtain first defect information;

[0007] Based on the first defect information, a corresponding first defect virtual image is superimposed and displayed on the target finished product.

[0008] In a second embodiment, the present application provides another finished product quality inspection method, which is applied to a processing device that communicates externally with a virtual and real device. The method includes:

[0009] Receiving a finished product image of the target finished product sent by the virtual-reality device;

[0010] Using a built-in defect recognition model, performing defect recognition on the finished product image to obtain first defect information;

[0011] The first defect information is sent to the virtual-reality device, so that the virtual-reality device overlays and displays a corresponding first defect virtual image on the target finished product based on the first defect information.

[0012] In the third embodiment, the present application provides another garment quality inspection method, which is applied to virtual and real equipment. The method includes:

[0013] In response to the wake-up operation, a virtual function menu is displayed; wherein the virtual function menu includes a shooting control;

[0014] When an operation on the shooting control is detected, shooting is started to collect a garment image of the target garment;

[0015] Perform defect recognition based on the garment image to obtain defect information;

[0016] Based on the defect information, a corresponding defect virtual image is superimposed and displayed on the target garment.

[0017] In a fourth embodiment, the present application provides a finished product quality inspection system. The system includes:

[0018] A virtual-real device, used for collecting a finished product image of a target finished product and sending the finished product image to a processing device;

[0019] A processing device, which is in communication with the virtual device, and is used to identify defects on the finished product image to obtain defect information; and return the defect information to the virtual device;

[0020] The virtual-reality device is also used to overlay and display a corresponding defect virtual image on the target finished product based on the defect information.

[0021] In a fifth embodiment, the present application provides an electronic device. The electronic device includes a memory and a processor, wherein the memory is used to store a program; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in each method embodiment provided in the present application;

[0022] In a sixth embodiment, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the steps in each method embodiment provided by the present application can be implemented.

[0023] In a seventh embodiment, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the processor is enabled to implement the steps in each method embodiment provided by the present application.

[0024] The technical solution provided by the embodiment of the present application is that when the virtual-real device performs defect recognition based on the finished product image of the target finished product collected, it is realized by a processing device that communicates externally with the virtual-real device. Specifically, the virtual-real device will send the finished product image to the processing device, and the processing device will use the built-in defect recognition model to perform defect recognition on the received finished product image to obtain the first defect information, and send the first defect information to the virtual-real device. This solution uses an external processing device (such as a smart box) to perform real-time defect reasoning on the finished product image collected by the virtual-real device, which can help ensure the accuracy, real-time and stability of the defect detection results. Furthermore, based on the received first defect information, the virtual-real device can superimpose and display the corresponding first defect virtual image at the corresponding position of the target finished product, so as to realize the mixed reality display of the defect and the target finished product, which makes it easy for users to see the defects of the target finished product at a glance and improve work efficiency. In addition, the above-mentioned target finished product can be a garment. In this scenario, this solution innovatively realizes complete intelligent garment quality inspection through virtual-real devices in the garment industry, which provides a new way to implement garment quality inspection. In addition, users can wake up virtual and real devices by interacting with them, so that the virtual and real devices can display the virtual function menus they provide, and operate the controls on the virtual function menus to achieve functions such as starting shooting and manually marking defects, which forms a closed loop of user experience and product iteration. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1a A schematic diagram of the structure of a finished product quality inspection system provided in one embodiment of the present application;

[0027] Figure 1b A timing diagram of the finished product quality inspection principle provided in one embodiment of the present application;

[0028] Figure 2a and 2b A schematic diagram of the interaction principle between the virtual and real device and the user provided in the embodiment of the present application;

[0029] Figure 3 and Figure 4 A schematic diagram of a finished product quality inspection method provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the flow chart of the garment quality inspection method provided in the other two embodiments of the present application;

[0031] Figure 6 and Figure 7 A schematic diagram of the structure of a finished product quality inspection device provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of a garment quality inspection device provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0034] At present, in ordinary garment factories, garment quality inspection (garment quality inspection) is basically completed manually by quality inspectors. However, with manual quality inspection, quality inspectors are often highly subjective, and it is difficult to fully understand the common types and weak positions of defects in different categories of garments, and it is impossible to ensure that defects are found continuously and stably, and that defects are not missed or misjudged. In some intelligent garment factories, although some intelligent means are used to achieve garment quality inspection, quality inspectors still need to constantly click on tablet computers (PADs) to manually input the types of defects found, whether to rework, and select rework procedures, etc. These require quality inspectors to keep turning their heads to switch perspectives, which is complicated to operate. Moreover, from a technical perspective, most other quality inspection solutions are still to compare or prompt process steps. Specifically, it is to place the overall model of the quality inspection operation process into the corresponding equipment in advance, and then call the model through the space in which it is located, and then make compliance judgments or prompts on the quality inspector's operating steps. This quality inspection method is rigid and not a traditional quality inspection, and can only be regarded as a process standardization inspection or correction. In addition, other quality inspection solutions do not have continuous learning capabilities, and the model cannot learn automatically after being implanted, making it difficult to correct or iterate omissions or errors in detection in a timely manner.

[0035] In order to solve the above-mentioned problems existing in the quality inspection of ready-made clothes, the present application provides a complete set of virtual-real combined finished product (such as ready-made clothes) quality inspection automation detection solutions based on virtual-real equipment. The solution can use automated detection algorithms to automatically identify finished product defects (flaws), which can greatly improve the accuracy and stability of ready-made clothing quality inspection; in addition, the defect location can be virtually marked on the physical finished product, and the defect type and local defect map can be displayed. This mixed reality combining virtual and real can save quality inspectors from turning their heads back and forth to switch perspectives, and there is no need to click on a tablet computer (PAD) to enter the defect type, etc., and the quality inspection solution is simple.

[0036] Among them, the above-mentioned virtual-reality device can be but is not limited to mixed reality (MR) glasses. MR glasses use mixed reality technology to combine virtual information and the real world, so that users can see virtual objects in the real world, and these virtual objects and real objects can interact with each other. That is, it can be understood that MR glasses provide a new way of interaction by enhancing the visual experience of the real world and enabling users to see and interact with virtual objects.

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0038] In some processes described in the specification, claims and the above-mentioned figures of this application, multiple operations appearing in a specific order are included, and these operations may not be executed or executed in parallel in the order in which they appear in this article. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to different types. The term "or / and" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example: A or / and B, indicating that A can exist alone, A and B can exist at the same time, and B can exist alone; the character " / " in this application generally indicates that the front and back associated objects are an "or" relationship. It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such product or system. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the product or system including the elements. In addition, the following embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0039] The technical solutions provided by each embodiment of the present application are introduced and explained below.

[0040] FIG1 shows a flow chart of a finished product quality inspection method provided by an embodiment of the present application. The execution subject of the method is a virtual-real device, which can be, but is not limited to, MR glasses. As shown in FIG1 , the finished product quality inspection method includes the following steps:

[0041] 101. Collecting a finished product image of a target finished product;

[0042] 102. Perform defect recognition based on the finished product image to obtain first defect information;

[0043] 103. Based on the first defect information, a corresponding first defect virtual image is superimposed and displayed on the target finished product.

[0044] In practical applications, a virtual device can be bound to a workstation of a quality inspector. The virtual device has an application for implementing the method embodiment provided in the present application, and the application can be automatically started when the virtual device is turned on. The startup of the virtual device can be achieved by, but not limited to, operating the startup control set on the virtual device or the startup control set on the processing device connected to the virtual device. In addition, according to the settings, the application can also realize the automatic login of the corresponding user (such as a quality inspector) according to the workstation bound to the virtual device.

[0045] After the application is started, it starts to capture images through the camera and detects finished product defects (i.e., flaws) in the captured images.

[0046] During specific implementation, a common camera and a depth camera are provided on the virtual-real device. Common cameras, such as RGB cameras, are used for shooting, and the images shot are planar images (i.e., two-dimensional images). Depth cameras, such as TOF (Time of flight) cameras, are used for shooting to obtain depth information of the photographed object. In the present application, there is no specific limitation on the number of common cameras and depth-sensing cameras arranged on the virtual-real device, but at least one common camera may be arranged at the center of the virtual-real device (such as MR glasses). In addition, the application on the virtual-real device provides a virtual function menu, so that the user can select the function of the virtual-real device, select the defect type, mark the defect area, confirm the automatic defect recognition result, take defect pictures, and so on by interacting with the virtual-real device. In the present application, the interaction between the user and the virtual-real device can be, but is not limited to, achieved through gestures or voice, and preferably, through gesture interaction. For example, after the application on the virtual-reality device is started, the user wearing the virtual-reality device can interact with the virtual-reality device through gestures, such as opening the palm to wake up the virtual function menu of the virtual-reality device (i.e., the application), and clicking the shooting control in the virtual function menu. At this time, the virtual-reality device will first use the ordinary camera and the depth-sensing camera on it to scan and shoot the surrounding space environment in the user's field of view before formal use, and transmit the shooting results to the processor inside the virtual-reality device for spatial modeling to generate a three-dimensional space model (i.e., a three-dimensional space model of the real space environment), and obtain the first type of coordinate position and the second type of coordinate position of the user in the three-dimensional space model. Among them, the first type of coordinate position refers to the relative position, which refers to the position in the first three-dimensional space coordinate system. The first three-dimensional space coordinate system is a relative coordinate system that describes the position of other objects based on the position of a reference point or reference object; the second type of coordinate position refers to the position in the second three-dimensional space coordinate system. The second three-dimensional space coordinate system refers to a three-dimensional space coordinate system in which all coordinates are described based on the position of a fixed coordinate system origin.

[0047] Among them, the role of spatial modeling here is to achieve the fusion interaction of people, virtual world and real world, and to process multi-dimensional signal input and output (such as signals generated by user gestures, voice, etc.). For example, spatial modeling is one of the key links for the seamless integration of the virtual world and the real world, allowing users to interact with objects in the virtual world through virtual and real devices, and these interactions can be reflected in the real world, or information can be obtained from the real world and reflected in the virtual world. For example, the three-dimensional spatial model generated based on spatial modeling can be holographically projected for defect information to generate mixed reality scenes (such as superimposing virtual defect marks, defect types, etc. on the corresponding positions of the real target finished products). Determining the first type of coordinate position (relative coordinate position) and the second type of coordinate position of the user in the three-dimensional spatial model can be used to locate the user's orientation, facilitate gesture recognition, etc. In addition, the processor inside the virtual and real device can be a holographic processor (Holographic Processing Unit, HPU), which is a co-processing chip developed specifically for spatial modeling and visual holographic processing.

[0048] Afterwards, the application can call the ordinary camera at the center of the virtual and real device to continuously shoot two-dimensional images. Since the ordinary camera is at the center of the virtual and real device, when the user wears the virtual and real device, the ordinary camera at this center position will be located at the center of the human eye. The shooting angle is almost the same as the overlapping field of view of the human eyes, so it can ensure that the finished product image that the user sees when inspecting the finished product can be clearly captured.

[0049] Based on the above content, the “capturing the finished product image of the target finished product” in 101 above may include:

[0050] Using an ordinary camera set at the center of the virtual and real equipment, the finished product image of the target product is collected in real time.

[0051] In the present application, the target finished product can be any finished product that has been processed and needs to be inspected, such as ready-made clothing and processed bags, suitcases, shoes, etc. For finished products that need quality inspection, you can pre-select and set the target finished product types that require quality inspection, so that you can subsequently carry out targeted and autonomous finished product quality inspection to ensure the accuracy of quality inspection. For example, the virtual function menu provided by the virtual-reality device has a finished product type control. When the user clicks on the finished product type control through a gesture, a virtual finished product type option page will be projected and displayed. The virtual finished product type option page displays multiple finished product type options. The user can complete the pre-setting of the target finished product type that requires quality inspection by clicking the corresponding finished product type option from the multiple finished product type options for the target finished product.

[0052] The target finished product to be inspected will be placed on the workstation desktop bound to the virtual and real equipment, such as Figure 2aThe garment 3 shown in the figure. The target finished product is photographed by an ordinary camera set at the center position. After the finished product image of the target finished product is collected, the application will send the finished product image to an external processing device. The external processing device is an intelligent small box (AI small box), which can communicate with the virtual and real device through wired or wireless means. For example, the virtual and real device is provided with a typec interface (a hardware interface of a universal serial bus (USB)), then the virtual and real device and the external processing device can communicate through a wired manner. Correspondingly, the application on the virtual and real device can transmit the finished product image to the external processing device through the typec interface, and the image identifier (image ID) of the finished product image and the corresponding three-dimensional space coordinate system when shooting will also be recorded inside the virtual and real device (such as in the memory), and the three-dimensional space coordinate system can be determined based on the three-dimensional space model generated above.

[0053] Furthermore, the external processing device is pre-installed with a corresponding defect recognition model, which can be constructed and pre-trained using an artificial intelligence machine learning algorithm, such as a deep learning-based detection algorithm. The processing device can use this defect recognition model to identify defects in the received finished product image, and then return corresponding defect information (such as the two-dimensional coordinate position of the defect in the finished product image and the defect type, etc.) to the application in the virtual device based on the recognition result.

[0054] That is, in one example, the above 102 “performing defect identification based on the finished product image to obtain first defect information” may include:

[0055] 1021. Send the finished product image to an external processing end, and the processing end uses a built-in defect recognition model to perform defect recognition on the finished product image to obtain the first defect information;

[0056] 1022. Receive the first defect information sent by the processing end

[0057] The first defect information includes the two-dimensional coordinate position of the defect in the finished product image and the defect type. In addition, the first defect information may also include an image identifier of the finished product image.

[0058] Of course, in another example, if the virtual-real device has a strong data processing capability, the above step 102 can also be implemented by the virtual-real device itself. In this embodiment, it is preferred to implement the defect recognition function through an external processing device, which can reduce the number of device layouts of the virtual device, such as no need to deploy a processor for defect recognition, etc., thereby reducing the weight of the virtual-real device and providing users with a better wearing experience. In addition, this implementation uses an external processing device (such as a smart box) to perform real-time defect reasoning on the finished product image taken by the camera on the virtual-real device, which can facilitate the accuracy and real-time nature of the defect detection results, and achieve true real-time defect detection rather than being limited to process standardization detection based on spatial modeling.

[0059] Afterwards, the application in the virtual-reality device can obtain the three-dimensional space coordinate system corresponding to the time when the finished product image with the image identifier was taken from the previously recorded information based on the image identifier contained in the first defect information, and map the two-dimensional coordinate position of the defect to the three-dimensional space coordinate system to determine the three-dimensional coordinate position of the defect, thereby generating a holographic defect identifier and a projection of the defect type at the three-dimensional coordinate position.

[0060] It should be noted here that the three-dimensional space coordinate system corresponding to the finished image captured in this application belongs to the aforementioned second three-dimensional space coordinate system (that is, a three-dimensional space coordinate system in which all coordinates are described based on the position of a fixed coordinate system origin).

[0061] Based on the above content, the above 103 “based on the first defect information, superimposing and displaying the corresponding first defect virtual image on the target finished product” includes:

[0062] 1031. Determine a three-dimensional coordinate position corresponding to the defect based on the two-dimensional coordinates of the defect; the three-dimensional coordinate position is a position of the defect in a three-dimensional space coordinate system corresponding to the defect when the finished product image is acquired;

[0063] 1032. Based on the three-dimensional coordinate position, a first virtual defect mark is superimposed and displayed at a corresponding position of the target finished product, and a corresponding defect type is associated with the first virtual defect mark and displayed.

[0064] In the present application, the shape of the virtual defect mark (such as the first virtual defect mark here and the second virtual defect mark described below) can be a box mark, specifically, it can be but not limited to a circle, polygon (such as a rectangle, square, pentagon) and other box marks.

[0065] Figure 2aAn example of superimposing and displaying two first virtual defect marks 131 at the corresponding positions of the target finished product is shown, wherein one of the first virtual defect marks (virtual defect mark 1311) is associated with a defect type of broken wire, and the other first virtual defect mark (virtual defect mark 1312) is associated with a defect type of pleating. The user operates the virtual defect mark 1311 corresponding to "broken wire" or "broken wire" through gestures, and then the corresponding holographic defect local image 14 will be displayed, so that the user can make a manual secondary judgment based on the defect local image 14 to determine whether the automatically identified defect is correct. The manual secondary judgment can be achieved by the user operating the corresponding holographic operation control through gestures or voice.

[0066] For example, see Figure 2a , corresponding operation controls, such as the first operation control 141 and the second operation control 142, may be displayed in association with one side (such as the bottom) of the defect local diagram 14. If the user operates the second operation control 142, it is confirmed that the defect automatically identified by manual judgment is correct. If the user operates the first operation control 141, it is confirmed that the defect automatically identified by manual judgment is wrong. In this case, a modification prompt projection may be further displayed to prompt the user whether to make an incorrect modification to the defect. If the user operates "No", it may indicate that there is no defect at this position, so that the virtual defect mark 1311 and the corresponding defect type displayed at this position may disappear; and if the user operates "Yes", a holographic input box or defect type option page may be displayed for the user to input or select the corresponding defect type, and the defect type projection "broken line" displayed at this position may be changed to the defect type input or selected by the user, so as to correct the defect type.

[0067] Combined with the above Figure 2a The method provided in this embodiment may further include the following steps:

[0068] 104. In response to a user operating a displayed defect type, projecting and displaying a defect local image marked by the first virtual defect mark associated with the defect type, so that the user can perform a corresponding operation on the defect based on the defect local image;

[0069] Among them, performing corresponding operations on the defect includes: determining whether the defect is correct and modifying the defect type.

[0070] It should be noted here that: when the user modifies the defect type for the defect, the modified defect type and the two-dimensional coordinate position and image identification of the corresponding defect can also be returned to the processing end so that the processing end can optimize the defect recognition model therein.

[0071] Furthermore, the method provided in this embodiment may also include the following steps:

[0072] 105. When a wake-up operation triggered by a user is detected, a virtual function menu is displayed;

[0073] 106. When the user operates the annotation control in the virtual function menu, a corresponding second defect virtual image is superimposed and displayed on the target finished product according to the second defect information selected by the user;

[0074] 107. When the user operates the clear control in the virtual function menu, all defective virtual images disappear.

[0075] In specific implementation, it is possible but not limited to projecting and displaying a virtual function menu when a user performs a wake-up gesture (such as opening a palm), such as Figure 2b The virtual function menu 11 shown includes several function controls, such as a clear control 111, a next control 112 and a mark control 113 (a control for manually marking). In addition, the virtual function menu may also include other function controls, such as a shooting control. Figure 2b The function controls included in the virtual function menu 11 given in the figure are only schematic, and are not limited to the virtual function menu to include only the several function controls shown here. When the user operates the clear control 111, all displayed defect virtual images such as virtual defect marks, projected defect types, etc. will be cleared. When the user operates the next control 112, defect detection will begin for the next target finished product. When the user operates the marking control 113, the user can manually mark the defects on the target finished product. Specifically, the corresponding second defect information will be superimposed and displayed on the target finished product according to the second defect information selected by the user.

[0076] For example, see Figure 2b When the user operates the annotation control 113, a holographic second virtual defect mark 132 can be displayed, and the user can drag the second virtual defect mark 132 through gestures to drag the second defect mark 132 to the top of the defect manually determined by the user to mark the defect. After dragging is completed, a virtual option page 12 can be projected and displayed, and multiple defect types (such as pleats, broken lines, etc.) are displayed in the virtual option page 12; wherein, the virtual option page 12 can be automatically projected and displayed or projected and displayed by detecting that the user has performed a corresponding gesture, etc., which is not limited here. Furthermore, the user can select a variety of defect types in the virtual option page 12 through interactive methods such as gestures or voice. After the selection is completed, the virtual-reality device will associate the defect type selected by the user with the second virtual defect mark 132 for display.

[0077] It should be supplemented with respect to the above example that the second virtual defect mark can also be displayed above the defect manually determined by the user on the target finished product in other ways. For example, after the user operates the annotation control 113, and it is detected within a preset time period that the user clicks a position on the target finished product through a gesture, a holographic second virtual defect mark can be directly displayed at the position clicked by the user, so that the user no longer needs to move the second virtual defect mark, and manual defect marking is simple. In addition, in the present application, in addition to moving virtual defect marks (such as the second virtual defect mark, etc.) through gestures, the user can also perform other operations such as zooming in and out.

[0078] Based on the above example content, in one embodiment, the above 106 of “superimposing and displaying the corresponding second defect virtual image on the target finished product according to the second defect information selected by the user” may specifically include:

[0079] 1061. Display a second virtual defect mark;

[0080] 1062. According to the user's dragging operation on the second virtual defect mark, superimpose and display the second virtual defect mark at a corresponding position on the target finished product;

[0081] 1063. Display a virtual option page, wherein the virtual option page displays multiple defect type options;

[0082] 1064. Based on the user's selection operation on the multiple defect type options, the selected defect type is associated with the second virtual defect mark and displayed.

[0083] Alternatively, in another example, the above step of “superimposing and displaying a corresponding second defect virtual image on the target finished product according to the second defect information selected by the user” in 106 may specifically include:

[0084] 1061', determining the operation position of the user on the target finished product (for example, the operation position may be a click position made by the user through a gesture);

[0085] 1062', displaying a second virtual defect mark at the operating position;

[0086] 1063', displaying a virtual option page, wherein the virtual option page displays a plurality of defect type options;

[0087] 1064'. According to the user's selection operation on the plurality of defect type options, the selected defect type is associated with the second virtual defect mark and displayed.

[0088] From the human-computer interaction between people and virtual and real devices described above, it can be seen that users can control the function menu of virtual and real devices through gestures to select virtual and real functions, select defect types, divide defect areas, confirm automatic defect recognition results, take finished product images, etc., forming a closed loop of user experience and product iteration.

[0089] Furthermore, the method of this embodiment may also include the following steps:

[0090] 108. Send the second defect information selected by the user to the processing end so that the processing end optimizes the defect recognition model using the second defect information. The second defect information includes: the image identifier of the corresponding finished product image, the two-dimensional coordinate position of the defect in the finished product image, and the defect type.

[0091] As can be seen from the above, this embodiment provides a complete sample return path. In addition to using the processing end (such as the smart box, which is a defect marking platform) to automatically detect finished product defects, it also provides a manual defect selection function, that is, you can manually select the defect position that is missed by the automatic detection. After manual selection, the virtual and real equipment will automatically return the defect to the processing end so that its built-in defect recognition model can be iterated and upgraded in time to achieve model optimization. Obviously, this provides a complete closed loop for the iteration and improvement of the defect recognition model.

[0092] The present application also provides a finished product quality inspection method in another embodiment, which is applied to a processing end that communicates with a virtual-real device, and the processing end may be an intelligent small box (AI small box) external to the virtual-real device. Figure 4 As shown, the finished product quality inspection method includes the following steps:

[0093] 201. Receive a finished product image of the target finished product sent by the virtual-reality device;

[0094] 202. Using a built-in defect recognition model, perform defect recognition on the finished product image to obtain first defect information;

[0095] 203. Send the first defect information to the virtual-reality device, so that the virtual-reality device overlays and displays a corresponding first defect virtual image on the target finished product based on the first defect information.

[0096] In the present application, the defect virtual image is also a holographic image.

[0097] For the specific implementation description of 201-203 and the processing equipment and virtual-real equipment, please refer to the relevant content in other embodiments. In addition, in addition to the above steps, this embodiment may also include other steps. For other steps and specific implementations that may be included, please refer to the relevant content in other embodiments, and no further details will be given here.

[0098] In summary, the finished product quality inspection solution provided in this embodiment can be briefly described as follows:

[0099] See Figure 1b As shown, first, the virtual-real device scans and shoots the space environment in which it is located, so as to generate three-dimensional space coordinates based on the shooting results for spatial modeling. Furthermore, the finished product image of the target finished product taken will be transmitted to an external processing device (such as a smart box), and the processing device will call the built-in defect recognition model to perform defect recognition reasoning on the finished product image, and return the identified defect information (such as the two-dimensional coordinate position of the defect in the finished product image, the defect type, etc.) to the virtual-real kidney, so that the virtual-real device can perform holographic projection based on the received defect information, and realize the display of holographic virtual defect marks and defect type projection at the corresponding position on the target finished product image.

[0100] The target finished product described in the above embodiments may be a garment. For this scenario, the present application also provides a garment quality inspection method, which is applied to virtual-real equipment. For the specific description of the virtual-real equipment, please refer to the relevant content in other embodiments. Figure 5 As shown, the garment quality inspection method comprises the following steps:

[0101] 301. In response to the wake-up operation, display a virtual function menu; wherein the virtual function menu includes a shooting control;

[0102] 302. When an operation on the shooting control is detected, starting shooting to collect images of the target garment;

[0103] 303. Perform defect recognition based on the garment image to obtain defect information;

[0104] 304. Based on the defect information, a corresponding defect virtual image is superimposed and displayed on the target garment.

[0105] For the specific implementation description of 301 to 304, please refer to the relevant content in other embodiments. In addition, in addition to the above steps, this embodiment may also include other steps. For other steps and their specific implementation, please refer to the relevant content in other embodiments, and no further details will be given here.

[0106] This application also provides a finished product quality inspection system. Figure 1a As shown, the finished product quality inspection system includes:

[0107] The virtual-real device 1 is used to collect the finished product image of the target finished product and send the finished product image to the processing device;

[0108] Processing device 2, which is in external communication with the virtual device, is used to identify defects of the finished product image to obtain defect information; and return the defect information to the virtual device;

[0109] The virtual-reality device 1 is also used to overlay and display a corresponding defect virtual image on the target finished product based on the defect information.

[0110] The target finished product may be, but is not limited to, ready-made clothing.

[0111] For detailed descriptions of the specific implementation of the processing devices, virtual and real devices and functions of each device included in the system provided in this embodiment, please refer to the relevant content in other embodiments, and no further details will be given here.

[0112] Figure 6 The structure diagram of the finished product quality inspection device provided in one embodiment of the present application is shown, and the device is deployed in a virtual and real device. Figure 6 The finished product quality inspection device includes: a collection module 41, an identification module 42 and a display module 43.

[0113] The acquisition module 41 is used to acquire the finished product image of the target finished product;

[0114] An identification module 42, configured to perform defect identification based on the finished product image to obtain first defect information;

[0115] The display module 43 is used to overlay and display a corresponding first defect virtual image on the target finished product based on the first defect information.

[0116] Optionally, when the recognition module 42 is used to perform defect recognition based on the finished product image to obtain the first defect information, it can be specifically used to: send the finished product image to an external processing device, and the processing device uses a built-in defect recognition model to perform defect recognition on the finished product image to obtain the first defect information; receive the first defect information returned by the processing end; wherein the first defect information includes the two-dimensional coordinate position of the defect and the defect type.

[0117] Optionally, when the display module 43 is used to superimpose and display a corresponding defect virtual image on the target finished product based on the first defect information, it can be specifically used to: determine the three-dimensional coordinate position corresponding to the defect based on the two-dimensional coordinate position of the defect; the three-dimensional coordinate position is the position of the defect in the three-dimensional space coordinate system corresponding to the defect when the finished product image is acquired; based on the three-dimensional coordinate position, superimpose and display a first virtual defect mark on the corresponding position of the target finished product, and associate the corresponding defect type with the first virtual defect mark and display it.

[0118] Optionally, the device also includes: a response module, which is used to respond to a user's operation on a displayed defect type, project and display a defect local image identified by the first virtual defect mark associated with the defect type, so that the user can perform corresponding operations on the defect based on the defect local image; wherein, performing corresponding operations on the defect includes: determining whether the defect is correct and modifying the defect type.

[0119] Optionally, the above-mentioned display module 43 is also used to display a virtual function menu when a wake-up operation triggered by the user is detected; when the user operates the marking control in the virtual function menu, the corresponding second defect virtual image is superimposed and displayed on the target finished product according to the second defect information selected by the user; and the device also includes a clearing module, which is used to make all defective virtual images disappear when the user operates the clearing control in the virtual function menu.

[0120] Optionally, the above-mentioned display module 43, when used to superimpose and display the corresponding first defect virtual image on the target finished product according to the second defect information selected by the user, may be used to: display a second virtual defect mark; superimpose and display the second virtual defect mark at a corresponding position on the target finished product according to the user's dragging operation on the second virtual defect mark; display a virtual option page, in which a plurality of defect type options are displayed; and display the selected defect type in association with the second virtual defect mark according to the user's selection operation on the plurality of defect type options.

[0121] Optionally, the device further includes a sending module, configured to send the second defect information selected by the user to the processing end, so that the processing end optimizes the defect recognition model using the second defect information.

[0122] It should be noted here that the finished product quality inspection device provided in the above embodiment can implement the technical solution described in the above corresponding method embodiment. The specific implementation principles of the above modules or units can refer to the relevant contents in the above corresponding method embodiment, and will not be described in detail here.

[0123] Figure 7 FIG. 2 shows a schematic diagram of the structure of a finished product quality inspection device provided by another embodiment of the present application, wherein the device is deployed in a processing device externally connected to a virtual-real device. Figure 7 The finished product quality inspection device includes: a receiving module 51, an identification module 52 and a sending module 53.

[0124] A receiving module 51 is used to receive the finished product image of the target finished product sent by the virtual-reality device;

[0125] The recognition module 52 is used to use a built-in defect recognition model to perform defect recognition on the finished product image to obtain first defect information;

[0126] The sending module 53 is used to send the first defect information to the virtual-reality device, so that the virtual-reality device overlays and displays the corresponding first defect virtual image on the target finished product based on the first defect information.

[0127] It should be noted here that the finished product quality inspection device provided in the above embodiment can implement the technical solution described in the above corresponding method embodiment. The specific implementation principles of the above modules or units can refer to the relevant contents in the above corresponding method embodiment, and will not be described in detail here.

[0128] Figure 8 FIG. 2 shows a schematic diagram of the structure of a garment quality inspection device provided by another embodiment of the present application, and the device is deployed in a virtual and real device. Figure 8 , the garment quality inspection device comprises:

[0129] A display module 61 is used to display a virtual function menu in response to a wake-up operation; wherein the virtual function menu includes a shooting control;

[0130] A shooting module 62, configured to start shooting to acquire a garment image of a target garment when an operation on the shooting control is detected;

[0131] An identification module 63, configured to perform defect identification based on the garment image to obtain defect information;

[0132] The display module 61 is further configured to overlay and display a corresponding defect virtual image on the target garment based on the defect information.

[0133] It should be noted here that: the garment quality inspection device provided in the above embodiment can implement the technical solution described in the above corresponding method embodiment. The specific implementation principles of the above modules or units can be found in the relevant contents of the above corresponding method embodiment, and will not be described in detail here.

[0134] Fig. 9 FIG. 1 is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present application. Fig. 9 As shown, the electronic device includes: a memory 71 and a processor 72. The memory 71 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. Specifically,

[0135] The memory 71 is used to store programs;

[0136] The processor 72 is coupled to the memory 71 and is used to execute the program stored in the memory for the steps or functions in the methods provided in the embodiments of the present application.

[0137] Further, if Fig. 9 The electronic device also includes: a communication component 73, a power component 74, an audio component 75 and other components. Fig. 9 Only some components are shown schematically, which does not mean that the electronic device only includes Fig. 9 The electronic device may be a virtual device or a processing device that communicates externally with a virtual device.

[0138] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the method steps provided in the above embodiments can be implemented.

[0139] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the processor is enabled to implement the method steps or functions provided in the above embodiments.

[0140] Through the description of the above implementation modes, those skilled in the art can clearly understand that each implementation mode can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A finished product quality inspection method, characterized in that: Applied to virtual and real equipment; the method comprises: Collecting a finished product image of a target finished product; Perform defect recognition based on the finished product image to obtain first defect information; Based on the first defect information, a corresponding first defect virtual image is superimposed and displayed on the target finished product.

2. The method according to claim 1, characterized in that: Defect recognition is performed based on the finished product image to obtain first defect information, including: The finished product image is sent to an external processing device, and the processing device uses a built-in defect recognition model to perform defect recognition on the finished product image to obtain the first defect information; receiving the first defect information returned by the processing end; The first defect information includes the two-dimensional coordinate position of the defect and the defect type.

3. The method according to claim 2, characterized in that Based on the first defect information, a corresponding defect virtual image is superimposed and displayed on the target finished product, including: Based on the two-dimensional coordinate position of the defect, determine the three-dimensional coordinate position corresponding to the defect; the three-dimensional coordinate position is the position of the defect in the three-dimensional space coordinate system corresponding to the defect when the finished product image is collected; Based on the three-dimensional coordinate position, a first virtual defect mark is superimposed and displayed on a corresponding position of the target finished product, and a corresponding defect type is displayed in association with the first virtual defect mark.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: In response to a user operating a displayed defect type, projecting and displaying a defect local image marked by the first virtual defect mark associated with the defect type, so that the user can perform a corresponding operation on the defect based on the defect local image; Among them, performing corresponding operations on the defect includes: determining whether the defect is correct and modifying the defect type.

5. The method according to any one of claims 2 to 3, characterized in that Also includes: When a wake-up operation triggered by the user is detected, a virtual function menu is displayed; When the user operates the annotation control in the virtual function menu, a corresponding second defect virtual image is superimposed and displayed on the target finished product according to the second defect information selected by the user; When the user operates the clear control in the virtual function menu, all defective virtual images disappear.

6. The method according to claim 5, characterized in that According to the second defect information selected by the user, a corresponding first defect virtual image is superimposed and displayed on the target finished product; displaying a second virtual defect mark; According to the user's dragging operation on the second virtual defect mark, superimposing and displaying the second virtual defect mark at a corresponding position on the target finished product; Displaying a virtual option page, wherein the virtual option page displays a plurality of defect type options; According to the user's selection operation on the plurality of defect type options, the selected defect type is associated with the second virtual defect mark and displayed.

7. The method according to claim 5, characterized in that Also includes: The second defect information selected by the user is sent to the processing end so that the processing end optimizes the defect recognition model by using the second defect information.

8. A finished product quality inspection method, characterized in that: Applicable to a processing terminal for external communication with a virtual and real device; the method comprises: Receiving a finished product image of a target finished product sent by the virtual-reality device; Using a built-in defect recognition model, performing defect recognition on the finished product image to obtain first defect information; The first defect information is sent to the virtual-reality device, so that the virtual-reality device overlays and displays a corresponding first defect virtual image on the target finished product based on the first defect information.

9. A method for quality inspection of ready-made garments, characterized in that: Applied to virtual and real equipment; the method comprises: In response to the wake-up operation, a virtual function menu is displayed; wherein the virtual function menu includes a shooting control; When an operation on the shooting control is detected, shooting is started to collect a garment image of the target garment; Perform defect recognition based on the garment image to obtain defect information; Based on the defect information, a corresponding defect virtual image is superimposed and displayed on the target garment.

10. A finished product quality inspection system, characterized in that: include: A virtual-real device, used for collecting a finished product image of a target finished product and sending the finished product image to a processing device; A processing device, in external communication with the virtual-real device, for identifying defects in the finished product image to obtain defect information; and returning the defect information to the virtual-real device; The virtual-reality device is also used to overlay and display a corresponding defect virtual image on the target finished product based on the defect information.

11. An electronic device, characterized in that: include: A memory and a processor; wherein, The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the finished product quality inspection method described in any one of claims 1 to 8, or to implement the steps in the garment quality inspection method described in claim 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program; when the computer program is executed by a computer, it can implement the steps in the finished product quality inspection method described in any one of claims 1 to 8, or implement the steps in the garment quality inspection method described in claim 9.

13. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, enables the processor to implement the steps in the finished product quality inspection method described in any one of claims 1 to 8, or the steps in the garment quality inspection method described in claim 9.

Images