Detection method, detection device, electronic device, storage medium, and program

Abstract

A detection method, a detection device, an electronic device, a storage medium, and a program are provided. 【Solution means】When it is determined that there is a target operation on a target object in the video data of the target winding package carriage, the method includes determining a target area targeted by the target operation, where the target winding package carriage includes two areas each having N placement bodies for placing winding packages, the target area is one of the two areas, and N is a positive integer. The method also includes obtaining a plurality of target images that can cover the target area, determining a target winding package targeted by the target operation based on the plurality of target images, and generating presentation information for the target winding package.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and particularly to a detection method, a detection device, an electronic device, a storage medium, and a program.

Background Art

[0002] In the manufacturing industry of winding packages, before packaging, the produced winding packages are usually transported to a designated area (such as a warehouse) by a winding package trolley for storage. After being stored for a certain period, the winding package trolley carrying the winding packages is transported out of this designated area to perform subsequent processes such as packaging. During storage, it is necessary to ensure that the winding packages on the winding package trolley are not removed so as not to affect the subsequent packaging process.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, when a defect of a winding package is discovered when the winding package trolley carrying the winding package leaves this designated area, an inspection by a person is required to identify the specific winding package, and obviously this method is inefficient.

Means for Solving the Problems

[0004] The present disclosure provides a detection method, a detection device, an electronic device, a storage medium, and a program.

[0005] According to a first aspect of the present disclosure, a detection method applied to a cloud terminal is provided, and the method includes: When it is determined that there is a target operation on a target object in the video data of a target winding package trolley, determining a target area targeted by the target operation, where the target winding package trolley includes two areas each having N placement bodies for placing winding packages, the target area is one of the two areas, and N is a positive integer, and Obtaining a plurality of target images that can cover the target area; Based on the plurality of target images, determining a target winding package to be the object of the target operation, and generating presentation information for the target winding package.

[0006] According to a second aspect of the present disclosure, there is provided a detection device applied to a cloud terminal, the device comprising: An information determination unit for determining a target area to be the object of the target operation and obtaining a plurality of target images that can cover the target area when it is determined that there is a target operation on a target object in video data of a target winding package trolley, wherein the target winding package trolley includes two areas each having N placement bodies for placing winding packages, the target area is one of the two areas, and N is a positive integer; A detection unit for determining a target winding package to be the object of the target operation based on the plurality of target images; And a presentation unit for generating presentation information for the target winding package.

[0007] According to a third aspect of the present disclosure, there is provided an electronic device, the device comprising: At least one processor; A memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, cause the at least one processor to execute any one of the methods in the embodiments of the present disclosure.

[0008] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute any one of the methods in the embodiments of the present disclosure.

[0009] According to a fifth aspect of the present disclosure, a program is provided, which, when executed by a processor, implements any one of the methods in the embodiments of the present disclosure.

[0010] According to the solution of the present disclosure, by using a plurality of acquired target images, a target winding package targeted by the target operation can be identified, and further, presentation information for the target winding package can be generated. In this way, compared with the conventional inspection method by humans, the solution of the present disclosure does not need to rely on manual work. When there is a target operation on a target object, the specific information of the winding package for the target operation can be quickly identified, the inspection efficiency can be improved, the automation and intelligence of the whole process can be realized, a large amount of labor costs and time costs can be reduced, the influence on the subsequent packaging process can be avoided, and the normal operation of the work in the workplace can be ensured.

[0011] It should be understood that the content described herein is not intended to describe the key points or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. For other features of the present disclosure, understanding is promoted through the following description.

[0012] In the accompanying drawings, unless otherwise specified, the same reference numerals indicate the same or similar components or elements throughout the plurality of accompanying drawings. These accompanying drawings are not necessarily drawn to scale. It should be understood that these drawings show only some embodiments provided by the present disclosure and should not be regarded as limiting the scope of the present disclosure.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 3A

Figure 3B

Figure 3C

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Mode for Carrying Out the Invention

[0014] Hereinafter, the present disclosure will be described in more detail with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar elements. Also, in the accompanying drawings, although various aspects of the embodiments are shown, these accompanying drawings are not necessarily drawn to scale unless otherwise stated.

[0015] Furthermore, for a better understanding of the present disclosure, many specific details are described in the following specific examples. Those skilled in the art should understand that the present disclosure can be implemented similarly even without some details. In some examples, well-known methods, means, components, and circuits, etc. are not described in detail so that the gist of the present disclosure can be clear.

[0016] The solution of the present disclosure proposes a detection method for improving the inspection efficiency of winding packages.

[0017] Specifically, FIG. 1 is a schematic flowchart of a detection method according to an embodiment of the present disclosure. This method can be selectively applied to electronic devices such as personal computers, servers, and server clusters.

[0018] Furthermore, the method includes at least a part of at least the following content. As shown in FIG. 1, it includes the following.

[0019] In step S101, when it is determined that there is a target operation on a target object in the video data of the target winding package cart, determine the target area targeted by the target operation.

[0020] Here, the target winding package cart includes two areas, and each area has N (where N is a positive integer) placement bodies for placing winding packages. For example, in one example, as shown in FIG. 2A, placement areas are respectively provided on both sides of the target winding package cart, each placement area includes a placement body for placing a winding package, and furthermore, 9 winding packages can be placed in each placement area.

[0021] Furthermore, the target area targeted by the target operation is one of the two areas (for example, two placement areas).

[0022] Here, the video data of the target winding package carriage is the video data while the target winding package carriage is moving in the storage area or during storage. For example, in one example, when the start position where the target winding package carriage enters the storage area is detected, a plurality of image collection devices located in the storage area are activated, and video collection is performed on the target winding package carriage traveling in this storage area to obtain video data. The video data can be obtained by using a method. The image collection device in this example may specifically be equipped with a camera. For example, it can be understood that video collection is periodically performed on the placement areas on both sides of the target winding package carriage located in the storage area to check whether there is a target operation, for example, to check whether there is a target operation of carrying out the winding package.

[0023] Alternatively, in another example, when it is detected that a target object appears around the target winding package carriage, the camera is activated, and video collection is performed on the placement areas on both sides of the target winding package carriage to detect whether there is a target operation.

[0024] In step S102, a plurality of target images capable of covering the target area are acquired.

[0025] For example, in one example, a plurality of video frames covering the target area are selected from the video data of the target winding package carriage to obtain a plurality of target images.

[0026] Here, it should be noted that the selected multiple target images can maximally reflect the entire process of the target object performing the target operation. In this way, the winding package to be the target of the subsequent target operation can be accurately found. For example, the selected multiple target images (i.e., multiple video frames) include images before and after the target operation. In other words, the multiple target images can represent the changes in the winding package before and after the target operation. For example, the selected multiple target images at least include target image 1 (image before the target operation) as shown in FIG. 2B and target image 2 (image after the target operation) as shown in FIG. 2C.

[0027] In step S103, based on the multiple target images, determine the target winding package to be the target of the target operation, and generate presentation information for the target winding package.

[0028] Here, in one example, the presentation information for the target winding package includes, but is not limited to, the identification information of the target winding package trolley, the position information and identification information of the target winding package, etc. In this way, it can help people process the abnormal target winding package trolley in a timely manner and prevent it from affecting the subsequent packaging process of the winding package.

[0029] In this way, the solution of the present disclosure can use the acquired multiple target images to identify the target winding package to be the target of the target operation, and further generate presentation information for the target winding package. In this way, compared with the conventional inspection method by people, the solution of the present disclosure does not need to rely on manual work. When there is a target operation on the target object, it can quickly identify the specific information of the winding package for the target operation, improve the inspection efficiency, realize the automation and intelligence of the whole process, reduce a large amount of labor costs and time costs, avoid the influence on the subsequent packaging process, and ensure the normal operation of the work in the workplace.

[0030] In a specific example, a sensing component is provided at the starting position (e.g., the entrance) of the storage area, and an image collection device 1 and an image collection device 2 are provided in the storage area. This sensing component is used to detect whether the target winding package trolley has reached the starting position. The image collection device 1 and the image collection device 2 respectively perform video collection on two placement areas of the target winding package trolley from different angles. For example, in one example, first, when the sensing component detects that the target winding package trolley has reached the starting position of the storage area, it sends a detection signal to the cloud (or server). Then, in response to the detection signal, the cloud generates a collection signal and sends it to the image collection device 1 and the image collection device 2, so that the image collection device 1 and the image collection device 2 perform video collection on two placement areas of this target winding package trolley from different angles. Finally, after the cloud receives the video data collected by the image collection device 1 and the image collection device 2, if it detects that there is a target operation on the target object (e.g., the operator active in the storage area) in the video data, it determines the placement area (i.e., the target area) targeted by the target operation, and further obtains a plurality of target images of the target area from the video data. At this time, the cloud can determine the target winding package targeted by the target operation based on the obtained plurality of target images and generate presentation information.

[0031] Thereby, the solution of the present disclosure can detect in real time whether there is a target operation. When it is determined that there is a target operation, it quickly determines the target winding package targeted by the target operation and immediately presents it to the employee so as not to affect the subsequent packaging process of the winding package. In this way, the automation and intelligence of the whole process can be realized, and the normal operation of the work in the workplace can be ensured.

[0032] Figure 3A is the second schematic flowchart of the detection method according to an embodiment of the present disclosure. This method can be selectively applied to electronic devices such as personal computers, servers, and server clusters. The relevant content of the methods shown in FIGS. 1 and 2 above can also be applied in this embodiment and will not be repeated herein.

[0033] Furthermore, the method at least includes at least a part of the following content. As shown in FIG. 3A, it includes the following.

[0034] In step S301, when it is determined that there is a target operation on the target object in the video data of the target winding package carriage, determine the target area targeted by the target operation.

[0035] Here, the target winding package carriage includes two areas, each area has N placement bodies for placing winding packages, the target area is one of the two areas, and N is a positive integer.

[0036] In step S302, obtain a plurality of target images that can cover the target area.

[0037] In step S303, input each target image in the plurality of target images into a target detection model to obtain an initial mask image of each target image.

[0038] Here, based on a preset winding package prompt, the target detection model can identify the area where each winding package is located in the input image, mask the area where each winding package is located in the image using a mask version, and obtain a masked image.

[0039] In step S304, based on the identification information of the target winding package carriage, obtain the identification information of the winding packages to be placed by each placement body in the target winding package carriage.

[0040] Note that in this example, the winding package to be placed by each placement body in the target area of the target winding package carriage refers to the winding package arranged on each placement body according to a preset arrangement rule (or order), that is, the winding package that should be theoretically placed by each placement body. Based on this, after obtaining the identification information of the target winding package carriage, the identification information of the winding package theoretically placed by each placement body in this target winding package carriage can be obtained. In this way, subsequent rapid detection or rapid targeting for specific problems can be assisted.

[0041] Here, the execution order of step S303 and step S304 can be interchanged or executed synchronously, and the present disclosure is not limited thereto.

[0042] In step S305, the identification information of the winding package to be placed by each placement body in the target winding package carriage is mapped to each mask plate at different positions in the initial mask image of each target image, so as to obtain a target mask image with the identification information of the winding package corresponding to each target image.

[0043] Here, the target mask image with the identification information of the winding package corresponding to the target image can represent the identification information of each winding package actually included in the target image.

[0044] Note that the above "mapping" can refer to adding the identification information of the winding package to the placement body where the winding package should theoretically be located based on a preset arrangement rule. Furthermore, since each winding package in the mask image is masked by the mask plate, the above "mapping" can further refer to adding the identification information of the winding package to the mask plate on the placement body where the winding package should theoretically be located based on a preset arrangement rule.

[0045] For example, as shown in FIG. 3B, using the target detection model, the region where each winding package is located in the target image 1 is masked using a mask plate to obtain an initial mask image. Then, after obtaining the labeling information of the winding packages that should be theoretically placed by each placement body on the target winding package carriage, based on the preset placement rules, the labeling information of the winding packages is added to the mask plate on the placement body where the winding package should theoretically be located in the initial mask image of the target image 1, so as to obtain a target mask image with the labeling information of the winding packages corresponding to the target image 1.

[0046] Furthermore, as shown in FIG. 3C, using the target detection model, the region where each winding package is located in the target image 2 is masked using a mask plate to obtain an initial mask image. Then, after obtaining the labeling information of the winding packages that should be theoretically placed by each placement body on the target winding package carriage, based on the preset placement rules, the labeling information of the winding packages is added to the mask plate on the placement body where the winding package should theoretically be located in the initial mask image of the target image 2, so as to obtain a target mask image with the labeling information of the winding packages corresponding to the target image 2.

[0047] Note that, as shown in FIG. 3C, since there are missing winding packages, accordingly, there is no mask plate on the placement body where the missing winding package is located in the initial mask image of the target image 2. In this case, the labeling information of the missing winding package can be added to the placement body where the missing winding package should theoretically be located in the initial mask image of the target image 2, and in this way, it is possible to support rapid targeting for subsequent specific problems.

[0048] In step S306, the target mask images of different target images are compared, and based on the comparison result, the target winding package to be the object of the target operation is determined.

[0049] In step S307, presentation information for the target winding package is generated.

[0050] Thus, the solution of the present disclosure can first use a model to detect the acquired images (for example, a plurality of target images) and obtain an initial mask image for each target image. Next, the labeling information of the winding package can be mapped to the initial mask image of each target image to obtain a target mask image with the labeling information of the winding package corresponding to each target image. Finally, based on the comparison between the target mask images of different target images, the target winding package to be the object of the target operation can be determined. The above process can determine the target winding package to be the object of the target operation without relying on manual work, realize the automation and intelligence of the whole process, further reduce a large amount of labor costs and time costs, and further ensure the normal operation of the subsequent work in the workplace.

[0051] Furthermore, in a specific example, the target winding package to be the object of the target operation can be obtained as follows. Specifically, comparing the target mask images of the above-mentioned different target images and determining the target winding package to be the object of the target operation based on the comparison result (for example, step S306 above) includes the following.

[0052] In step S306-1, compare the target mask images of different target images and determine the position of the placement body on which the winding package is not placed.

[0053] In step S306-2, based on the position of the placement body on which the winding package is not placed, determine the target winding package to be the object of the target operation.

[0054] For example, in the examples of FIGS. 3B and 3C, after comparing the target mask image of target image 1 and the target mask image of target image 2, as the target winding package to be the object of the target operation, a winding package with label 6 is obtained.

[0055] As a result, the solution of the present disclosure can quickly determine the relevant information (such as labeling information) of the winding package to be the target of the target operation by utilizing the differences between the target mask images of different target images. Compared with the inspection method by those in the prior art, the above process can quickly determine the winding package to be the target of the target operation without relying on manual work, further reducing a large amount of labor costs and time costs, and improving the inspection efficiency of the winding package.

[0056] Furthermore, in one example, the target detection model may be a Segment Anything Model (SAM) based on prior information, or may be other segmentation models having the ability to generate mask images, and the present disclosure is not limited thereto.

[0057] Furthermore, in one example, as shown in FIG. 4, the target detection model includes at least a prior knowledge feature layer, a dot pattern segmentation layer, and an image segmentation layer.

[0058] Specifically, in one example, the prior knowledge feature layer is used to obtain target prior information based on a preset winding package prompt and the input image, and the input image is one of a plurality of target images. Here, the target prior knowledge information can be used to identify the winding package by guiding the image segmentation layer, and by dividing and cutting out the area where the winding package is located, the identification and segmentation capabilities of the image segmentation layer can be enhanced, and thus it can be used to generate a mask.

[0059] Furthermore, in another example, the halftone division layer is used to divide a halftone presentation image to obtain a plurality of sub-images to be processed with halftone positions indicated, where the halftone positions in the plurality of sub-images to be processed do not overlap, and the halftone presentation image is obtained by processing an input image using halftones. For example, as shown in FIG. 5, first, using halftones, an input image, for example, the target image 1 shown in FIG. 3B, is processed to obtain a halftone presentation image corresponding to the target image 1. Next, the obtained halftone presentation image is divided, for example, divided row by row, to obtain a plurality of sub-images to be processed, and the halftones between the respective sub-images to be processed do not overlap with each other. In this way, it is useful for performing image processing in a batch manner on each sub-image to be processed, effectively avoiding repeated recognition, and laying a foundation for further improving the identification efficiency.

[0060] In yet another example, the image division layer identifies the winding packages in each sub-image to be processed based on the target prior knowledge information, divides the regions where each winding package is located, and further masks the regions where each winding package is located in the sub-image to be processed using a mask plate to obtain a sub-mask image of each sub-image to be processed. Further, after obtaining the sub-mask image of each sub-image to be processed, based on the sub-mask image of each sub-image to be processed, for example, by combining the sub-mask images of each sub-image to be processed, it is used to obtain an initial mask image of the input image.

[0061] Thereby, the solution of the present disclosure can utilize the target prior knowledge information to enhance the identification and division capabilities of the image division layer, can realize batch-type image processing based on the halftone presentation image, and thus can improve the division efficiency. In this way, it provides support for automatically and intelligently obtaining the winding package targeted for the target operation, and also provides support for improving the inspection efficiency.

[0062] In a specific example of the present disclosure, the prior knowledge feature layer includes at least a semantics prior knowledge layer and a similar map prior knowledge layer.

[0063] Here, in one example, the semantic prior knowledge layer is used to obtain semantic prior knowledge features based on at least the winding package features corresponding to the preset winding package prompts. For example, in one example, the winding package features corresponding to the preset winding package prompts can be directly used as the semantic prior knowledge features.

[0064] Furthermore, in another example, the semantic prior knowledge layer can also obtain semantic prior knowledge features as follows. Specifically, as shown in FIG. 6, the Semantics prior knowledge layer performs feature fusion on the winding package features corresponding to the preset winding package prompts and the image features of the input image (for example, the global feature map of the input image), Semantics and is used to obtain a feature map representing prior knowledge (that is, Semantics prior knowledge features). For example, the winding package features corresponding to the preset winding package prompts and the image features of the input image are multiplied element by element to obtain a feature map for representing semantic prior knowledge features.

[0065] Note that in this example, when the dimension of the winding package features corresponding to the preset winding package prompts does not match the dimension of the image features of the input image, an upsampling process (for example, bilinear interpolation process) is performed on the winding package features corresponding to the preset winding package prompts so that the dimension of the processed winding package features is the same as the dimension of the image features of the input image, and then the two need to be feature-fused. In this way, the feature information of the obtained semantic prior knowledge features is made more abundant, laying a foundation for further enhancing the identification and segmentation capabilities of the image segmentation layer.

[0066] Furthermore, in another example, the similar map prior knowledge layer is used to estimate the regions where each winding package is located in the input image based on the similarity between the winding package features corresponding to the preset winding package prompts and the image features of the input image, and to obtain a target similarity map.

[0067] Note that it should be noted that the above-mentioned target prior knowledge information includes the semantics prior knowledge features and the target similarity map.

[0068] Furthermore, in one example, the similarity map prior knowledge layer can determine the target similarity map as follows. Specifically, the similarity map prior knowledge layer is specifically used to estimate the regions where each winding package exists in the input image based on the similarity between the obtained semantics prior knowledge features and the image features of the input image, and then obtain the target similarity map. For example, as shown in FIGS. 6 and 7, specifically, For the obtained Semantics Aggregation processing is performed on the prior knowledge features to obtain the prior knowledge features after aggregation processing. For example, Semantics The pixel values of the feature vector representing the prior knowledge features are added column by column (or the average value, etc.) to obtain the feature vector after aggregation processing. Semantics A plurality of sub-feature vectors of the global feature map of the input image are obtained. For example, in one example, the feature vector representing the global feature map is divided, for example, divided row by row, to obtain a plurality of sub-feature vectors. At this time, the dimension of each obtained sub-feature vector is the same as the dimension of the feature vector after aggregation processing. In this way, it is useful for calculating the similarity between the two. The similarity between each sub-feature vector in the plurality of sub-feature vectors and the feature vector after aggregation processing is obtained, and based on the similarity, the target similarity map is obtained. This is used for

[0069] Thereby, the regions where the winding packages are located in the input image can be accurately specified. Subsequently, the winding packages in the image can be accurately identified and separated, laying a foundation for obtaining the mask image.

[0070] Furthermore, in one example, the prior knowledge feature layer can include a feature enhancement layer. For example, in order to perform feature enhancement on the target similarity map, the obtained target similarity map can be input into the feature enhancement layer to obtain a feature-enhanced target similarity map. In this case, the image segmentation layer can specifically perform identification and segmentation based on the semantic prior knowledge features and the feature-enhanced target similarity map, thereby further improving the accuracy of image identification and segmentation.

[0071] Alternatively, in another example, the prior knowledge feature layer can include a labeling layer. For example, as shown in FIG. 6, the obtained target similarity map (or the feature-enhanced target similarity map) is input into the labeling layer, and labeling processing is performed on the input target similarity map to obtain a label feature map. In this case, the target prior knowledge information can specifically include the semantic prior knowledge features and the label feature map.

[0072] Here, in the label feature map (for example, using "0" and "1" for labeling), if the value of a certain area is 1, it indicates that the area is a positive area, that is, there is a winding package or a part of the winding package, and otherwise it indicates a negative area. This helps the image segmentation layer to ignore the negative area and focus on segmenting the positive area, thereby further enhancing the identification and segmentation ability of the winding package, and thus helping to more accurately identify and separate the winding package in the image, and effectively improving the efficiency of identification and segmentation.

[0073] It should be noted that the above-mentioned prior knowledge feature layer can include a feature enhancement layer, or a labeling layer, or a feature enhancement layer and a labeling layer, etc., or can also include other processing layers for improving image recognition and segmentation capabilities, and can be set according to actual needs during actual application. It should be noted that the present disclosure is not specifically limited thereto.

[0074] In this way, the solution of the present disclosure utilizes the semantic prior knowledge features and the target similarity map in the target prior knowledge information, enabling the image segmentation layer to focus on the identification and segmentation of the winding package in the image, improving the image identification and segmentation capabilities of the image segmentation layer, and thereby effectively improving the identification accuracy and efficiency.

[0075] The solution of the present disclosure also provides a detection device applied to the cloud, as shown in FIG. 8. A detection device applied to a cloud terminal, When it is determined that there is a target operation on a target object in the video data of the target winding package cart, an information determination unit 801 for determining a target area targeted by the target operation and acquiring a plurality of target images capable of covering the target area. The target winding package cart includes two areas each having N placement bodies for placing the winding packages, the target area is one of the two areas, and N is a positive integer. The information determination unit 801, A detection unit 802 for determining a target winding package targeted by the target operation based on the plurality of target images, And a presentation unit 803 for generating presentation information for the target winding package.

[0076] In a specific example of the solution of the present disclosure, the detection unit specifically Inputs each target image in the plurality of target images into a target detection model to obtain an initial mask image of each target image. The target detection model can identify the area where each winding package is located in the input image based on a preset winding package prompt, and mask the area where each winding package is located in the image using a mask plate to obtain a masked image. Obtaining the identification information of the winding package to be placed by each placement body in the target winding package cart based on the identification information of the target winding package cart. Mapping the label information of the winding package to be placed on each placement body in the target winding package carriage to each mask plate at different positions in the initial mask image of each target image, obtaining a target mask image with the label information of the winding package corresponding to each target image, where the target mask image with the label information of the winding package corresponding to the target image can represent the label information of each winding package actually included in the target image, and Comparing the target mask images of different target images and determining the target winding package to be the target of the target operation based on the comparison result.

[0077] In a specific example of the solution of the present disclosure, the detection unit specifically Compares the target mask images of different target images and determines the positions of the placement bodies on which the winding packages are not placed. Based on the positions of the placement bodies on which the winding packages are not placed, determines the target winding package to be the target of the target operation.

[0078] In a specific example of the solution of the present disclosure, the target detection model includes at least a prior knowledge feature layer, a dot pattern segmentation layer, and an image segmentation layer. The prior knowledge feature layer is used to obtain target prior knowledge information based on a preset winding package prompt and an input image that is one of a plurality of target images. The dot pattern segmentation layer is used to segment a dot pattern prompt image obtained by processing the input image using dot patterns, obtaining a plurality of processing target sub-images with dot pattern positions indicated, where the dot pattern positions in different processing target sub-images do not overlap. The image segmentation layer is used to identify the winding packages in each processing target sub-image based on the target prior knowledge information, mask the regions where each winding package is located in the processing target sub-image using a mask plate, obtain a sub-mask image of each processing target sub-image, and obtain an initial mask image of the input image based on the sub-mask images of each processing target sub-image.

[0079] In a specific example of the solution of the present disclosure, the prior knowledge feature layer includes at least a semantics prior knowledge layer and a similarity map prior knowledge layer, The semantics prior knowledge layer is used to obtain semantics prior knowledge features based on at least the winding package features corresponding to the preset winding package prompts, The similarity map prior knowledge layer is used to estimate the regions where each winding package is located in the input image based on the similarity between the winding package features corresponding to the preset winding package prompts and the image features of the input image, so as to obtain a target similarity map, The target prior knowledge information includes the semantics prior knowledge features and the target similarity map.

[0080] In a specific example of the solution of the present disclosure, the Semantics prior knowledge layer specifically features the winding package features corresponding to the preset winding package prompts and the image features of the input image, and Semantics is used to obtain prior knowledge features.

[0081] In a specific example of the solution of the present disclosure, the similarity map prior knowledge layer specifically is used to estimate the regions where each winding package exists in the input image based on the similarity between the obtained semantics prior knowledge features and the image features of the input image, so as to obtain a target similarity map.

[0082] For the specific functions and exemplary descriptions of each module of the device according to the embodiments of the present disclosure, reference can be made to the relevant descriptions of the corresponding steps in the embodiments of the above method, which will not be repeated here.

[0083] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0084] FIG. 9 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 9, the electronic device includes a memory 910 and a processor 920, and a computer program executable by the processor 920 is stored in the memory 910. The number of the memory 910 and the processor 920 can be one or more. The memory 910 can store one or more computer programs, and when the one or more computer programs are executed by the electronic device, the electronic device is caused to execute the method provided by the above method embodiment. The electronic device can further include the following. The communication interface 930 is used to communicate with an external device and perform data interaction and transmission.

[0085] When the memory 910, the processor 920, and the communication interface 930 are independently implemented, the memory 910, the processor 920, and the communication interface 930 are connected to each other via a bus and can communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be classified into an address bus, a data bus, a control bus, etc. For ease of explanation, only a single thick line is shown in FIG. 9, but it does not represent only a single bus or a single type of bus.

[0086] Optionally, in a specific implementation form, when the memory 910, the processor 920, and the communication interface 930 are integrated on one chip, the memory 910, the processor 920, and the communication interface 930 can communicate with each other via an internal interface.

[0087] The above-mentioned processor may be a Central Processing Unit (CPU), and it should be understood that it may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware assemblies, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. Note that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0088] Furthermore, optionally, the memory may include a read-only memory and a random access memory, and may further include a non-volatile random access memory. The memory may be either a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. Here, the non-volatile memory can include ROM (Read-Only Memory), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically EPROM), or flash memory. The volatile memory can include a random access memory (Random Access Memory, RAM) that functions as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), synchronous DRAM (Synchronous DRAM, SDRAM), double data rate SDRAM (Double Data Rate SDRAM, DDR SDRAM), enhanced SDRAM (Enhanced SDRAM, ESDRAM), synchlink DRAM (Synchlink DRAM, SLDRAM), and direct RAMBUS RAM (Direct RAMBUS RAM, DR RAM).

[0089] In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of it may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present disclosure are generated in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or may be transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, Bluetooth®, microwave, etc.). The computer-readable storage medium may be any available medium accessible by a computer, or a data storage device including a server, a data center, etc. integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy (registered trademark) disk, hard disk, magnetic tape), an optical medium (such as a Digital Versatile Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)), etc. It should be noted that the computer-readable storage medium referred to in the present disclosure may be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0090] Those skilled in the art can understand that all or some of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0091] In the description of the embodiments of the present disclosure, the description of reference terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or features described in relation to the embodiment or example are included in at least one embodiment or example of the present disclosure. And the specific features, structures, materials, or features described can be combined in any one or more embodiments or examples in an appropriate manner. Furthermore, those skilled in the art may combine the different embodiments or examples described in the present disclosure and the features of the different embodiments or examples within a non-conflicting range.

[0092] In the description of the embodiments of the present disclosure, " / " represents the meaning of "or" unless otherwise specified. For example, A / B may represent either A or B. The "and / or" in the present disclosure only explains the relationship of related objects, indicating that three types of relationships may exist. For example, A and / or B can represent the following. There are three situations where A exists alone, A and B exist simultaneously, and B exists alone.

[0093] In the description of the embodiments of the present disclosure, the terms "first" and "second" are used only for the purpose of description and should not be construed as indicating or implying relative importance, nor should they be construed as implying the number of the technical features shown. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, "a plurality" means two or more unless otherwise specified.

[0094] The above are only exemplary embodiments of the present disclosure and do not limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle scope of the present disclosure should all be included within the protection scope of the present disclosure.

Claims

1. A detection method applied to a cloud terminal, comprising: Based on video data obtained by performing video collection on two placement areas of a target winding package cart located in a storage area, when it is determined that there is a target operation of carrying out a winding package on the target body in the video data of the target winding package cart, determining a target area where the target winding package targeted by the target operation is located, wherein the two placement areas of the target winding package cart each have N placement bodies for placing winding packages, the target area is one of the two placement areas, and N is a positive integer; Obtaining a plurality of target images capable of covering the target area; Based on the plurality of target images, determining the target winding package targeted by the target operation, and generating presentation information for the target winding package. The detection method.

2. Based on the plurality of target images, determining the target winding package targeted by the target operation includes: Inputting each target image in the plurality of target images into a target detection model to obtain an initial mask image of each target image, wherein the target detection model can identify the area where each winding package is located in the input image based on a preset winding package prompt, and mask the area where each winding package is located in the image using a mask version to obtain a masked image; Obtaining the identification information of the winding packages to be placed by each placement body in the target winding package cart based on the identification information of the target winding package cart; Mapping the identification information of the winding packages to be placed by each placement body in the target winding package cart to each mask version at different positions in the initial mask image of each target image to obtain a target mask image with the identification information of the winding package corresponding to each target image, wherein the target mask image with the identification information of the winding package corresponding to the target image can represent the identification information of each winding package actually included in the target image; Comparing the target mask images of different target images, and determining the target winding package targeted by the target operation based on the comparison result. The detection method according to Claim 1.

3. Comparing the target mask images of the different target images and determining the target winding package to be the target of the target operation based on the comparison result, comparing the target mask images of different target images and determining the position of the placement body on which the winding package is not placed, and determining the target winding package to be the target of the target operation based on the position of the placement body on which the winding package is not placed, The detection method according to claim 2.

4. The target detection model includes at least a prior knowledge feature layer, a halftone segmentation layer, and an image segmentation layer, The prior knowledge feature layer is used to obtain target prior knowledge information based on a preset winding package prompt and an input image that is one of a plurality of target images, The halftone segmentation layer is used to divide a halftone prompt image obtained by processing the input image using halftones, and obtain a plurality of processing target sub-images with halftone positions indicated, where the halftone positions in different processing target sub-images do not overlap, The image segmentation layer is used to identify the winding packages in each processing target sub-image based on the target prior knowledge information, mask the regions where each winding package is located in the processing target sub-image using a mask plate to obtain a sub-mask image of each processing target sub-image, and obtain an initial mask image of the input image based on the sub-mask images of each processing target sub-image, The detection method according to claim 2.

5. The prior knowledge feature layer includes at least a semantics prior knowledge layer and a similarity map prior knowledge layer, The semantics prior knowledge layer is used to obtain semantics prior knowledge features based on at least the winding package features corresponding to a preset winding package prompt, The similarity map prior knowledge layer is used to estimate the regions where each winding package is located in the input image based on the similarity between the winding package features corresponding to a preset winding package prompt and the image features of the input image, and obtain a target similarity map, The target prior knowledge information includes the semantics prior knowledge features and the target similarity map, The detection method according to claim 4.

6. Specifically, the semantics prior knowledge layer It is used to obtain semantic prior knowledge features by fusing the winding package features corresponding to the preset winding package prompts and the image features of the input image. The detection method according to claim 5.

7. Specifically, the similar map prior knowledge layer Based on the similarity between the obtained semantic prior knowledge features and the image features of the input image, it is used to estimate the regions where each winding package exists in the input image and obtain a target similarity map. The detection method according to claim 5.

8. A detection device applied to a cloud terminal, An information determination unit for determining a target region where a target winding package, which is the target of the target operation, is located and acquiring a plurality of target images capable of covering the target region, based on video data obtained by performing video collection on two placement regions of a target winding package trolley located in a storage area. The two placement regions of the target winding package trolley each have N placement bodies for placing winding packages, the target region is one of the two placement regions, and N is a positive integer. A detection unit for determining the target winding package that is the target of the target operation based on the plurality of target images; And a presentation unit for generating presentation information for the target winding package. Detection device.

9. At least one processor; A memory communicatively connected to the at least one processor, The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is caused to execute the method according to any one of claims 1 to 7. Electronic device.

10. A non-transitory computer-readable storage medium for storing instructions that cause a computer to execute the method according to any one of claims 1 to 7.

11. A program for realizing the method according to any one of claims 1 to 7 when executed by a processor in a computer.

Images