Open building block splicing prompt method, device, electronic device and storage medium

Abstract

An embodiment of the present invention provides a method for prompting the splicing of open-type building blocks, comprising: obtaining an image to be recognized of a target area at a current moment, wherein the image to be recognized includes open-type building blocks; performing image recognition on the image to be recognized to obtain the real-time construction status of the open-type building blocks in the image to be recognized at the current moment; and providing prompts to users for splicing the open-type building blocks based on the real-time construction status. By obtaining an image of the open-type building blocks in the target area in real time, using image recognition technology to identify the real-time construction status of the open-type building blocks in the image at the current moment, and providing real-time prompts to users for splicing, the users can know the construction status of the open-type building blocks in advance, discover their own splicing errors in a timely manner, and avoid having to re-splice after errors occur after splicing is completed. This reduces the difficulty of splicing the open-type building blocks, thereby reducing the error rate of splicing the open-type building blocks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an open-type building block splicing prompt method, device, electronic device and storage medium. Background Art

[0002] Block programming can be used to develop children's intelligence. Typical block programming suppliers provide blocks and blueprints. The blueprints include the selection of components and the order in which they should be assembled. Users can then directly create the desired finished product by following the steps outlined in the blueprints. However, since open-ended blocks lack blueprints and a fixed connection sequence, structural errors can easily occur during assembly. For example, insufficient mobility can lead to the inability to perform the desired actions after programming the finished product, forcing the user to reassemble the blocks. Consequently, existing open-ended block programming suffers from the difficulty of block assembly, resulting in a high error rate. Summary of the Invention

[0003] An embodiment of the present invention provides an open-type building block splicing prompt method, which aims to solve the problem that existing open-type building block programming has high splicing difficulty and leads to high splicing error rate. By acquiring images of open-type building blocks in a target area in real time, using image recognition technology to identify the real-time construction status of the open-type building blocks in the image at the current moment, and providing real-time prompts to splicing users, the splicing users can know the construction status of the open-type building blocks in advance and discover their own splicing errors in time, avoiding errors after splicing is completed and having to re-splice, thereby reducing the splicing difficulty of the open-type building blocks and thus reducing the splicing error rate of the open-type building blocks.

[0004] In a first aspect, an embodiment of the present invention provides an open-type building block splicing prompt method, comprising the following steps:

[0005] Obtaining an image to be recognized of the target area at a current moment, wherein the image to be recognized includes open building blocks;

[0006] Performing image recognition on the image to be recognized to obtain a real-time building state of the open-ended building blocks in the image to be recognized at the current moment;

[0007] According to the real-time building status, the user is prompted on the splicing of the open building blocks.

[0008] Furthermore, performing image recognition on the image to be recognized to obtain the real-time building state of the open-ended building blocks in the image to be recognized at the current moment includes:

[0009] Performing image recognition on the image to be recognized, and extracting the to-be-built state of the open-type building blocks in the image to be recognized;

[0010] Obtaining the built state of the open building blocks in the image to be recognized before the current moment;

[0011] According to the built state and the to-be-built state, a real-time built state of the open-ended building blocks in the image to be recognized at the current moment is determined.

[0012] Furthermore, the open-type building blocks include building block assemblies, each of which includes a splicing portion and a connecting portion. For the same building block assembly, the splicing portions are connected via the connecting portion, and for different building block assemblies, they are connected via their respective splicing portions. The performing of image recognition on the image to be recognized and extracting the state of the open-type building blocks to be built in the image to be recognized includes:

[0013] Performing feature extraction on the image to be identified to extract feature points of the splicing portion and feature lines of the connecting portion of the building block assembly to be spliced;

[0014] Connecting the joint feature points through the connection feature lines to obtain a node graph of the building block assembly to be spliced, wherein the nodes in the node graph of the building block assembly to be spliced ​​are the joint feature points, and the connections in the node graph are the connection feature lines;

[0015] Based on the node graph of the to-be-assembled building block assembly, the to-be-assembled state of the open-type building blocks in the to-be-recognized image is obtained.

[0016] Furthermore, the feature extraction of the image to be identified to extract the feature points of the splicing portion and the feature lines of the connection portion of the building block assembly to be spliced ​​includes:

[0017] Determining a to-be-recognized area of ​​the to-be-recognized image, wherein the to-be-recognized area includes building block components to be assembled;

[0018] Feature extraction is performed on the area to be identified, and feature points of the splicing portion and feature lines of the connecting portion of the building block assembly to be spliced ​​are extracted.

[0019] Furthermore, the step of prompting the user on the splicing of the open-ended building blocks according to the real-time building status includes:

[0020] Calculating the activity of the open building blocks in the image to be recognized at the current moment according to the real-time building state;

[0021] When the activity level is less than a preset activity level threshold, a user who is assembling the open-ended building blocks is prompted, and the activity level threshold is determined according to a program to be executed.

[0022] Furthermore, obtaining the image to be identified of the target area at the current moment includes:

[0023] At a current moment, a plurality of cameras arranged in the target area are used to obtain multi-angle images of the target area;

[0024] An image that meets a preset condition is selected from the multi-angle images as the image to be identified in the target area at the current moment.

[0025] Furthermore, selecting an image that meets a preset condition from the multi-angle image as the image to be identified in the target area at the current moment includes:

[0026] In the multi-angle image, calculating the image area of ​​the component to be spliced;

[0027] From the multi-angle images, an image whose image area of ​​the component to be spliced ​​is greater than or equal to a preset area threshold is selected as the image to be identified of the target area at the current moment, wherein the area threshold is determined according to the number of splicing parts of the component to be spliced.

[0028] In a second aspect, an embodiment of the present invention provides a behavior detection device, comprising:

[0029] An acquisition module, configured to acquire an image to be identified of a target area at a current moment, wherein the image to be identified includes open building blocks;

[0030] a recognition module, configured to perform image recognition on the image to be recognized and obtain a real-time construction state of the open-ended building blocks in the image to be recognized at the current moment;

[0031] The prompt module is used to prompt the user of the splicing of the open building blocks according to the real-time building status.

[0032] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the open-type building block splicing prompt method provided in an embodiment of the present invention are implemented.

[0033] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the open-type building block splicing prompt method provided in the embodiment of the invention are implemented.

[0034] In an embodiment of the present invention, an image to be recognized of a target area at a current moment is obtained, wherein the image to be recognized includes open-type building blocks; image recognition is performed on the image to be recognized to obtain the real-time construction status of the open-type building blocks in the image to be recognized at the current moment; and based on the real-time construction status, a prompt is provided to the user for assembling the open-type building blocks. By acquiring an image of the open-type building blocks in the target area in real time, using image recognition technology to identify the real-time construction status of the open-type building blocks in the image at the current moment, and providing a real-time prompt to the user for assembling the open-type building blocks, the user can know the construction status of the open-type building blocks in advance, discover their own assembly errors in a timely manner, and avoid having to reassemble the blocks due to errors after assembly is completed. This reduces the difficulty of assembling the open-type building blocks, thereby reducing the error rate of assembling the open-type building blocks. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 This is a flow chart of an open-type building block splicing prompt method provided by an embodiment of the present invention;

[0037] Figure 2 This is a structural diagram of an open-type building block splicing prompt device provided by an embodiment of the present invention;

[0038] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0040] See Figure 1 , Figure 1 This is a flow chart of an open-type building block splicing prompt method provided by an embodiment of the present invention. Figure 1 As shown, the open building block splicing prompt method includes the following steps:

[0041] 101. Obtain an image to be recognized of the target area at the current moment.

[0042] In the embodiment of the present invention, the target area is a splicing area of ​​open-type building blocks. The target area may be a specific area on a desk or a workbench, and the splicing user splices the open-type building blocks in the target area.

[0043] After the open blocks are assembled, the server can obtain an image of the target area in real time, thereby obtaining an image to be recognized at the current moment. The image to be recognized includes the open blocks. Specifically, the image to be recognized includes the open blocks being assembled.

[0044] Specifically, the target area is the image acquisition area of ​​the camera. The camera acquires an image of the target area, and the image to be identified is obtained and uploaded to the server.

[0045] 102. Perform image recognition on the image to be recognized to obtain the real-time construction status of the open building blocks in the image to be recognized at the current moment.

[0046] In the embodiment of the present invention, image recognition refers to the technology of using computers to process, analyze and understand images to identify targets and objects of various different patterns, and is a practical application of deep learning algorithms.

[0047] The above-mentioned open building block splicing prompt method is deployed in the server. The server obtains the image to be recognized from the camera, and performs image recognition on the image to be recognized through the image recognition engine in the server to obtain the real-time construction status of the open building blocks in the image to be recognized at the current moment.

[0048] The above-mentioned real-time building status refers to the current building status, which includes a normal building status and an abnormal building status. The normal building status indicates that there is no error in the open-ended building blocks spliced ​​by the user, and the abnormal building status indicates that there is an error in the open-ended building blocks spliced ​​by the user.

[0049] 103. Prompt users on the splicing of open building blocks based on the real-time building status.

[0050] In an embodiment of the present invention, after performing image recognition on the image to be recognized, the server obtains the real-time construction status of the open building blocks in the image to be recognized at the current moment. When the real-time construction status is in a normal construction state, the server can prompt the splicing user that the splicing is normal; when the real-time construction status is in an abnormal construction state, the server can prompt the splicing user that the splicing is abnormal.

[0051] The prompt can be a voice prompt or a light prompt. In an embodiment of the present invention, the prompt can be performed through a display screen, which displays the real-time building status so that the splicing user can intuitively understand the real-time building status of the open building blocks.

[0052] In an embodiment of the present invention, an image to be recognized of a target area at a current moment is obtained, wherein the image to be recognized includes open-type building blocks; image recognition is performed on the image to be recognized to obtain the real-time construction status of the open-type building blocks in the image to be recognized at the current moment; and based on the real-time construction status, a prompt is provided to the user for assembling the open-type building blocks. By acquiring an image of the open-type building blocks in the target area in real time, using image recognition technology to identify the real-time construction status of the open-type building blocks in the image at the current moment, and providing a real-time prompt to the user for assembling the open-type building blocks, the user can know the construction status of the open-type building blocks in advance, discover their own assembly errors in a timely manner, and avoid having to reassemble the blocks due to errors after assembly is completed. This reduces the difficulty of assembling the open-type building blocks, thereby reducing the error rate of assembling the open-type building blocks.

[0053] Optionally, in the step of performing image recognition on the image to be recognized and obtaining the real-time construction state of the open-type building blocks in the image to be recognized at the current moment, image recognition can be performed on the image to be recognized to extract the state to be constructed of the open-type building blocks in the image to be recognized; the constructed state of the open-type building blocks in the image to be recognized before the current moment is obtained; and the real-time construction state of the open-type building blocks in the image to be recognized at the current moment is determined based on the constructed state and the state to be constructed.

[0054] In the embodiment of the present invention, the above-mentioned to-be-built state refers to the state of the part that has not been built, and the above-mentioned built state refers to the state of the part that has been built. The splicing process of the open building blocks is to splice the part that has not been built to the part that has been built, so as to obtain a spliced ​​finished product.

[0055] Furthermore, the unbuilt portion can be the unbuilt portion in the user's hand or on their desk. The built state can be the real-time built state at the previous moment, and the current real-time built state is equal to the real-time built state at the previous moment plus the pending state at the current moment. When the current moment is the initial moment of the open block assembly, the built state can be 0 or empty.

[0056] It should be noted that the "built state" refers to the state of a completed portion. However, a completed portion may be severely obstructed. Therefore, the present invention uses the previous real-time state as the "built state," avoiding image recognition of the completed portion and reducing the recognition error rate. Furthermore, by eliminating the need to perform image recognition on the severely obstructed completed portion, computational speed can be increased.

[0057] Through the current state to be built and the built state before the current moment, the building state of the open building blocks being assembled can be monitored in real time, so that the user who is assembling can be prompted in time according to the real-time building state of the open building blocks at the current moment.

[0058] Optionally, the open building block includes a building block assembly, and the building block assembly includes a splicing part and a connecting part. For the same building block assembly, the splicing parts are connected through the connecting part, and for different building block assemblies, they are connected through their respective splicing parts. In the step of performing image recognition on the image to be identified and extracting the state to be built of the open building block in the image to be identified, feature extraction can be performed on the image to be identified, and the splicing part feature points and connecting part feature lines of the building block assembly to be spliced ​​are extracted; the splicing part feature points are connected through the connecting part feature lines to obtain a node graph of the building block assembly to be spliced, the nodes in the node graph of the building block assembly to be spliced ​​are the splicing part feature points, and the connections in the node graph are the connecting part feature lines; based on the node graph of the building block assembly to be spliced, the state to be built of the open building block in the image to be identified is obtained.

[0059] In an embodiment of the present invention, in the same building block assembly, two splicing parts are connected by a connecting part, and one connecting part includes at least one male end and at least one female end. The male end and the female end cooperate to connect the splicing parts of different building block assemblies, thereby splicing different building block assemblies.

[0060] The to-be-assembled building block components refer to building block components that have not yet been assembled onto the building block body, while the building block body is a combination of already assembled building block components. The image to be recognized includes the to-be-assembled building block components and the already assembled building block component combination at the current moment, wherein the to-be-assembled building block components correspond to the to-be-assembled state, and the already assembled building block component combination corresponds to the already-assembled state.

[0061] Furthermore, the aforementioned building block assembly to be assembled may refer to a building block assembly that the user holds in his hand and is ready to assemble into a building block body.

[0062] A pre-trained feature extraction network can be used to extract features from the building blocks to be assembled. Feature points and connection lines of the building blocks to be assembled are extracted, and the feature points are connected through the connection lines to obtain a node graph of the building blocks to be assembled. The node graph of the building blocks to be assembled can be understood as an abstract representation of the building blocks to be assembled. The node graph of the building blocks to be assembled can be input into a pre-trained graph convolutional network for processing to obtain the ready-to-assemble state of the open-ended building blocks in the image to be identified.

[0063] In the embodiment of the present invention, only the building blocks to be assembled are used for feature extraction, the building blocks to be assembled are abstracted into a node graph, and the current state to be built is calculated through the node graph of the building blocks to be assembled, which can improve the calculation speed.

[0064] Optionally, in the step of performing feature extraction on the image to be identified and extracting feature points of the splicing portion and feature lines of the connection portion of the building block assembly to be assembled, the area to be identified of the image to be identified can be determined, and the area to be identified includes the building block assembly to be assembled; and feature extraction is performed on the area to be identified, and feature points of the splicing portion and feature lines of the connection portion of the building block assembly to be assembled are extracted.

[0065] In an embodiment of the present invention, target detection can be performed on the image to be identified through a target detection network deployed in the server. The object of the target detection is the building block component to be spliced, and the position area of ​​the building block component to be spliced ​​in the image to be identified is obtained, so that the position area of ​​the building block component to be spliced ​​in the image to be identified is determined as the area to be identified.

[0066] In one possible embodiment, after obtaining the area to be identified, the area to be identified can be expanded to obtain an expanded area to be identified, and feature extraction is performed on the expanded area to be identified to extract the feature points of the splicing portion and the feature lines of the connection portion of the building block assembly to be spliced. Specifically, the area to be identified is expanded in the direction of the building block body until the expanded area to be identified includes a portion of the building block body, for example, until the area to be identified is expanded to include one-third of the area of ​​the building block body. In this way, it can be determined to which part of the building block body the user wants to splice the building block assembly to be spliced, so that when the user splices the building block assembly to be spliced ​​to the building block body, information about the splicing of the splicing portion of the building block assembly to be spliced ​​and the splicing portion of the building block body can be obtained.

[0067] Optionally, in the step of prompting the user to assemble the open-ended building blocks based on the real-time building status, the activity of the open-ended building blocks in the image to be identified at the current moment can be calculated based on the real-time building status; when the activity is less than a preset activity threshold, the user to assemble the open-ended building blocks is prompted, and the activity threshold is determined according to the program to be executed.

[0068] In an embodiment of the present invention, the real-time building state can be obtained by adding the state to be built and the state to be built. Specifically, the state to be built can be the node graph of the building block assembly to be spliced, and the built state can be the node graph of the building block master. The node graph of the building block assembly to be spliced ​​is connected with the node graph of the building block master to obtain a real-time node graph, and the real-time node graph is input into the pre-trained 3D graph convolutional network for processing to obtain the activity of the open building blocks in the image to be identified at the current moment. In an embodiment of the present invention, the input object of the 3D graph convolutional network is the real-time node graph of the open building blocks. The real-time node graph is a three-dimensional node graph, and the 3D graph convolutional network is the activity of each node in the real-time node graph.

[0069] In a possible embodiment, the splicing end of the building block assembly to be spliced ​​includes a degree of mobility, and the degree of mobility is determined according to the movable range of the building block assembly spliced ​​on the splicing end. If the movable range is 360 degrees, the degree of mobility is 1, and if the movable range is 0 degrees, the degree of mobility is 0.

[0070] The activity threshold can be determined based on the program to be executed, which can be determined in advance. For example, if a program corresponding to a rotation action is to be executed between one building block component and another building block component, the activity of the node corresponding to the splicing end of the two components needs to be 1, that is, after the two building blocks are spliced, the range of motion is 360 degrees. The activity of the node corresponding to the splicing end of the two building blocks after splicing can be expressed as follows:

[0071] R=r1·r2

[0072] The above R is the activity of the node corresponding to the splicing end of the two building block components, r1 is the activity corresponding to the splicing end of one building block component, and r2 is the activity corresponding to the splicing end of the other building block component.

[0073] When the activity level is less than the preset activity level threshold, it means that the current open block cannot complete the program instructions to be executed and there is a structural error.

[0074] The activity of the open building blocks in the image to be identified at the current moment is calculated through the real-time building status. When the activity is less than the preset activity threshold, the user of the open building blocks is prompted, and the error prompt can be given to the user in time.

[0075] Optionally, in the step of obtaining the image to be identified of the target area at the current moment, multi-angle images of the target area can be obtained at the current moment by setting up multiple cameras in the target area; and an image that meets preset conditions is selected from the multi-angle images as the image to be identified of the target area at the current moment.

[0076] In an embodiment of the present invention, multiple cameras can be set up in the target area to collect images from various angles of the user in the process of splicing open building blocks, and images that meet preset conditions are selected as the images to be recognized in the target area at the current moment, thereby improving the accuracy of image recognition.

[0077] Optionally, in the step of selecting an image that meets preset conditions from the multi-angle image as the image to be identified in the target area at the current moment, the image area of ​​the component to be spliced ​​can be calculated in the multi-angle image; and from the multi-angle image, an image whose image area of ​​the component to be spliced ​​is greater than or equal to a preset area threshold is selected as the image to be identified in the target area at the current moment, wherein the area threshold is determined according to the number of splicing parts of the component to be spliced.

[0078] In an embodiment of the present invention, by calculating the image area of ​​the components to be spliced ​​in multi-angle images, one or more images with better image quality are determined as the images to be recognized in the target area at the current moment, thereby avoiding the influence of occlusion or angle, improving the accuracy of image recognition, and thus improving the accuracy of prompts when prompting users, and reducing the splicing error rate of open-ended building blocks.

[0079] It should be noted that the more splicing parts there are, the larger the volume of the component to be spliced, and the larger the area occupied in the image to be identified. In order to eliminate the influence of the shooting angle, the area threshold is set to be positively correlated with the number of splicing parts of the component to be spliced. This can improve the image quality of the image to be identified and thus improve the accuracy of image recognition.

[0080] It should be noted that the open building block splicing prompt method provided by the embodiment of the present invention can be applied to devices such as smart phones, computers, and servers.

[0081] Optional, see Figure 2 , Figure 2 : is a structural diagram of an open building block splicing prompt device provided by an embodiment of the present invention, such as Figure 2 As shown, the device includes:

[0082] An acquisition module 201 is configured to acquire an image to be recognized of a target area at a current moment, wherein the image to be recognized includes open building blocks;

[0083] The recognition module 202 is configured to perform image recognition on the image to be recognized and obtain the real-time construction status of the open-ended building blocks in the image to be recognized at the current moment;

[0084] The prompt module 203 is used to prompt the user of the splicing of the open building blocks according to the real-time building status.

[0085] Furthermore, the identification module 202 includes:

[0086] an identification submodule, configured to perform image recognition on the image to be identified and extract the state of the open-ended building blocks in the image to be identified;

[0087] An acquisition submodule, configured to acquire the built state of the open-ended building blocks in the image to be recognized before the current moment;

[0088] The determination submodule is configured to determine the real-time construction state of the open-type building blocks in the image to be identified at the current moment according to the constructed state and the to-be-constructed state.

[0089] Furthermore, the open building block includes a building block assembly, and the building block assembly includes a splicing portion and a connecting portion. For the same building block assembly, the splicing portions are connected by the connecting portion, and for different building block assemblies, they are connected by their respective splicing portions. The identification submodule includes:

[0090] An extraction unit, configured to perform feature extraction on the image to be identified, and extract feature points of the splicing portion and feature lines of the connecting portion of the building block assembly to be spliced;

[0091] a connecting unit, configured to connect the joint feature points through the connection feature lines to obtain a node graph of the building block assembly to be joined, wherein the nodes in the node graph of the building block assembly to be joined are the joint feature points, and the connections in the node graph are the connection feature lines;

[0092] The processing unit is configured to obtain a state of the open-type building blocks in the image to be identified based on a node graph of the building block assembly to be assembled.

[0093] Furthermore, the extraction unit includes:

[0094] A determination subunit, configured to determine a region to be identified of the image to be identified, wherein the region to be identified includes building block components to be assembled;

[0095] The extraction subunit is used to perform feature extraction on the area to be identified, and extract the feature points of the splicing part and the feature lines of the connecting part of the building block assembly to be spliced.

[0096] Furthermore, the prompt module 203 includes:

[0097] A calculation submodule, configured to calculate the activity of the open building blocks in the image to be identified at the current moment according to the real-time building status;

[0098] The prompt submodule is used to prompt the user of the open-ended building blocks when the activity level is less than a preset activity level threshold, wherein the activity level threshold is determined according to the program to be executed.

[0099] Furthermore, the acquisition module 201 includes:

[0100] an acquisition submodule, configured to acquire multi-angle images of the target area at a current moment through a plurality of cameras arranged in the target area;

[0101] The selection submodule is used to select an image that meets preset conditions from the multi-angle images as the image to be identified in the target area at the current moment.

[0102] Furthermore, the selection submodule includes:

[0103] a calculation unit, configured to calculate the image area of ​​the component to be spliced ​​in the multi-angle image;

[0104] a selection unit, configured to select, from the multi-angle images, an image whose image area of ​​the component to be spliced ​​is greater than or equal to a preset area threshold as the image to be identified of the target area at the current moment, wherein the area threshold is determined according to the number of splicing parts of the component to be spliced.

[0105] It should be noted that the behavior detection device provided in the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform business analysis at the map level.

[0106] The behavior detection device provided in the embodiment of the present invention can implement each process implemented by the open-type building block splicing prompt method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0107] See also Figure 3 , Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 3 As shown, it includes: a memory 302, a processor 301, and a computer program of an open-type building block splicing prompt method stored in the memory 302 and executable on the processor 301, wherein:

[0108] The processor 301 is configured to call the computer program stored in the memory 302 and execute the following steps:

[0109] Obtaining an image to be recognized of the target area at a current moment, wherein the image to be recognized includes open building blocks;

[0110] Performing image recognition on the image to be recognized to obtain a real-time building state of the open-ended building blocks in the image to be recognized at the current moment;

[0111] According to the real-time building status, the user is prompted on the splicing of the open building blocks.

[0112] Furthermore, the processor 301 performs image recognition on the image to be recognized to obtain the real-time building state of the open-ended building blocks in the image to be recognized at the current moment, including:

[0113] Performing image recognition on the image to be recognized, and extracting the to-be-built state of the open-type building blocks in the image to be recognized;

[0114] Obtaining the built state of the open building blocks in the image to be recognized before the current moment;

[0115] According to the built state and the to-be-built state, a real-time built state of the open-ended building blocks in the image to be recognized at the current moment is determined.

[0116] Furthermore, the open-type building block executed by the processor 301 includes a building block assembly, the building block assembly includes a splicing portion and a connecting portion, for the same building block assembly, the splicing portions are connected via the connecting portion, and for different building block assemblies, they are connected via their respective splicing portions, and the performing of image recognition on the image to be recognized and extracting the to-be-built state of the open-type building block in the image to be recognized includes:

[0117] Performing feature extraction on the image to be identified to extract feature points of the splicing portion and feature lines of the connecting portion of the building block assembly to be spliced;

[0118] Connecting the joint feature points through the connection feature lines to obtain a node graph of the building block assembly to be spliced, wherein the nodes in the node graph of the building block assembly to be spliced ​​are the joint feature points, and the connections in the node graph are the connection feature lines;

[0119] Based on the node graph of the to-be-assembled building block assembly, the to-be-assembled state of the open-type building blocks in the to-be-recognized image is obtained.

[0120] Furthermore, the processor 301 performs feature extraction on the image to be identified to extract the joint feature points and connection feature lines of the building block assembly to be assembled, including:

[0121] Determining a to-be-recognized area of ​​the to-be-recognized image, wherein the to-be-recognized area includes building block components to be assembled;

[0122] Feature extraction is performed on the area to be identified, and feature points of the splicing portion and feature lines of the connecting portion of the building block assembly to be spliced ​​are extracted.

[0123] Furthermore, the processor 301 prompts the user on the splicing of the open-ended building blocks according to the real-time building status, including:

[0124] Calculating the activity of the open building blocks in the image to be recognized at the current moment according to the real-time building state;

[0125] When the activity level is less than a preset activity level threshold, a user who is assembling the open-ended building blocks is prompted, and the activity level threshold is determined according to a program to be executed.

[0126] Furthermore, the processor 301 executes the step of obtaining the image to be identified of the target area at the current moment, including:

[0127] At a current moment, a plurality of cameras arranged in the target area are used to obtain multi-angle images of the target area;

[0128] An image that meets preset conditions is selected from the multi-angle images as the image to be identified in the target area at the current moment.

[0129] Furthermore, the processor 301 selects an image that meets a preset condition from the multi-angle image as the image to be identified in the target area at the current moment, including:

[0130] In the multi-angle image, calculating the image area of ​​the component to be spliced;

[0131] From the multi-angle images, an image whose image area of ​​the component to be spliced ​​is greater than or equal to a preset area threshold is selected as the image to be identified of the target area at the current moment, wherein the area threshold is determined according to the number of splicing parts of the component to be spliced.

[0132] It should be noted that the electronic device provided by the embodiment of the present invention can be applied to devices such as smart phones, computers, servers, etc. that can perform behavior detection.

[0133] The electronic device provided in the embodiment of the present invention can implement each process implemented by the open building block splicing prompt method in the above method embodiment and can achieve the same beneficial effects. To avoid repetition, it will not be described here.

[0134] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements the various processes of the open-type building block splicing prompt method or the application-side open-type building block splicing prompt method provided in the embodiment of the present invention, and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0135] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0136] The above disclosure is merely a preferred embodiment of the present invention and certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. An open-type building block splicing prompt method, characterized in that: The open building block includes a building block assembly, and the building block assembly includes a splicing portion and a connecting portion. For the same building block assembly, the splicing portions are connected via the connecting portion, and for different building block assemblies, they are connected via their respective splicing portions. The method includes the following steps: Obtaining an image to be recognized of the target area at a current moment, wherein the image to be recognized includes open building blocks; Perform image recognition on the image to be identified to obtain the real-time construction state of the open-type building blocks in the image to be identified at the current moment; specifically perform feature extraction on the image to be identified to extract the splicing feature points and connection feature lines of the building block assembly to be spliced; connect the splicing feature points through the connection feature lines to obtain a node graph of the building block assembly to be spliced, wherein the nodes in the node graph of the building block assembly to be spliced ​​are the splicing feature points, and the connections in the node graph are the connection feature lines; based on the node graph of the building block assembly to be spliced, obtain the state to be constructed of the open-type building blocks in the image to be identified, and obtain the constructed state of the open-type building blocks in the image to be identified before the current moment; determine the real-time construction state of the open-type building blocks in the image to be identified at the current moment based on the constructed state and the state to be constructed; According to the real-time building status, the user is prompted on the splicing of the open building blocks.

2. The method according to claim 1, wherein The step of extracting features from the image to be identified to extract feature points of the joints and feature lines of the connection parts of the building block assembly to be assembled includes: Determining a to-be-recognized area of ​​the to-be-recognized image, wherein the to-be-recognized area includes building block components to be assembled; Feature extraction is performed on the area to be identified, and feature points of the splicing portion and feature lines of the connecting portion of the building block assembly to be spliced ​​are extracted.

3. The method according to claim 2, wherein Prompting the user on the splicing of the open building blocks according to the real-time building status includes: Calculating the activity of the open building blocks in the image to be recognized at the current moment according to the real-time building state; When the activity level is less than a preset activity level threshold, a user who is assembling the open-ended building blocks is prompted, and the activity level threshold is determined according to a program to be executed.

4. The method according to claim 3, wherein The step of obtaining the image to be identified of the target area at the current moment includes: At a current moment, a plurality of cameras arranged in the target area are used to obtain multi-angle images of the target area; An image that meets a preset condition is selected from the multi-angle images as the image to be identified in the target area at the current moment.

5. The method according to claim 4, wherein The selecting an image that meets a preset condition from the multi-angle images as the image to be identified in the target area at the current moment includes: In the multi-angle image, calculating the image area of ​​the component to be spliced; From the multi-angle images, an image whose image area of ​​the component to be spliced ​​is greater than or equal to a preset area threshold is selected as the image to be identified of the target area at the current moment, wherein the area threshold is determined according to the number of splicing parts of the component to be spliced.

6. A behavior detection device, characterized in that: The open building block includes a building block assembly, wherein the building block assembly includes a splicing portion and a connecting portion. For the same building block assembly, the splicing portions are connected by the connecting portion, and for different building block assemblies, the splicing portions are connected by their respective splicing portions. The device includes: An acquisition module, configured to acquire an image to be identified of a target area at a current moment, wherein the image to be identified includes open building blocks; The recognition module is configured to perform feature extraction on the image to be recognized, extract joint feature points and connection feature lines of the building block assembly to be assembled; connect the joint feature points through the connection feature lines to obtain a node graph of the building block assembly to be assembled, wherein the nodes in the node graph of the building block assembly to be assembled are the joint feature points, and the connections in the node graph are the connection feature lines; obtain the to-be-assembled state of the open-type building blocks in the image to be recognized based on the node graph of the building block assembly to be assembled; obtain the assembled state of the open-type building blocks in the image to be recognized before the current moment; and determine the real-time assembled state of the open-type building blocks in the image to be recognized at the current moment based on the assembled state and the to-be-assembled state. The prompt module is used to prompt the user of the splicing of the open building blocks according to the real-time building status.

7. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the open-type building block splicing prompt method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the open-type building block splicing prompt method according to any one of claims 1 to 5.

Images