Voice interaction method and platform for multi-terminal cooperation

Abstract

The application provides a voice interaction method and platform for multi-terminal cooperation, and relates to data processing technology, wherein the method comprises the following steps: in response to the movement of an inspection robot to any inspection space, controlling the inspection robot to perform image acquisition based on a target space task of the corresponding inspection space to obtain a space image of the corresponding inspection space; determining whether there is a construction personnel in the inspection space based on the space image, and determining an inspection mode of the inspection robot for the corresponding inspection space based on the determination result; when the inspection mode is determined as a scheduling mode, controlling the inspection robot to output a building material scheduling determination signal, and sending scheduling voice information obtained based on the building material scheduling determination signal to a space management terminal of the corresponding inspection space; when the inspection mode is determined as a verification mode, controlling the inspection robot to verify the inspection space based on the target space task, and generating verification voice information sent to the space management terminal of the corresponding inspection space. The application at least improves the cooperation efficiency.

Description

Technical Field

[0001] This invention relates to data processing technology, and more particularly to a voice interaction method and platform for multi-terminal collaboration. Background Technology

[0002] Given that in the existing construction site inspection and management process, inspection robots can often only perform single image acquisition or fixed patrol tasks, lacking the ability to dynamically interact with the actual construction environment (such as whether the workers are on duty).

[0003] When dealing with building material scheduling and construction operation verification, a significant amount of manual intervention is typically required for cross-terminal information transmission and confirmation, leading to a lag in information synchronization between on-site construction personnel, inspection robots, and the space management terminal. This lack of adaptive collaborative voice interaction among multiple terminals not only increases the cost of ineffective human-machine communication and reduces the response speed of building material scheduling, but also results in low efficiency of on-site status verification in unmanned operation scenarios, making it difficult to meet the demands of modern intelligent construction scenarios for high-frequency, automated, and highly efficient collaborative operations.

[0004] Therefore, there is an urgent need to provide a voice interaction method and platform for multi-terminal collaboration that can improve collaborative efficiency. Summary of the Invention

[0005] Based on the above problems, this invention is proposed to provide a voice interaction method and platform for multi-terminal collaboration that overcomes or at least partially solves the above problems.

[0006] According to one aspect of the present invention, a voice interaction method for multi-terminal collaboration is provided, comprising the following steps: The inspection robot moves along the predetermined inspection route of the corresponding construction site to any inspection space, and the inspection robot is controlled to perform image acquisition based on the target space task of the corresponding inspection space to obtain the spatial image of the corresponding inspection space. Based on spatial images, determine whether there are construction personnel in the inspection space, and based on the determination results, determine the inspection mode of the inspection robot for the corresponding inspection space. When the inspection mode is determined to be the scheduling mode, the inspection robot is controlled to output a building material scheduling determination signal and send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space. When the inspection mode is determined to be the verification mode, the inspection robot is controlled to verify the inspection space based on the target space task, and generates verification voice information to be sent to the space management terminal of the corresponding inspection space based on the verification results.

[0007] Optionally, in the method according to the present invention, controlling the inspection robot to perform image acquisition based on the target spatial task of the corresponding inspection space to obtain a spatial image of the corresponding inspection space includes: Determine the spatial structure surfaces that make up the inspection space, and retrieve the target space task of the corresponding inspection space based on the inspection sequence number of the corresponding inspection space obtained from the predetermined inspection route. When a construction work item is determined to exist on any spatial structural surface based on the target spatial task, the spatial structural surface is designated as the data acquisition structural surface, and the spatial center point of the corresponding inspection space is obtained. The inspection direction of the inspection robot is determined based on the predetermined inspection route and the construction direction of each acquisition structure surface corresponding to the spatial center point is determined based on the target spatial task. Based on the inspection orientation, the adjustment orientation of the inspection robot corresponding to each construction orientation is determined, and the inspection robot is controlled to rotate in sequence according to the different adjustment orientations in order to collect data on each structural surface and obtain a spatial image of the corresponding inspection space.

[0008] Optionally, in the method according to the present invention, acquiring a spatial image of the corresponding inspection space by acquiring each acquisition structure surface includes: In response to the inspection robot rotating to any adjustment orientation, the light unit pre-set on the inspection robot is controlled to emit an indicator light toward the acquisition structure surface corresponding to the adjustment orientation; Fill the construction work items corresponding to the structure surface to be collected into the type slots located in the preset group photo template to obtain the group photo voice information; The voice unit pre-set in the inspection robot is controlled to output the obtained group photo voice information, and the image unit pre-set in the inspection robot is controlled to perform acquisition based on a preset acquisition frequency and a continuous preset acquisition duration, so as to determine each acquired sub-image as a spatial image.

[0009] Optionally, in the method according to the present invention, determining whether construction personnel exist in the inspection space based on spatial images, and determining the inspection mode of the inspection robot corresponding to the inspection space based on the determination result, includes: Each acquired sub-image included in the spatial image is identified, and in response to the determination based on the identification result that any acquired sub-image contains a face region indicating any construction worker, the acquired sub-image is identified as an identifier sub-image; otherwise, it is identified as a construction sub-image. When it is determined that all the sub-images collected corresponding to the same spatial image are construction sub-images, it is determined that there are no construction personnel on the collected structural surface, and the inspection mode of the inspection robot corresponding to the collected structural surface is set to the verification mode; When at least one sub-image of the same spatial image is identified as the identifier sub-image, the construction worker image that is bound to the acquired structural surface is compared with the identifier sub-image that is determined based on the time sequence to obtain the first similarity. The first similarity is compared with the preset similarity, and based on the comparison result, it is determined whether there are construction personnel on the collected structural surface, so as to determine the inspection mode of the inspection robot corresponding to the inspection space.

[0010] Optionally, in the method according to the present invention, a first similarity is compared with a preset similarity, and based on the comparison result, it is determined whether there are construction personnel on the collected structural surface, so as to determine the inspection mode of the inspection robot corresponding to the inspection space, including: If the first similarity is greater than the preset similarity, the collection structure surface corresponding to the first similarity is determined as the identification structure surface, and it is determined that there are construction personnel on the identification structure surface, so that the inspection mode of the inspection robot corresponding to the collection structure surface is determined as the scheduling mode. If the first similarity is less than or equal to the preset similarity, the collected structural surface corresponding to the first similarity is identified as an abnormal structural surface, and it is determined that there are no construction personnel on the abnormal structural surface, so that the inspection mode of the inspection robot corresponding to the collected structural surface is determined as the verification mode. The face region corresponding to the marked structural surface is determined as the baseline region, and the face region corresponding to the abnormal structural surface is determined as the comparison region; The second similarity is obtained by comparing all comparison areas corresponding to the same inspection space with all reference areas; If any second similarity obtained from the response is greater than the preset similarity, the inspection mode of the identifier structure surface corresponding to the second similarity will be switched from the scheduling mode to the verification mode.

[0011] Optionally, in the method according to the present invention, controlling the inspection robot to output a building material scheduling determination signal and sending the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space includes: Based on the construction work items, determine the construction materials and construction scope of the corresponding structural surface, and identify any construction sub-image of the structural surface to obtain each construction material area indicating the construction materials and the construction scope area indicating the construction scope of the materials. If any building material area is located within the construction area, the building material area is determined to have construction attributes; otherwise, it is determined to have surplus attributes. Determine the area of ​​the corresponding construction zone, and sum the building material areas of each building material zone with construction attributes to obtain the total building material area; The difference between the area of ​​the designated area and the total area of ​​the building materials is calculated, and the resulting empty area is compared with the area of ​​the building materials. When it is determined that the available area is less than the building material area, the inspection mode of the corresponding structural surface to be collected will be switched from the scheduling mode to the verification mode. When the available area is determined to be greater than or equal to the building material area, the available sub-area corresponding to the available area is determined based on the construction scope area, and the inspection robot is controlled to output the building material scheduling determination signal obtained from the available sub-area and the building material area with surplus attributes, so as to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space.

[0012] Optionally, in the method according to the present invention, controlling the inspection robot to output a building material scheduling determination signal obtained from the vacant sub-region and the building material region with surplus attributes, so as to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space, including: Obtain the corresponding empty sub-region, the empty center point of any building material region, and the center point of the building material region, and overlap the empty sub-region and the building material region in such a way that the empty center point coincides with the center point of the building material region. When it is determined that the empty sub-region surrounds the building material region, obtain the integer multiple of the building material area corresponding to the empty area, and obtain the surplus value of all building material regions with the surplus attribute. When the integer multiple value is determined to be greater than the margin value, the inspection robot is controlled to output the obtained building material scheduling determination signal; The system receives voice feedback from construction workers, including pre-defined vocabulary, and fills the material difference corresponding to the surplus value of the integer multiple value into the pre-defined retrieval template. The obtained dispatch voice information is then sent to the space management terminal of the corresponding inspection space.

[0013] Optionally, in the method according to the present invention, the inspection robot is controlled to check the inspection space based on the target space task, and based on the check result, a check voice message is generated and sent to the space management terminal of the corresponding inspection space, including: Based on the construction work item, retrieve the corresponding construction verification categories of the collected structural surface, and control the inspection robot to identify the construction sub-images corresponding to the collected structural surface based on the construction verification categories to obtain the actual verification results; The actual verification results are compared and calculated with the preset standard results of the same construction verification category, and the obtained verification standard deviation is compared with the preset verification threshold. If the standard deviation of the response verification is less than or equal to the preset verification threshold, the corresponding data collection structure is determined as a compliant structure, and a verification result with the corresponding compliance status is generated. If the standard deviation of the response verification exceeds the preset verification threshold, the corresponding collected structural surface is identified as a structural surface to be rectified, and the construction verification category with the corresponding standard deviation is identified as a deviation verification item, so as to generate an abnormal verification result containing deviation verification items and verification standard deviation. Based on the construction orientation and verification results of the corresponding collected structural surface, verification voice information is generated and sent to the space management terminal of the corresponding inspection space.

[0014] Optionally, in the method according to the invention, the method further includes: The response will send the dispatch voice information or verification voice information to the space management terminal of the corresponding inspection space, determine the inspection space as the current space, and determine other inspection spaces connected to the current space and located after the current space as the arrival space based on the predetermined inspection route; Acquire the transitional ground image between the current space and the destination space, and identify the transitional ground image to obtain the constructed ground area and the remaining ground area; Retrieve the construction material type for the corresponding constructed ground area, and determine the accessibility of the constructed ground area based on the construction material type; When the type of construction material is determined to be cement, the passage attribute is set to impassable, and the corresponding constructed ground area is set as an interference avoidance area. When the type of construction material is determined to be brick, the accessibility attribute is set to passable, and the corresponding constructed ground area is set as a safe walking area. Based on the remaining ground area and the safe walking area, a cross-space movement path is planned from the current space to the destination space, and the inspection robot is controlled to move along the cross-space movement path to the destination space.

[0015] According to another aspect of the present invention, a voice interaction platform for multi-terminal collaboration is provided, comprising: The image acquisition module is configured to respond to the inspection robot moving along the predetermined inspection route of the corresponding construction site to any inspection space, and control the inspection robot to perform image acquisition based on the target space task of the corresponding inspection space to obtain the spatial image of the corresponding inspection space. The mode determination module is configured to determine whether construction personnel exist in the inspection space based on the spatial image, and to determine the inspection mode of the inspection robot corresponding to the inspection space based on the determination result. The scheduling execution module is configured to control the inspection robot to output a building material scheduling determination signal when the inspection mode is determined to be the scheduling mode, and to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space. The verification execution module is configured to, when the inspection mode is determined to be verification mode, control the inspection robot to verify the inspection space based on the target space task, and generate verification voice information to be sent to the space management terminal of the corresponding inspection space based on the verification results.

[0016] According to the solution of the present invention, the voice interaction method and platform for multi-terminal collaboration provided by the present invention have at least the following beneficial effects: First, this invention controls an inspection robot to acquire images of the inspection space and intelligently determines the presence of construction personnel based on the spatial images, thereby adaptively determining the inspection mode as either a scheduling mode or a verification mode. This allows the inspection operation to dynamically adapt to the actual on-site working conditions, facilitating rapid collaboration in manned scenarios and performing independent verification in unmanned scenarios, significantly reducing meaningless human-machine interaction processes and improving overall execution efficiency. Secondly, this invention obtains the spatial center point of the inspection space and, combined with the predetermined inspection route and target space task, accurately calculates the inspection robot's inspection orientation, construction orientation, and various adjustment orientations. This acquisition method, which matches the viewing angle to the spatial structure surface, effectively reduces image distortion and information loss caused by tilted shooting; coupled with the indicator light from the light unit and the prompts from the voice unit, it improves the standardization of the acquisition process and the accuracy of subsequent image recognition. Next, when identifying on-site personnel, this invention not only extracts facial regions for temporal comparison but also performs a first similarity comparison between the collected images and the images of the assigned construction personnel. Simultaneously, a second similarity cross-verification is conducted between the baseline area and the comparison area throughout the entire inspection space. This mechanism effectively filters out misjudgments caused by unauthorized personnel entering the site, moving across work areas, or changing positions, ensuring that scheduling permissions are only granted to genuine on-duty personnel and improving the rigor of mode switching. Then, in the scheduling mode, the present invention accurately calculates the area difference by identifying the building material area and the construction area to evaluate the scheduling space. Furthermore, by aligning the center point of the vacant area with the center point of the building material area, it verifies whether the vacant sub-area can completely surround the building material area, eliminating invalid scheduling scenarios where the area meets the standard but the shape does not match. Combined with the voice feedback from the on-site construction personnel, the final scheduling voice information is generated, which not only leverages the efficiency advantage of automatic spatial calculation but also takes into account the actual feasibility of on-site scheduling, reducing on-site congestion and secondary handling. Then, in the verification mode, this invention calls the corresponding recognition algorithm according to the construction verification category, calculates the quantitative difference between the actual verification result and the preset standard result, and compares it with the preset verification threshold. This mechanism not only reduces the subjective judgment bias of manual acceptance, but also automatically generates abnormal verification results containing deviation verification items and verification standard deviations; by converting the verification results into verification voice information and sending it to the space management terminal, managers can quickly locate problem points and rectification directions without having to look through graphic reports. Finally, when the inspection robot moves across the inspection space, this invention can automatically acquire images of the transitional ground and dynamically determine the passage attributes based on the type of construction materials (such as cement or brick) extracted from the constructed ground areas. By designating impassable uncured areas as interference avoidance zones and load-bearing areas as safe walking zones to plan cross-zone movement paths, this effectively prevents the robot from blindly moving and damaging newly poured ground and other finished construction products, and reduces the interference of inspection operations on normal on-site processes. Attached Figure Description

[0017] Figure 1 A flowchart of a voice interaction method for multi-terminal collaboration according to an embodiment of the present invention is shown; Figure 2 A structural block diagram of a voice interaction platform for multi-terminal collaboration according to another embodiment of the present invention is shown. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0019] To address the problems existing in the prior art, the inventors proposed the solution of this invention. One embodiment of this invention provides a voice interaction method for multi-terminal collaboration, which can be executed in a computing device, wherein the computing device can be understood as a terminal with data processing capabilities, such as a computer.

[0020] Figure 1 A flowchart of a voice interaction method for multi-terminal collaboration according to an embodiment of the present invention is shown, such as... Figure 1 As shown, the method begins with step S1, which includes the following: The inspection robot moves along the predetermined inspection route of the corresponding construction site to any inspection space, and the robot is controlled to perform image acquisition based on the target space task of the corresponding inspection space to obtain the spatial image of the corresponding inspection space.

[0021] For example, in this embodiment, the server can receive the location arrival signal sent by the inspection robot through the wireless communication network, confirm that the inspection robot has moved to any inspection space along the predetermined inspection route of the corresponding construction site, retrieve the target space task corresponding to the inspection space from the internal task database, and control the image acquisition unit on the inspection robot to perform image acquisition operation according to the task requirements, and finally obtain the spatial image of the corresponding inspection space.

[0022] Here, it can be explained that the predetermined inspection route is a travel path pre-planned based on the spatial layout of the construction site, the division of construction areas, and the work progress. The path sequentially connects all the inspection spaces to be inspected, ensuring comprehensive coverage and orderly progress of the inspection work. The inspection space is an independent work area on the construction site divided according to spatial structure or construction procedures. Different inspection spaces may correspond to different construction contents and work progress. The target space task is the inspection operation instruction pre-configured and bound to each inspection space, which may include different types such as construction quality verification, building material inventory statistics, and personnel attendance verification. The specific content can be dynamically adjusted according to the construction stage of the corresponding inspection space. The spatial image is a set of image data covering the corresponding inspection space, which is used to provide a data foundation for subsequent personnel identification and construction status judgment.

[0023] Furthermore, in this embodiment, the aforementioned "controlling the inspection robot to perform image acquisition based on the target space task of the corresponding inspection space to obtain a spatial image of the corresponding inspection space" may further include the following steps: Determine the spatial structure surfaces that make up the inspection space, and retrieve the target space task of the corresponding inspection space based on the inspection sequence number of the corresponding inspection space obtained from the predetermined inspection route. When a construction work item is determined to exist on any spatial structural surface based on the target spatial task, the spatial structural surface is designated as the data acquisition structural surface, and the spatial center point of the corresponding inspection space is obtained. The inspection direction of the inspection robot is determined based on the predetermined inspection route and the construction direction of each acquisition structure surface corresponding to the spatial center point is determined based on the target spatial task. Based on the inspection orientation, the adjustment orientation of the inspection robot corresponding to each construction orientation is determined, and the inspection robot is controlled to rotate in sequence according to the different adjustment orientations in order to collect data on each structural surface and obtain a spatial image of the corresponding inspection space.

[0024] For example, in this embodiment, after the inspection robot arrives at the target inspection space, the server can identify the boundary and enclosure structure of the current inspection space through the spatial sensing unit on the inspection robot itself, and determine all the spatial structural surfaces that make up the inspection space. It can be noted that the spatial structural surfaces may include different forms of structural surfaces such as the working walls, ground, ceiling and temporary working facades at the construction site. Here, each inspection space in the predetermined inspection route is configured with a unique inspection sequence number. The inspection robot can match the corresponding inspection sequence number according to its own position and retrieve the target space task bound to the sequence number from the task database. The target space task records the construction arrangements and work progress corresponding to each spatial structural surface. Subsequently, the server can match the construction work items in the target space task with each spatial structure surface one by one. If it is determined that a certain spatial structure surface should have a construction work item that is being carried out or to be verified, then the structure surface is marked as the acquisition structure surface and used as the key object for subsequent image acquisition. At the same time, the spatial center point of the current inspection space can be calculated through spatial modeling. It can be noted that the spatial center point can be used as the reference station point for the inspection robot to perform acquisition actions. Based on this, the server determines the initial inspection orientation of the inspection robot when it is at the center point of the space, by combining the travel direction of the predetermined inspection route. At the same time, based on the spatial orientation of each acquisition structure surface in the target space task, the construction orientation of the center point of the space towards each acquisition structure surface is calculated. It can be noted that the construction orientation is the direction directly opposite when acquiring the corresponding spatial structure surface.

[0025] Finally, the server uses the initial inspection orientation as a reference to calculate the angle of rotation required to turn to each construction orientation and the corresponding adjustment orientation. Following the preset acquisition sequence, the inspection robot controls its body to rotate to different adjustment orientations in turn, and completes image acquisition for each acquisition structure surface one by one. Finally, all the acquired image data are integrated into a spatial image of the corresponding inspection space.

[0026] It can be explained that, in this embodiment, by using the spatial center point as a reference and matching the acquisition orientation according to the structural surface, the acquisition viewpoint can be directly facing each working structural surface, reducing image distortion and information omission caused by tilted shooting, and improving the acquisition accuracy of spatial images and the accuracy of subsequent recognition results.

[0027] Furthermore, in this embodiment, the aforementioned "acquiring data from each acquisition structure surface to obtain a spatial image of the corresponding inspection space" may further include the following steps: In response to the inspection robot rotating to any adjustment orientation, the light unit pre-set on the inspection robot is controlled to emit an indicator light toward the acquisition structure surface corresponding to the adjustment orientation; Fill the construction work items corresponding to the structure surface to be collected into the type slots located in the preset group photo template to obtain the group photo voice information; The voice unit pre-set in the inspection robot is controlled to output the obtained group photo voice information, and the image unit pre-set in the inspection robot is controlled to perform acquisition based on a preset acquisition frequency and a continuous preset acquisition duration, so as to determine each acquired sub-image as a spatial image.

[0028] For example, in this embodiment, after the inspection robot rotates to the adjustment orientation corresponding to a certain acquisition structure surface and stabilizes its body posture, it can trigger the activation of its onboard light unit to project an indicator light towards the acquisition structure surface. It can be explained that the indicator light can form a calibration spot on the surface of the structure surface that matches the field of view of the image acquisition, which is used to clarify the coverage area of ​​the current image acquisition and can also serve as a prompt to the on-site personnel, letting them know that the current work surface is in the acquisition state. After that, the server can extract the construction work items corresponding to the collection structure from the target space task, and at the same time fill the work item content into the category slots reserved in the preset group photo template to generate the corresponding group photo voice information. Here, the preset group photo template can be a pre-configured voice broadcast template, while the category slots are used to carry the construction content information of the corresponding work surface. The rest of the template can contain fixed voice content such as collection prompts and work specification prompts. Subsequently, the inspection robot plays the generated group photo audio information through its onboard voice unit, synchronously sending the corresponding construction work item and collection prompts to the site. At the same time, it simultaneously starts the image unit to perform continuous image acquisition according to the pre-configured acquisition frequency. The acquisition process is continuously preset for the acquisition duration, and the multiple frames obtained within the acquisition period are used as each acquisition sub-image to jointly form the spatial image of the corresponding acquisition structure surface.

[0029] It can be noted that, in this embodiment, the preset acquisition frequency and preset acquisition duration can be configured according to the area of ​​the acquisition structure and the complexity of the work content. This embodiment does not limit this. The multiple frames of sub-images obtained by continuous acquisition can cover different instantaneous states of the screen, reduce the probability of occlusion or blurring in a single frame image, and provide more sufficient image data support for subsequent personnel identification and construction verification.

[0030] Step S2 includes the following: Based on spatial images, it is determined whether construction personnel are present in the inspection space, and based on the determination results, the inspection mode of the inspection robot for the corresponding inspection space is determined.

[0031] For example, in this embodiment, after the server acquires the spatial image uploaded by the inspection robot, it can perform feature extraction and target detection on the spatial image through a preset image recognition algorithm to identify whether the human body features or facial features of construction workers appear within the image range. Based on the recognition results, the server matches the inspection mode of the inspection robot to the inspection space. Here, the inspection mode includes two types: scheduling mode and verification mode. It can be noted that the scheduling mode is suitable for scenarios where there are corresponding workers in the inspection space. The inspection robot can interact and collaborate with the on-site construction workers to complete collaborative tasks such as material scheduling confirmation. The verification mode is suitable for scenarios where there are no corresponding workers in the inspection space. The inspection robot can independently perform inspection tasks such as construction quality verification and on-site status recording.

[0032] It can be noted that, in this embodiment, by switching the corresponding inspection mode according to the on-site personnel's presence status, the inspection operation can be adapted to different on-site operation scenarios, reducing meaningless human-computer interaction processes and improving the overall execution efficiency of the inspection operation.

[0033] Furthermore, in this embodiment, the aforementioned "determining whether construction personnel exist in the inspection space based on spatial images, and determining the inspection mode of the inspection robot corresponding to the inspection space based on the determination result" may also include the following steps: Each acquired sub-image included in the spatial image is identified, and in response to the determination based on the identification result that any acquired sub-image contains a face region indicating any construction worker, the acquired sub-image is identified as an identifier sub-image; otherwise, it is identified as a construction sub-image. When it is determined that all the sub-images collected corresponding to the same spatial image are construction sub-images, it is determined that there are no construction personnel on the collected structural surface, and the inspection mode of the inspection robot corresponding to the collected structural surface is set to the verification mode; When at least one sub-image of the same spatial image is identified as the identifier sub-image, the construction worker image that is bound to the acquired structural surface is compared with the identifier sub-image that is determined based on the time sequence to obtain the first similarity. The first similarity is compared with the preset similarity, and based on the comparison result, it is determined whether there are construction personnel on the collected structural surface, so as to determine the inspection mode of the inspection robot corresponding to the inspection space.

[0034] For example, in this embodiment, after the server obtains all the sub-images of the corresponding acquisition structure surface, it calls the preset face detection algorithm to perform feature recognition on each frame of the acquisition sub-image and extract the region in the image that matches the face features. In one scenario, if the recognition result shows that a face region corresponding to a construction worker exists in a certain sub-image, it means that a person is present in the work area corresponding to the structure surface within the field of view of the current acquisition frame. This person may be an on-duty construction worker assigned to the work surface, or it may be another on-site worker who is temporarily passing by. It is not possible to directly confirm their job matching relationship with the construction work item. In this case, the server marks the acquisition sub-image as an identification sub-image for subsequent matching and verification of personnel identity. In another scenario, if no valid face area is detected, it means that no one is present in the field of view corresponding to the current acquisition frame, and the working surface of the acquisition structure is in a complete presentation state without any personnel obstruction. In this case, the server marks the acquired sub-image as a construction sub-image. It can be noted that the construction sub-image can fully reflect the actual on-site situation such as the arrangement of building materials and the construction effect of the working surface. There is no loss of working surface information caused by personnel body obstruction. It can be directly used for subsequent construction status verification and building material identification, without the need for additional image repair and completion processing of obstructed areas. Furthermore, if, after frame-by-frame determination, all sub-images corresponding to the same acquisition structure are construction sub-images, then it is determined that there are no construction personnel within the working range of the acquisition structure. In this case, the inspection mode of the inspection robot corresponding to the acquisition structure is set to the verification mode, and the robot independently carries out the subsequent construction quality verification work.

[0035] If at least one frame of the sub-images corresponding to the same acquisition structure surface exists, then the construction worker image that is bound to the construction work item of the acquisition structure surface is retrieved from the pre-configured task database. It can be explained that the construction worker image is a pre-recorded face image of the on-duty worker assigned to the construction work item. By sorting all the sub-images according to the acquisition time sequence, the last frame of the sub-image in the time sequence is selected, and the face sub-region is extracted from it. The face sub-region is then compared with the face feature of the bound construction worker reference image to calculate the first similarity that represents the degree of matching between the two.

[0036] Based on this, the server can compare the calculated first similarity with the preset similarity, and determine whether there are construction personnel in the corresponding positions on the collected structure surface according to the comparison results, and finally determine the inspection mode of the inspection robot for the collected structure surface.

[0037] It can be noted that, in this embodiment, by combining multi-frame image screening with personnel identity verification, the probability of misjudgment caused by irrelevant personnel accidentally entering the collection screen can be reduced, the accuracy of personnel on duty determination can be improved, and the switching of inspection mode can be more in line with the actual on-site operation status.

[0038] Furthermore, in this embodiment, the aforementioned "comparing the first similarity with the preset similarity and determining whether there are construction personnel on the collected structural surface based on the comparison result, so as to determine the inspection mode of the inspection robot corresponding to the inspection space" may also include the following steps: If the first similarity is greater than the preset similarity, the collection structure surface corresponding to the first similarity is determined as the identification structure surface, and it is determined that there are construction personnel on the identification structure surface, so that the inspection mode of the inspection robot corresponding to the collection structure surface is determined as the scheduling mode. If the first similarity is less than or equal to the preset similarity, the collected structural surface corresponding to the first similarity is identified as an abnormal structural surface, and it is determined that there are no construction personnel on the abnormal structural surface, so that the inspection mode of the inspection robot corresponding to the collected structural surface is determined as the verification mode. The face region corresponding to the marked structural surface is determined as the baseline region, and the face region corresponding to the abnormal structural surface is determined as the comparison region; The second similarity is obtained by comparing all comparison areas corresponding to the same inspection space with all reference areas; If any second similarity obtained from the response is greater than the preset similarity, the inspection mode of the identifier structure surface corresponding to the second similarity will be switched from the scheduling mode to the verification mode.

[0039] For example, in this embodiment, the server can numerically compare the calculated first similarity with the preset similarity, and classify and label the collected structural surfaces according to the comparison results; In one scenario, when the first similarity is greater than the preset similarity, it indicates that the currently collected face matches the identity of the construction worker bound to the structure. In this case, the server can determine that there are corresponding on-duty workers on the structure and mark the collected structure as an identification structure. At the same time, the inspection robot's inspection mode for the corresponding structure is configured as a scheduling mode to support subsequent human-computer interaction operations such as building material scheduling. In another scenario, when the first similarity is less than or equal to the preset similarity, it indicates that the match between the currently collected face and the construction worker's identity bound to the structure is insufficient. This may be due to irrelevant personnel accidentally entering the collection range, misidentification caused by personnel's faces being obscured, or temporary stay by personnel not in this position. In this case, it is determined that there is no corresponding on-duty worker for the structure, and the collected structure is marked as an abnormal structure. At the same time, the inspection mode of the inspection robot corresponding to the collected structure is configured to the verification mode, and the verification process is executed according to the unmanned operation scenario. Furthermore, after completing the personnel identification of the collected structural surfaces, the server can extract the face regions corresponding to all identified structural surfaces as reference regions, and simultaneously extract the face regions corresponding to all abnormal structural surfaces as comparison regions. Cross-face feature comparison is carried out for all reference regions and comparison regions within the same inspection space. The feature matching degree between each set of comparison regions and reference regions is calculated one by one to obtain the corresponding second similarity. It can be noted that, here, this cross-comparison process is used to verify the personnel correlation between different structural surfaces within the same inspection space and to investigate misjudgment of single structural surfaces caused by personnel leaving their posts or moving around temporarily. If the second similarity obtained from any set of comparisons is greater than the preset similarity, it means that the personnel appearing on the corresponding abnormal structural surface are the same person as the on-duty personnel identified on a certain identification structural surface. That is, the personnel are not the exclusive on-duty personnel of the identification structural surface and have moved across work surfaces. This does not meet the personnel matching requirements for the identification structural surface to initiate scheduling interaction. At this time, the server will switch the inspection mode of the identification structural surface with the second similarity from scheduling mode to verification mode and cancel the scheduling interaction permission of the collection structural surface.

[0040] It can be noted that, in this embodiment, by combining identity verification with cross-validation, the misjudgment of patterns caused by the presence of personnel who are not in the relevant positions can be further filtered, reducing the probability of erroneous initiation of scheduling interactions, and making the inspection mode more suitable for the actual state of personnel flow at the construction site. In this embodiment, all processes involving the collection and recognition of facial data will be communicated to the staff in advance to meet the requirements for sensitive data collection.

[0041] Step S3 includes the following: When the inspection mode is determined to be the scheduling mode, the inspection robot is controlled to output a building material scheduling confirmation signal, and the scheduling voice information obtained based on the building material scheduling confirmation signal is sent to the space management terminal of the corresponding inspection space.

[0042] For example, in this embodiment, when the server determines that the inspection mode corresponding to the current inspection space is the scheduling mode, it can issue an interactive execution command to the inspection robot, controlling the inspection robot to output a building material scheduling confirmation signal through its own voice playback unit. It can be explained that the building material scheduling confirmation signal can be in the form of voice broadcast, synchronizing the building material inventory calculation results and scheduling space information of the current work surface to the on-site construction personnel. After obtaining the confirmation feedback from the construction personnel, the server can further generate standardized scheduling voice information based on the on-site confirmed scheduling content, and transmit the scheduling voice information to the space management terminal of the corresponding current inspection space.

[0043] Here, the space management terminal can be understood as a terminal device deployed at each inspection space operation site for use by on-site management personnel. It can be used to receive various scheduling and verification information generated during the inspection process, and realize the synchronous retention and follow-up processing of on-site operation data. The scheduling voice information includes the type of building materials, the quantity to be scheduled, the expected placement location, etc. of the corresponding work surface. It can be played on the space management terminal in the form of voice broadcast, or it can be converted into text information for storage.

[0044] It can be noted that, in this embodiment, by using the inspection robot to output the scheduling confirmation signal and the space management terminal to simultaneously receive the scheduling voice information, the on-site building material scheduling needs can be quickly confirmed and uploaded synchronously, reducing the intermediate links of manual reporting of scheduling information and improving the response speed and information transmission accuracy of building material allocation at the construction site.

[0045] Furthermore, in this embodiment, the aforementioned "controlling the inspection robot to output a building material scheduling determination signal and sending the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space" may also include the following steps: Based on the construction work items, determine the construction materials and construction scope of the corresponding structural surface, and identify any construction sub-image of the structural surface to obtain each construction material area indicating the construction materials and the construction scope area indicating the construction scope of the materials. If any building material area is located within the construction area, the building material area is determined to have construction attributes; otherwise, it is determined to have surplus attributes. Determine the area of ​​the corresponding construction zone, and sum the building material areas of each building material zone with construction attributes to obtain the total building material area; The difference between the area of ​​the designated area and the total area of ​​the building materials is calculated, and the resulting empty area is compared with the area of ​​the building materials. When it is determined that the available area is less than the building material area, the inspection mode of the corresponding structural surface to be collected will be switched from the scheduling mode to the verification mode. When the available area is determined to be greater than or equal to the building material area, the available sub-area corresponding to the available area is determined based on the construction scope area, and the inspection robot is controlled to output the building material scheduling determination signal obtained from the available sub-area and the building material area with surplus attributes, so as to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space.

[0046] For example, in this embodiment, the server can combine the construction work items corresponding to the collected structural surface to extract the type of construction materials to be used on the work surface, as well as the standard construction range boundary information corresponding to the type of construction materials. Then, it can select any frame of construction sub-image corresponding to the collected structural surface for image semantic segmentation and recognition, and extract each piece of construction material area corresponding to the construction materials in the construction sub-image, as well as the construction range area corresponding to the preset construction range. It can be noted that the construction range area is the image area corresponding to the construction operation defined according to the construction process requirements, and the construction material area is the image area where the construction materials are stacked. For each identified building material area, the server can determine whether its position in the construction sub-image is inside the boundary of the construction area. In one scenario, if a building material area is entirely located within the construction area, the server will mark that building material area as having construction attributes, indicating that the building materials have been arranged in the work area and are construction materials to be used. In another case, if the building materials area is outside the boundary of the construction area, it is marked as having a surplus attribute, which means that the building materials are spare building materials stored outside the work area and can participate in subsequent scheduling and arrangement. Based on this, the server can further calculate the actual area of ​​the construction area by converting the pixel ratio to the actual size. At the same time, it can calculate the actual area of ​​each building material in all building material areas with construction attributes, and then sum up the areas of each building material to obtain the total area of ​​the building materials that have been arranged within the construction area. The server calculates the difference between the area of ​​the construction zone and the total area of ​​the building materials to obtain the unused area within the construction zone that is not occupied by building materials. Then, it compares the unused area with the area of ​​a single building material to determine whether there is enough space within the construction zone to accommodate new building materials. In one case, if the calculated free area is less than the area of ​​a single building material, it means that the available space within the current construction area is insufficient to accommodate an additional building material, and there is no need to carry out building material scheduling. In this case, the server switches the inspection mode corresponding to the collected structural surface from the scheduling mode to the verification mode, and then performs the construction quality verification of the structural surface according to the verification process. In another scenario, if the available area is greater than or equal to the area of ​​a single building material, it indicates that there is room for additional building materials within the construction area. In this case, the server locates the available sub-area corresponding to the available area based on the spatial distribution of the construction area. It then generates a building material scheduling confirmation signal by combining the location and quantity information of all building material areas with surplus attributes. The server controls the inspection robot to output this signal to the site. After the on-site construction personnel confirm the scheduling plan, the server generates standardized scheduling voice information based on the location of the available sub-area, the quantity and type of schedulable building materials, and sends it to the space management terminal of the corresponding inspection space.

[0047] It can be explained that, in this embodiment, by identifying the construction area boundary and calculating and matching the building material layout area, the building material scheduling needs of the work surface can be assessed, invalid scheduling scenarios with no storage space can be filtered out, and the placement location and quantity of the scheduling building materials can be clearly defined, so that the building material scheduling instructions are more in line with the actual space conditions of the on-site operation, improving the feasibility and accuracy of scheduling execution, and reducing secondary handling and on-site congestion caused by space calculation deviations.

[0048] Furthermore, in this embodiment, the aforementioned "controlling the inspection robot to output the building material scheduling determination signal obtained from the vacant sub-region and the building material region with surplus attributes, so as to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space" may also include the following steps: Obtain the corresponding empty sub-region, the empty center point of any building material region, and the center point of the building material region, and overlap the empty sub-region and the building material region in such a way that the empty center point coincides with the center point of the building material region. When it is determined that the empty sub-region surrounds the building material region, obtain the integer multiple of the building material area corresponding to the empty area, and obtain the surplus value of all building material regions with the surplus attribute. When the integer multiple value is determined to be greater than the margin value, the inspection robot is controlled to output the obtained building material scheduling determination signal; The system receives voice feedback from construction workers, including pre-defined vocabulary, and fills the material difference corresponding to the surplus value of the integer multiple value into the pre-defined retrieval template. The obtained dispatch voice information is then sent to the space management terminal of the corresponding inspection space.

[0049] For example, in this embodiment, before generating the building material scheduling determination signal, the server can first verify the spatial compatibility between the vacant sub-region and a single building material: In other words, by calculating the image coordinates, the center point of the empty sub-region and the center point of any building material region with surplus attributes are obtained. Furthermore, by stacking the two center points in a way that they overlap, the outline of the building material region is mapped to the empty sub-region. It is then determined whether the boundary of the empty sub-region can completely surround the entire outline of the building material region. It can be noted that the empty sub-region belongs to the unoccupied part of the construction area and also follows the work boundary requirements defined by the construction process. The building material region corresponds to the outline of a single piece of building material image that is stored on site. The degree of matching between the two outlines determines the on-site executability of the scheduling plan. In addition, the center point overlap verification method can simulate the normal operation state of placing building materials in the center, eliminating the situation where only the area value meets the standard, but the shape of the region does not match the shape of the building material, thus making it impossible to place them in practice, thereby improving the practicality of space measurement. Furthermore, when the verification confirms that the empty sub-region can completely surround the single building material area, the server can calculate the ratio of the empty area to the single building material area, and take the integer part as the integer multiple value. This value represents the maximum number of building materials of the same specification that the current empty sub-region can accommodate. At the same time, the server counts the total number of all building material areas with the surplus attribute corresponding to the current collected structure face to obtain the surplus value. In this case, the surplus value represents the total number of spare building materials that can be dispatched to the construction area on site. When the integer multiple value is greater than the remaining value, it means that the available space within the current construction area is sufficient to accommodate all the spare building materials and has the space conditions for complete scheduling. At this time, the server can issue instructions to the inspection robot to control the inspection robot to output a building material scheduling confirmation signal, and synchronize information such as the quantity of schedulable building materials and the suggested placement location to the on-site construction personnel, and wait for feedback and confirmation from the on-site personnel. When the inspection robot's voice unit receives voice feedback from construction personnel containing preset vocabulary, it can determine that the on-site personnel approve of the scheduling plan. At this time, the server can calculate the difference between the building material value and the remaining value, fill the building material difference, the type of building material to be scheduled, the placement location, and other information into the corresponding slot of the preset retrieval template, generate standardized scheduling voice information, and finally send the scheduling voice information to the space management terminal of the corresponding inspection space for storage and synchronization. Here, the preset vocabulary can be pre-configured according to on-site work habits, and the preset retrieval template is a pre-configured standardized template for scheduling information, which includes fixed fields such as building material type, scheduling quantity, work location, and work time. The generated scheduling voice information can be directly broadcast on the space management terminal, making it convenient for on-site managers to synchronize scheduling progress.

[0050] It can be noted that in this embodiment, by combining spatial contour encirclement verification with quantity calculation, the spatial feasibility and quantity rationality of the scheduling scheme can be taken into account. At the same time, the final effect of the scheduling command is completed by combining the voice confirmation of the on-site personnel. This not only gives full play to the efficiency advantage of the robot's automatic calculation, but also preserves the decision-making authority of the on-site personnel, so that the building material scheduling process takes into account both efficiency and accuracy, and adapts to the flexible operation needs of the construction site.

[0051] Step S4 includes the following: When the inspection mode is determined to be the verification mode, the inspection robot is controlled to verify the inspection space based on the target space task, and generates verification voice information to be sent to the space management terminal of the corresponding inspection space based on the verification results.

[0052] For example, in this embodiment, when the server determines that the inspection mode corresponding to the current inspection space is the verification mode, it can issue a verification execution command to the inspection robot, thereby retrieving the corresponding verification dimension and judgment standard in the target space task bound to the current inspection space, and controlling the image acquisition unit and sensing unit on the inspection robot to carry out data acquisition and status verification of the working structure surface and construction point in the inspection space one by one. After comparing and calculating the actual on-site data obtained with the preset standard, the corresponding verification result is generated, and then standardized verification voice information is generated based on the verification result. Finally, the verification voice information is transmitted to the space management terminal corresponding to the current inspection space.

[0053] Here, the verification mode can cover multiple dimensions such as the completion of construction, compliance of building material placement, availability of safety protection facilities, and progress of work processes. The specific verification items are dynamically adjusted according to the configuration of the target space tasks and the construction stage. The verification voice information includes the verification points of the corresponding inspection space, details of compliance items, deviations to be rectified and the degree of deviation, etc. It can be directly broadcast on the space management terminal or converted into text for archiving.

[0054] It can be noted that in this embodiment, the inspection robot autonomously performs the entire process of verification and generates voice-based verification results. This allows for the complete verification of the inspection space without the assistance of on-site personnel. At the same time, the results are synchronized to the space management terminal in the form of voice information, making it convenient for on-site management personnel to quickly obtain the verification conclusions without having to consult each graphic report. This improves the efficiency of inspection and verification and information transmission in unmanned operation scenarios.

[0055] Furthermore, in this embodiment, the aforementioned "controlling the inspection robot to check the inspection space based on the target space task, and generating verification voice information to be sent to the space management terminal of the corresponding inspection space based on the verification results" may also include the following steps: Based on the construction work item, retrieve the corresponding construction verification categories of the collected structural surface, and control the inspection robot to identify the construction sub-images corresponding to the collected structural surface based on the construction verification categories to obtain the actual verification results; The actual verification results are compared and calculated with the preset standard results of the same construction verification category, and the obtained verification standard deviation is compared with the preset verification threshold. If the standard deviation of the response verification is less than or equal to the preset verification threshold, the corresponding data collection structure is determined as a compliant structure, and a verification result with the corresponding compliance status is generated. If the standard deviation of the response verification exceeds the preset verification threshold, the corresponding collected structural surface is identified as a structural surface to be rectified, and the construction verification category with the corresponding standard deviation is identified as a deviation verification item, so as to generate an abnormal verification result containing deviation verification items and verification standard deviation. Based on the construction orientation and verification results of the corresponding collected structural surface, verification voice information is generated and sent to the space management terminal of the corresponding inspection space.

[0056] For example, in this embodiment, after entering the verification mode, the server can retrieve the matching construction verification categories from the preset verification standard library according to the construction work item corresponding to the current collected structural surface. It can be explained that the construction verification category is a quality verification dimension set for different construction procedures, which is bound to the work content of the construction work item. The preset standard result is the qualified value of each construction verification category. Both are pre-configured according to the on-site construction acceptance standards. For example, for wall plastering, verification categories such as flatness, verticality, and squareness of internal and external corners can be configured. For tile laying, verification categories such as gap width, surface flatness, and hollow area ratio can be configured. That is, different verification categories correspond to different image recognition algorithms and judgment standards. After retrieval, the server can issue a verification command to the inspection robot, call the image recognition algorithm corresponding to each construction verification category, perform feature extraction and parameter calculation on the construction sub-image corresponding to the collected structural surface, and obtain the actual verification results corresponding to each verification dimension. The actual verification results are presented in the form of quantitative values, corresponding to the on-site measured data of each construction verification category. Subsequently, the server can extract the preset standard results corresponding to each construction inspection category, calculate the difference between the actual inspection results under the same inspection category and the preset standard results, obtain the inspection standard deviation for each inspection category, and then compare each inspection standard deviation with the preset inspection threshold. It can be noted that the preset inspection threshold is the upper limit of allowable deviation set according to the construction acceptance specification, which is used to determine whether the construction quality meets the qualification requirements. In one case, when the standard deviation of a certain construction inspection category is less than or equal to the preset inspection threshold, it indicates that the construction quality of that dimension meets the specifications. The server will mark the corresponding collected structural surface as a compliant structural surface, generate the corresponding compliance status inspection results, and record the qualification status of each inspection item of the structural surface. In another scenario, when the standard deviation of a certain construction inspection category is greater than the preset inspection threshold, it indicates that the construction quality of that dimension exceeds the allowable deviation range. The server can mark the corresponding collected structural surface as a structural surface to be rectified, and mark the construction inspection category that exceeds the threshold as a deviation inspection item, generating an abnormal inspection result that includes the deviation inspection item name and the corresponding standard deviation, clearly marking the problem item and the degree of deviation. Furthermore, after completing the determination of all verification items, the server can combine the construction orientation corresponding to the collected structural surface with the generated compliance or abnormal verification results to assemble and generate standardized verification voice information. It can be noted that the verification voice information can be organized according to a unified broadcasting logic, which can cover core information such as structural surface orientation, overall verification conclusion, deviation details and corresponding deviation values. This allows the server to send the verification voice information to the spatial management terminal of the corresponding inspection space, so that on-site management personnel can quickly grasp the verification results without having to consult graphic reports.

[0057] It can be noted that, in this embodiment, by matching the corresponding verification dimensions according to the construction work items and conducting quality judgment based on quantitative differences, the verification process can be made to conform to the acceptance standards of the corresponding procedures, reducing the subjective judgment bias of manual verification. At the same time, the verification results, which include location and details, are output in voice form, which makes it easier for on-site management personnel to quickly locate the problem points and rectification directions, thereby improving the efficiency of quality verification and the readability of results in unmanned verification scenarios.

[0058] Furthermore, considering that the ground construction status at the construction site changes dynamically with the work process, and that the inspection robot may easily damage the uncured work surface when moving between different inspection spaces, this embodiment may also include the following steps: The response will send the dispatch voice information or verification voice information to the space management terminal of the corresponding inspection space, determine the inspection space as the current space, and determine other inspection spaces connected to the current space and located after the current space as the arrival space based on the predetermined inspection route; Acquire the transitional ground image between the current space and the destination space, and identify the transitional ground image to obtain the constructed ground area and the remaining ground area; Retrieve the construction material type for the corresponding constructed ground area, and determine the accessibility of the constructed ground area based on the construction material type; When the type of construction material is determined to be cement, the passage attribute is set to impassable, and the corresponding constructed ground area is set as an interference avoidance area. When the type of construction material is determined to be brick, the accessibility attribute is set to passable, and the corresponding constructed ground area is set as a safe walking area. Based on the remaining ground area and the safe walking area, a cross-space movement path is planned from the current space to the destination space, and the inspection robot is controlled to move along the cross-space movement path to the destination space.

[0059] For example, in this embodiment, when the server completes the operation of sending scheduling voice information or verification voice information to the space management terminal, it marks the completion of all inspection operations in the current inspection space. At this time, the server marks the inspection space as the current space, and at the same time retrieves the preset predetermined inspection route, matches the inspection space that is adjacent to and connected to the current space and is located after the current space in the inspection movement sequence, marks it as the arrival space, and uses it as the next work point to be arrived at by the inspection robot. After that, the server can control the image acquisition unit on the inspection robot to acquire ground images of the corresponding transition channel between the current space and the destination space, and obtain the transition ground image. It can be explained that the transition ground area is the connecting channel between two adjacent inspection spaces, and is the necessary area for the inspection robot to move across spaces. Its ground construction status changes dynamically with the on-site construction progress, and there can be areas where the construction work has been completed and areas where the original ground has not been constructed at the same time. Furthermore, the server can call a preset ground semantic segmentation algorithm to identify the transitional ground image, and distinguish and extract the constructed ground area and the remaining ground area within the image range. The constructed ground area is the area where ground construction work has been completed, and the remaining ground area is the area where ground construction has not been carried out and the original site state has been maintained. After the area division is completed, the server can retrieve the construction material type corresponding to each constructed ground area from the corresponding on-site construction progress database, and match the corresponding access attribute according to the physical characteristics of the construction material type. The access attribute can be used to characterize whether the corresponding ground area can withstand the passage load of the inspection robot without causing damage to the finished construction product. In one scenario, when it is determined that the construction material type corresponding to a certain constructed ground area is cement, considering that newly poured cement ground has a curing period, and pressure during the curing period can easily cause quality problems such as surface depression and cracking, the passage attribute of this area is marked as impassable, and the constructed ground area is marked as an interference avoidance area. Passing through this type of area is prohibited during the path planning process.

[0060] In another case, when it is determined that the construction material type corresponding to a certain constructed ground area is brick building material, considering that the laid brick ground has the load-bearing capacity and passage will not damage the finished construction, the passage attribute of the area is marked as passable, and the constructed ground area is marked as a safe walking area, which can be included in the travel path range of the inspection robot. Finally, the server uses the remaining ground area and all safe walking areas within the transition zone as the traversable range to plan and generate a cross-zone movement path from the current space to the destination space. This path avoids all interference avoidance areas throughout and minimizes the travel distance. After the path planning is completed, the server can issue a corresponding advance command to the inspection robot, controlling the inspection robot to move along the generated cross-space movement path to the destination space and carry out the operation process of the next inspection space.

[0061] It can be explained that, in this embodiment, by recognizing the transitional ground condition and matching the access attributes with the building material type, the cross-area movement path of the inspection robot can be adapted to the dynamically changing ground construction state of the construction site, actively avoid uncured cement work surfaces, and prevent the robot from damaging the finished construction products. At the same time, it makes full use of the cured finished ground and the original ground to plan the travel route, ensuring the continuity and safety of the inspection route and reducing the interference of inspection operations on the on-site construction process.

[0062] Another embodiment of the present invention provides a voice interaction platform for multi-terminal collaboration. Figure 2 Its corresponding platform block diagram includes: The image acquisition module is configured to respond to the inspection robot moving along the predetermined inspection route of the corresponding construction site to any inspection space, and control the inspection robot to perform image acquisition based on the target space task of the corresponding inspection space to obtain the spatial image of the corresponding inspection space. The mode determination module is configured to determine whether construction personnel exist in the inspection space based on the spatial image, and to determine the inspection mode of the inspection robot corresponding to the inspection space based on the determination result. The scheduling execution module is configured to control the inspection robot to output a building material scheduling determination signal when the inspection mode is determined to be the scheduling mode, and to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space. The verification execution module is configured to, when the inspection mode is determined to be verification mode, control the inspection robot to verify the inspection space based on the target space task, and generate verification voice information to be sent to the space management terminal of the corresponding inspection space based on the verification results.

[0063] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of the invention.

[0064] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0065] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various aspects of the invention, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof.

[0066] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0067] Those skilled in the art will understand that the modules in the device of the embodiment can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiment can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components.

[0068] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention and form different embodiments.

[0069] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0070] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0071] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of explaining or limiting the subject matter of the invention.

Claims

1. A voice interaction method for multi-terminal collaboration, characterized in that, Includes the following steps: The inspection robot moves along the predetermined inspection route of the corresponding construction site to any inspection space, and the inspection robot is controlled to perform image acquisition based on the target space task of the corresponding inspection space to obtain the spatial image of the corresponding inspection space. Based on spatial images, determine whether there are construction workers in the inspection space, and based on the determination results, determine the inspection mode of the inspection robot for the corresponding inspection space. When the inspection mode is determined to be the scheduling mode, the inspection robot is controlled to output a building material scheduling determination signal and send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space. When the inspection mode is determined to be the verification mode, the inspection robot is controlled to verify the inspection space based on the target space task, and generates verification voice information to be sent to the space management terminal of the corresponding inspection space based on the verification results.

2. The method according to claim 1, characterized in that, The control inspection robot performs image acquisition based on the target space task of the corresponding inspection space, and obtains spatial images of the corresponding inspection space, including: Determine the spatial structure surfaces that make up the inspection space, and retrieve the target space task of the corresponding inspection space based on the inspection sequence number of the corresponding inspection space obtained from the predetermined inspection route. When a construction work item is determined to exist on any spatial structural surface based on the target spatial task, the spatial structural surface is designated as the data acquisition structural surface, and the spatial center point of the corresponding inspection space is obtained. The inspection direction of the inspection robot is determined based on the predetermined inspection route and the construction direction of each acquisition structure surface corresponding to the spatial center point is determined based on the target spatial task. Based on the inspection orientation, the adjustment orientation of the inspection robot corresponding to each construction orientation is determined, and the inspection robot is controlled to rotate in sequence according to the different adjustment orientations in order to collect data on each structural surface and obtain a spatial image of the corresponding inspection space.

3. The method according to claim 2, characterized in that, For each acquisition structure surface, a spatial image of the corresponding inspection space is obtained, including: In response to the inspection robot rotating to any adjustment orientation, the light unit pre-set on the inspection robot is controlled to emit an indicator light toward the acquisition structure surface corresponding to the adjustment orientation; Fill the construction work items corresponding to the structure surface to be collected into the type slots located in the preset group photo template to obtain the group photo voice information; The voice unit pre-set in the inspection robot is controlled to output the obtained group photo voice information, and the image unit pre-set in the inspection robot is controlled to perform acquisition based on a preset acquisition frequency and a continuous preset acquisition duration, so as to determine each acquired sub-image as a spatial image.

4. The method according to claim 3, characterized in that, Based on spatial images, the presence of construction workers in the inspection area is determined, and based on the determination results, the inspection mode of the corresponding inspection area for the inspection robot is determined, including: Each acquired sub-image included in the spatial image is identified, and in response to the determination based on the identification result that any acquired sub-image contains a face region indicating any construction worker, the acquired sub-image is identified as an identifier sub-image; otherwise, it is identified as a construction sub-image. When it is determined that all the sub-images collected corresponding to the same spatial image are construction sub-images, it is determined that there are no construction personnel on the collected structural surface, and the inspection mode of the inspection robot corresponding to the collected structural surface is set to the verification mode; When at least one sub-image of the same spatial image is identified as the identifier sub-image, the construction worker image that is bound to the acquired structural surface is compared with the identifier sub-image that is determined based on the time sequence to obtain the first similarity. The first similarity is compared with the preset similarity, and based on the comparison result, it is determined whether there are construction personnel on the collected structural surface, so as to determine the inspection mode of the inspection robot corresponding to the inspection space.

5. The method according to claim 4, characterized in that, The first similarity score is compared with a preset similarity score, and based on the comparison result, it is determined whether there are construction workers on the collected structural surface, so as to determine the inspection mode of the inspection robot corresponding to the inspection space, including: If the first similarity is greater than the preset similarity, the collection structure surface corresponding to the first similarity is determined as the identification structure surface, and it is determined that there are construction personnel on the identification structure surface, so that the inspection mode of the inspection robot corresponding to the collection structure surface is determined as the scheduling mode. If the first similarity is less than or equal to the preset similarity, the collected structural surface corresponding to the first similarity is identified as an abnormal structural surface, and it is determined that there are no construction personnel on the abnormal structural surface, so that the inspection mode of the inspection robot corresponding to the collected structural surface is determined as the verification mode. The face region corresponding to the marked structural surface is determined as the baseline region, and the face region corresponding to the abnormal structural surface is determined as the comparison region; The second similarity is obtained by comparing all comparison areas corresponding to the same inspection space with all reference areas; If any second similarity obtained from the response is greater than the preset similarity, the inspection mode of the identifier structure surface corresponding to the second similarity will be switched from the scheduling mode to the verification mode.

6. The method according to claim 4, characterized in that, The control inspection robot outputs a building material scheduling confirmation signal and sends the scheduling voice information obtained based on the building material scheduling confirmation signal to the space management terminal of the corresponding inspection space, including: Based on the construction work items, determine the construction materials and construction scope of the corresponding structural surface, and identify any construction sub-image of the structural surface to obtain each construction material area indicating the construction materials and the construction scope area indicating the construction scope of the materials. If any building material area is located within the construction area, the building material area is determined to have construction attributes; otherwise, it is determined to have surplus attributes. Determine the area of ​​the corresponding construction zone, and sum the building material area of ​​each building material zone with construction attributes to obtain the total building material area; The difference between the area of ​​the designated area and the total area of ​​the building materials is calculated, and the resulting empty area is compared with the area of ​​the building materials. When it is determined that the available area is less than the building material area, the inspection mode of the corresponding structural surface to be collected will be switched from the scheduling mode to the verification mode. When the available area is determined to be greater than or equal to the building material area, the available sub-area corresponding to the available area is determined based on the construction scope area, and the inspection robot is controlled to output the building material scheduling determination signal obtained from the available sub-area and the building material area with surplus attributes, so as to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space.

7. The method according to claim 6, characterized in that, The control inspection robot outputs building material scheduling determination signals obtained from vacant sub-regions and building material areas with surplus attributes, and sends the scheduling voice information obtained based on the building material scheduling determination signals to the space management terminal of the corresponding inspection space, including: Obtain the corresponding empty sub-region, the empty center point of any building material region, and the center point of the building material region, and overlap the empty sub-region and the building material region in such a way that the empty center point coincides with the center point of the building material region. When it is determined that the empty sub-region surrounds the building material region, obtain the integer multiple of the building material area corresponding to the empty area, and obtain the surplus value of all building material regions with the surplus attribute. When the integer multiple value is determined to be greater than the margin value, the inspection robot is controlled to output the obtained building material scheduling determination signal; The system receives voice feedback from construction workers, including pre-defined vocabulary, and fills the material difference corresponding to the surplus value of the integer multiple value into the pre-defined retrieval template. The obtained dispatch voice information is then sent to the space management terminal of the corresponding inspection space.

8. The method according to claim 4, characterized in that, The control inspection robot checks the inspection space based on the target space task, and generates inspection voice information based on the inspection results to be sent to the space management terminal of the corresponding inspection space, including: Based on the construction work item, retrieve the corresponding construction verification categories of the collected structural surface, and control the inspection robot to identify the construction sub-images corresponding to the collected structural surface based on the construction verification categories to obtain the actual verification results; The actual verification results are compared and calculated with the preset standard results of the same construction verification category, and the obtained verification standard deviation is compared with the preset verification threshold. If the standard deviation of the response verification is less than or equal to the preset verification threshold, the corresponding data collection structure is determined as a compliant structure, and a verification result with the corresponding compliance status is generated. If the standard deviation of the response verification exceeds the preset verification threshold, the corresponding collected structural surface is identified as a structural surface to be rectified, and the construction verification category with the corresponding standard deviation is identified as a deviation verification item, so as to generate an abnormal verification result containing deviation verification items and verification standard deviation. Based on the construction orientation and verification results of the corresponding collected structural surface, verification voice information is generated and sent to the space management terminal of the corresponding inspection space.

9. The method according to claim 8, characterized in that, The method further includes: The response will send the dispatch voice information or verification voice information to the space management terminal of the corresponding inspection space, determine the inspection space as the current space, and determine other inspection spaces connected to the current space and located after the current space as the arrival space based on the predetermined inspection route; Acquire the transitional ground image between the current space and the destination space, and identify the transitional ground image to obtain the constructed ground area and the remaining ground area; Retrieve the construction material type for the corresponding constructed ground area, and determine the accessibility of the constructed ground area based on the construction material type; When the type of construction material is determined to be cement, the passage attribute is set to impassable, and the corresponding constructed ground area is set as an interference avoidance area. When the type of construction material is determined to be brick, the accessibility attribute is set to passable, and the corresponding constructed ground area is set as a safe walking area. Based on the remaining ground area and the safe walking area, a cross-space movement path is planned from the current space to the destination space, and the inspection robot is controlled to move along the cross-space movement path to the destination space.

10. A voice interaction platform for multi-terminal collaboration, characterized in that, include: The image acquisition module is configured to respond to the inspection robot moving along the predetermined inspection route of the corresponding construction site to any inspection space, and control the inspection robot to perform image acquisition based on the target space task of the corresponding inspection space to obtain the spatial image of the corresponding inspection space. The mode determination module is configured to determine whether construction personnel exist in the inspection space based on the spatial image, and to determine the inspection mode of the inspection robot corresponding to the inspection space based on the determination result. The scheduling execution module is configured to control the inspection robot to output a building material scheduling determination signal when the inspection mode is determined to be the scheduling mode, and to send the scheduling voice information obtained based on the building material scheduling determination signal to the space management terminal of the corresponding inspection space. The verification execution module is configured to, when the inspection mode is determined to be verification mode, control the inspection robot to verify the inspection space based on the target space task, and generate verification voice information to be sent to the space management terminal of the corresponding inspection space based on the verification results.

Images