WebSocket-based three-dimensional ground penetrating radar grouping live broadcast method and system

Abstract

The invention provides a WebSocket-based three-dimensional ground penetrating radar grouping live broadcast method and system, and the method comprises the following steps: S1, starting a WebSocket service, building a connection between a collection device to be monitored designated by a client and a server, and storing an association relationship; s2, the server receives the radar data and analyzes the radar data into a time sequence radar map, a video frame picture, an AI interpretation result and vehicle positioning information; s3, the server side sends the message to a client corresponding to the specified acquisition equipment through WebSocket service; and S4, after the client receives the message, searching the time corresponding to the message on the time shaft, and realizing real-time dynamic display. According to the invention, multiple clients can monitor the ground penetrating radar data of different acquisition devices in real time, and the data can be monitored and acquired from multiple latitudes, so that the actual application requirements of a multi-dimensional real-time monitoring system are met.

Description

Technical Field

[0001] The present invention relates to a real-time monitoring method of a three-dimensional ground penetrating radar during an acquisition process, and in particular to a three-dimensional ground penetrating radar group live broadcast method based on WebSocket, and further to a system adopting the three-dimensional ground penetrating radar group live broadcast method based on WebSocket. Background Art

[0002] In the traditional ground-penetrating radar data monitoring system, the lack of effective real-time data transmission and multi-client monitoring mechanism has limited the real-time nature of data and the multi-dimensionality of monitoring. Especially in the field of three-dimensional ground-penetrating radar, if there are multiple detection teams / multiple collection vehicles conducting detection outside, due to the large amount of data, high real-time requirements, and the diversity of monitoring needs, traditional technical means are difficult to meet the needs of multiple clients to monitor the ground-penetrating radar data of different collection devices at the same time, and it is impossible to monitor the collection data of multiple detection teams / multiple collection vehicles in real time from multiple dimensions such as radar maps and video images. Therefore, the existing technology cannot meet the actual application needs of multi-dimensional real-time monitoring systems. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a three-dimensional ground penetrating radar group live broadcast method based on WebSocket, which aims to enable multiple clients to monitor the ground penetrating radar data of different acquisition devices in real time, and also to monitor the acquisition data from multiple latitudes, including radar maps, video images, real-time positions and interpretation results. On this basis, a system using the three-dimensional ground penetrating radar group live broadcast method based on WebSocket is further provided.

[0004] In this regard, the present invention provides a three-dimensional ground penetrating radar group live broadcast method based on WebSocket, comprising the following steps: Step S1, start the WebSocket service, establish a connection between the acquisition device specified by the client to be monitored and the server, and save the association relationship between the acquisition device and the server after the connection is established; Step S2: The server receives the radar data uploaded by the acquisition device and parses the radar data into time-series radar maps, video frames, AI interpretation results and vehicle positioning information; Step S3, the server sends the message to the client corresponding to the designated acquisition device through the WebSocket service; Step S4, after receiving the message, the client searches for the time corresponding to the message on the time axis, and collects the motion trajectory in the playlist to achieve real-time dynamic display of multi-dimensional collected information.

[0005] A further improvement of the present invention is that step S1 includes the following sub-steps: Step S101, setting the link address of the WebSocket service through the instruction ws / {equipmentId} / {clientId}, wherein equipmentId is the acquisition equipment ID, which is used for the corresponding client to specify the acquisition equipment to be monitored; clientId is the client ID; Step S102, the client creates a new WebSocket object, sets the link address, and then calls the connect() function to open and establish a connection; Step S103: the server associates the acquisition device ID with the client ID in a one-to-one relationship to obtain the association relationship, and saves the association relationship.

[0006] A further improvement of the present invention is that step S102 includes the following sub-steps: Step S1021, the client initiates a WebSocket protocol handshake by sending an HTTP request to the server. The HTTP request includes header information, an Upgrade header field, a Connection header field, and a Sec-WebSocket-Key header field, which are used to identify this handshake; Step S1022, the server returns an HTTP 101 status code through the WebSocket protocol, indicating that the protocol switch is successful. At the same time, the server includes a Sec-WebSocket-Accept header in the response; Step S1023, after the handshake is successful, the connection between the client and the server is upgraded to a WebSocket protocol connection, so as to perform two-way data transmission through the WebSocket protocol connection.

[0007] A further improvement of the present invention is that step S2 includes the following sub-steps: Step S201, the acquisition device collects each frame of data, records the track number trace, time time, acquisition device ID, camera image videoFrame and GPS positioning coordinate position of the frame data, and packages the 3D radar data after collecting a preset track number and uploads it to the server; In step S202, the server performs AI interpretation processing on the uploaded three-dimensional radar data to obtain the interpretation results aiResults and the radar map; the radar map includes the vertical vertical section ascan, the horizontal section bscan and the vertical cross section cscan of the three-dimensional radar map, and converts the radar map into JSON format for representing the message messag, and associates it with the time time recorded in step S201 to obtain the time series radar map.

[0008] A further improvement of the present invention is that, in step S202, the process of AI interpreting the uploaded data is as follows: first, the three-dimensional radar data of the preset number of lanes is divided into the horizontal axis in the x direction, the vertical axis in the y direction, and the horizontal front-back axis in the z direction, and the two-dimensional data of the horizontal and vertical sections of the x direction are taken out, and the largest data among them is set to max, wherein each data value value is calculated into a value within a range of 0 to 255 according to the formula value*(255 / max), and converted into an RGB grayscale color rgb(value,value,value); then, a png image of the same size is created according to the length and width of the two-dimensional data. The color corresponding to the two-dimensional data coordinates is set to the pixel point of the corresponding png image to form the horizontal slice bscan of the radar map; and the vertical vertical slice ascan corresponding to the y-axis and the vertical cross-section cscan of the z-axis are generated in the same way; then, the horizontal slice bscan is adjusted to an image of size 224*224, the pre-trained AI model is called, and the result recognized by AI is output as the interpretation result aiResults, which includes the position coordinate information bbox, score score and recognition type class.

[0009] A further improvement of the present invention is that, in step S202, the process of converting the radar spectrum into JSON format is implemented by the instruction [{time:100000000,trace:100,aiResults:[{class:"hole",score: 90,bbox:[1,2,3,4]}],ascan:"image / png;base64",bscan:"image / png;base64",cscan:"image / png;base64",videoFrame:"image / png;base64",position:{lon: 123.33,lat:23.33}}], where time is Unix The total number of milliseconds from the beginning of the epoch to the current moment; aiResults is the AI ​​interpretation result array, where class is the interpretation type; hole indicates a hole; score is a score indicating the accuracy of the result; bbox is the frame position information on the radar map, with the top left being the origin position, the first bit indicating the minimum x-axis position, the second bit indicating the minimum y-axis position, the third bit indicating the maximum x-axis position, and the fourth bit indicating the maximum y-axis position; videoFrame is the video frame, encoded in base64ASCII string; position is the longitude and latitude coordinates of the vehicle's GPS positioning, where lon indicates longitude and lat indicates latitude.

[0010] A further improvement of the present invention is that in step S3, when the server sends the message to the client corresponding to the designated acquisition device, synchronous message sending is adopted, and the implementation process of synchronous message sending is: first, the message message is added to an ordered array messageList, the array subscript of the ordered array messageList starts from 0, the first subscript added is 0, and the subsequent subscripts are increased by 1 in sequence, and then the message with subscript 0 is taken out by first-in-first-out and last-in-last-out methods, and after the taking out, all the data subscripts behind it are reduced by 1; then, the taken out message message is sent to the client; finally, it is determined whether the client returns a successful reception signal, if not, resend is selected after waiting for a preset time, and the next message message with subscript 0 is not taken out and sent until the client returns a successful reception.

[0011] A further improvement of the present invention is that step S4 includes the following sub-steps: Step S401, after receiving the message message, the client first adds the message message to the play sequence playSequence, and initializes a timeline for playback. The start time startTime of the timeline is the time time of the first message message of the play sequence playSequence, and the end time endTime of the timeline is the time time of the last message message of the play sequence playSequence. When the total time endTime-startTime is within 10 seconds, buffering is paused and waited. When the total time endTime-startTime is greater than 10 seconds, the timeline starts to play from the start time startTime at the set play speed, and the play speed is speed. The current play time curTime advances speed / 1 second every 1 second. Step S402, monitor the change of the current play time curTime, search the play sequence playSequence by binary search according to the current play time curTime, find the message message whose time time is less than the current play time curTime, take out the radar map and video screen in this part of the message message, and display them in the client window; Step S403, the interpretation result aiResults is drawn into a yellow highlighted box in the radar map according to the coordinate position of the position coordinate information bbox to indicate that a hidden danger is found there; Step S404, add a vehicle model to the map according to the longitude and latitude coordinates of the vehicle's GPS positioning to display it on the map so as to view the location of the vehicle in real time; and gather the GPS positioning coordinates position of all messages in the play sequence playSequence to draw the vehicle's driving trajectory on the map to obtain the multi-dimensional acquisition information of the three-dimensional ground penetrating radar displayed dynamically and in real time.

[0012] A further improvement of the present invention is that in the step S402, the implementation process of finding the message message whose time time is less than the current play time curTime in the message message by the binary search method is as follows: first, in the play sequence playSequence, determine the pointer left and the pointer right, which are used to point to the positions of the first element and the last element of the ordered array messageList respectively, and calculate the middle position mid by the formula mid=Math.floor((left + right) / 2), where Math.floor is rounded down; then compare, if the target value is equal to the element value of the middle position mid, the search is completed and the index of the middle position mid is returned; if the target value is less than the element value of the middle position mid, adjust the pointer right to mid - 1, and continue to search in the left half; if the target value is greater than the element value of the middle position mid, adjust the pointer left to mid + 1, and continue to search in the right half; repeat the above steps until the target value is found or the search range is empty; In the step S403, the implementation process of drawing a yellow highlighted box in the radar map is as follows: according to the position coordinate information bbox in the element of the interpretation result aiResults, according to the upper left and lower right coordinate positions, the specific position and length and height of the box are located on the horizontal slice bscan, where the length is the value of the third element of the position coordinate information bbox minus the first element, and the height is the value of the fourth element of the position coordinate information bbox minus the second element. The line width of the box is set to 2px, and the color is RGBA (255, 255, 0, 1) yellow, and then a yellow highlighted box is drawn to prompt the hidden dangers there.

[0013] The present invention also provides a three-dimensional ground penetrating radar group live broadcast system based on WebSocket, which adopts the three-dimensional ground penetrating radar group live broadcast method based on WebSocket as described above, and includes: Establish a connection module, start the WebSocket service, establish a connection between the acquisition device specified by the client to be monitored and the server, and save the association relationship between the acquisition device and the server after the connection is established; Radar data upload and analysis module: the server receives radar data uploaded by the acquisition device and analyzes the radar data into time-series radar maps, video frames, AI interpretation results and vehicle positioning information; Information sending module: the server sends the message to the client corresponding to the specified acquisition device through the WebSocket service; Real-time dynamic display module, after the client receives the message, it searches for the time corresponding to the message on the timeline and collects the motion trajectory in the playlist to achieve real-time dynamic display of multi-dimensional collected information.

[0014] Compared with the prior art, the beneficial effects of the present invention are: first start the WebSocket service, and establish a connection between the acquisition device specified by the client to be monitored and the server, and save the association relationship after the connection is established; then after the server receives the radar data uploaded by the acquisition device, the radar data is parsed into time-series radar maps, video frame images, AI interpretation results and vehicle positioning information; finally, the message is sent to the corresponding client through the server, the time corresponding to the message is found on the timeline, and the motion trajectory is collected in the playlist to realize the real-time dynamic display of multi-dimensional acquisition information. Therefore, the present invention can realize a three-dimensional ground-penetrating radar group live broadcast method and system based on WebSocket, can realize that multiple clients can monitor the ground-penetrating radar data of different acquisition devices in real time, and can also monitor the acquisition data from multiple latitudes, including radar maps, video images, real-time positions and interpretation results, to meet the actual application needs of multi-dimensional real-time monitoring systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the working process of an embodiment of the present invention. DETAILED DESCRIPTION

[0016] The preferred embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0017] This embodiment is applied to the real-time monitoring of the three-dimensional ground penetrating radar acquisition process. The three-dimensional ground penetrating radar of this embodiment is referred to as ground penetrating radar. In the process of road detection, when there are multiple detection teams / multiple collection vehicles conducting detection outside, it is often necessary to know the detection progress, current position and detection results of each detection team / collection vehicle in real time. Then, it is necessary to meet the actual application requirements of the multi-dimensional real-time monitoring system to realize the real-time monitoring of the detection progress, current position and detection results of multiple detection teams. However, the existing technology can only be applied to one collection vehicle / one detection team to monitor the radar map in real time. Obviously, it cannot meet the actual application requirements of the above-mentioned multiple detection teams / multiple collection vehicles.

[0018] In this regard, Figure 1 As shown, this embodiment provides a three-dimensional ground penetrating radar group live broadcast method based on WebSocket, comprising the following steps: Step S1, start the WebSocket service, establish a connection between the acquisition device specified by the client to be monitored and the server, and save the association relationship between the acquisition device and the server after the connection is established; Step S2: The server receives the radar data uploaded by the acquisition device and parses the radar data into time-series radar maps, video frames, AI interpretation results and vehicle positioning information; Step S3, the server sends the message to the client corresponding to the designated acquisition device through the WebSocket service; Step S4, after receiving the message, the client searches for the time corresponding to the message on the time axis, and collects the motion trajectory in the playlist to achieve real-time dynamic display of multi-dimensional collected information.

[0019] Step S1 in this embodiment includes the following sub-steps: Step S101, setting the link address of the WebSocket service through the instruction ws / {equipmentId} / {clientId}, the link address is also called the link URL, URL is the abbreviation of Uniform Resource Locator; wherein equipmentId is the acquisition equipment ID, which is used for the corresponding client to specify the acquisition equipment to be monitored; clientId is the client ID; Step S102, the client creates a new WebSocket object, sets the link address, and then calls the connect() function to open and establish a connection, that is, to establish a connection with the server through the link address; WebSocket is a network communication protocol, which is part of HTML5 and provides the ability to perform full-duplex communication on a single TCP connection; Step S103: the server associates the acquisition device ID with the client ID in a one-to-one relationship to obtain the association relationship, and saves the association relationship.

[0020] Step S102 in this embodiment is used to establish a connection through WebSocket, and its implementation process includes the following sub-steps: Step S1021, the client initiates a WebSocket protocol handshake by sending an HTTP request to the server. The HTTP request includes header information, an Upgrade header field, a Connection header field, and a Sec-WebSocket-Key header field, which are used to identify this handshake; Step S1022, the server returns an HTTP 101 status code through the WebSocket protocol, indicating that the protocol switch is successful. At the same time, the server includes a Sec-WebSocket-Accept header in the response; Step S1023, after the handshake is successful, the connection between the client and the server is upgraded to a WebSocket protocol connection, so as to perform two-way data transmission through the WebSocket protocol connection.

[0021] Step S2 in this embodiment is used to receive and analyze radar data, and preferably includes the following sub-steps: Step S201, the acquisition device records the track number trace, time time, acquisition device ID, camera image videoFrame and GPS positioning coordinate position of each frame of data collected, the GPS positioning coordinate position includes longitude lon and latitude lat, and packages the 3D radar data after collecting a preset number of track numbers and uploads them to the server; the preset number of track numbers can be set and adjusted according to actual conditions and needs, and the default setting is 400; In step S202, the server performs AI interpretation processing on the uploaded 3D radar data to obtain the interpretation results aiResults and radar maps; the radar maps include the vertical vertical section ascan, horizontal section bscan and vertical cross section cscan of the 3D radar map, which are represented by base64ASCII string encoding by default; the radar map is converted into JSON format to represent the message messag, and is associated with the time recorded in step S201 to obtain a time series radar map. The time series radar map includes the vertical vertical section ascan, horizontal section bscan and vertical cross section cscan described below; the video frame image and vehicle positioning information refer to the directly uploaded 3D radar data (including the camera image videoFrame) and the GPS positioning coordinate position respectively.

[0022] In step S202 of this embodiment, the process of AI interpreting the uploaded data is as follows: first, the three-dimensional radar data of a preset number of traces, such as the three-dimensional radar data of 400 traces, is divided into the horizontal axis of the x direction, the vertical axis of the y direction, and the horizontal front-back axis of the z direction, and the two-dimensional data of the x horizontal horizontal and vertical sections are taken out, and the largest data is set to max, wherein each data value value is calculated into a value within a range of 0 to 255 according to the formula value*(255 / max), and converted into RGB grayscale color rgb(value,value,value); then, a png image of the same size is created according to the length and width of the two-dimensional data, and the two-dimensional data is compared with the two-dimensional data. The colors corresponding to the coordinates are set to the pixels of the corresponding png images to form the horizontal section bscan of the radar map; and in the same way, the vertical vertical section ascan corresponding to the y-axis and the vertical cross-section cscan of the z-axis are generated; then, the horizontal section bscan is adjusted to an image of 224*224 size, that is, the bscan map is resized to 224*224 size, and the pre-trained AI model is called to output the AI ​​recognition result as the interpretation result aiResults, which includes the position coordinate information bbox, score score and recognition type class.

[0023] In step S202 of this embodiment, the process of converting the radar map into JSON format is implemented by the instruction [{time:100000000,trace:100,aiResults:[{class:"hole",score: 90,bbox:[1,2,3,4]}],ascan:"image / png;base64",bscan:"image / png;base64",cscan:"image / png;base64",videoFrame:"image / png;base64",position:{lon: 123.33,lat:23.33}}], where time is the total number of milliseconds from the beginning of the Unix epoch to the current time, that is, January 1, 1970 00:00:00 UTC The total number of milliseconds from the start to the current moment; aiResults is the AI ​​interpretation result array, where class is the interpretation type; hole indicates a hole; score is a score indicating the accuracy of the result, where the higher the value, the greater the probability, and the full score is 100; bbox is the frame position information on the radar map, with the top left being the origin, the first bit indicating the minimum x-axis position, the second bit indicating the minimum y-axis position, the third bit indicating the maximum x-axis position, and the fourth bit indicating the maximum y-axis position; videoFrame is the video frame, encoded in base64ASCII string; position is the longitude and latitude coordinates of the vehicle's GPS positioning, where lon indicates longitude and lat indicates latitude.

[0024] JSON (JavaScript Object Notation) is a lightweight data exchange format, commonly used for exchanging data between Web applications. The design of JSON format makes it easy to read and understand. Data is organized in the form of key-value pairs, with a clear structure. In addition, JSON format is compact, occupies fewer bytes, and is suitable for transmission on the network. In this embodiment, the message message is converted into JSON format, that is, the Java object is serialized into a string in JSON format. For example, if the object Person contains the attributes age age as an integer 32 and sex sex as a string type "male", then the string converted into JSON key-value pairs is represented as {"age": 32, "sex": "male"}.

[0025] In step S3 of this embodiment, the client ID associated with the current device ID in step S1 is queried, and the server uses synchronous message sending when sending the message message to the client corresponding to the specified acquisition device to ensure the time sequence of the client's playback. The implementation process of synchronous message sending is as follows: first add the message message to an ordered array messageList, the array subscript of the ordered array messageList starts from 0, the first subscript added is 0, and the subsequent subscripts are added by 1, and then the message with subscript 0 is taken out by first-in-first-out and last-in-last-out methods, and after taking it out, all the data subscripts behind it are reduced by 1; then, send the taken out message message to the client; finally, determine whether the client returns a successful reception signal, if not, choose to resend after waiting for a preset time, and the next message with subscript 0 is taken out and sent until the client returns a successful reception. The preset time can be set and adjusted according to actual conditions and needs. After waiting for the preset time period (i.e., the preset time), if the client still does not return a signal of successful reception, it is judged that the transmission has failed, and then the message is resent. The next message with an index of 0 is not retrieved and sent until the client returns a signal of successful reception, so as to ensure the accuracy and reliability of message transmission.

[0026] Step S4 in this embodiment includes the following sub-steps: Step S401, after receiving the message message, the client first adds the message message to the play sequence playSequence, and initializes a timeline for playback. The start time startTime of the timeline is the time time of the first message message of the play sequence playSequence, and the end time endTime of the timeline is the time time of the last message message of the play sequence playSequence. When the total time endTime-startTime is within 10 seconds, buffering is paused and waited. When the total time endTime-startTime is greater than 10 seconds, the timeline starts to play from the start time startTime at the set play speed. The play speed is speed, and the current play time curTime advances speed / 1 second every 1 second; for example, speed=1, it advances 1 second per second; speed=0.5, it advances 0.5 seconds per second; Step S402, monitor the change of the current play time curTime, search the play sequence playSequence by binary search according to the current play time curTime, find the message message whose time time is less than the current play time curTime, take out the radar map and video screen in this part of the message message, and display them in the client window; Step S403, the interpretation result aiResults is drawn into a yellow highlighted box in the radar map according to the coordinate position of the position coordinate information bbox to indicate that a hidden danger is found there; Step S404, add a vehicle model to the map according to the longitude and latitude coordinates of the vehicle's GPS positioning to display it on the map so as to view the location of the vehicle in real time; and gather the GPS positioning coordinates position of all messages in the play sequence playSequence to draw the vehicle's driving trajectory on the map to obtain the multi-dimensional acquisition information of the three-dimensional ground penetrating radar displayed dynamically and in real time.

[0027] In step S402 of the present embodiment, the implementation process of finding the message message whose time time is less than the current play time curTime in the message message by binary search is as follows: first, in the play sequence playSequence, determine the pointer left and the pointer right, which are used to point to the positions of the first element and the last element of the ordered array messageList respectively, and calculate the middle position mid by the formula mid=Math.floor((left + right) / 2), where Math.floor is rounded down; then compare, if the target value is equal to the element value of the middle position mid, the search ends and the index of the middle position mid is returned; if the target value is less than the element value of the middle position mid, adjust the pointer right to mid - 1, and continue to search in the left half; if the target value is greater than the element value of the middle position mid, adjust the pointer left to mid + 1, and continue to search in the right half; repeat the above steps until the target value is found or the search range is empty. The search range is empty when the pointer left exceeds the pointer right.

[0028] It should be noted that the binary search method is applicable to sorted arrays, and the play sequence playSequence of this embodiment is arranged in chronological order, and the play process is also played in chronological order, so it is very suitable for the binary search method to quickly find the message frame to be played at the current time.

[0029] In step S403 of this embodiment, the implementation process of drawing a yellow highlighted box in the radar map is as follows: according to the position coordinate information bbox in the element of the interpretation result aiResults, according to the upper left and lower right coordinate positions, the specific position and length of the box are located on the horizontal section bscan, where the length is the value of the third element of the position coordinate information bbox minus the first element, and the height is the value of the fourth element of the position coordinate information bbox minus the second element, and the line width of the box is set to 2px, and the color is RGBA (255, 255, 0, 1) yellow, and then a yellow highlighted box is drawn to indicate the hidden dangers there. Wherein bbox is the box position coordinate of the recognition result of the horizontal section bscan, also known as position coordinate information, and its content is a floating point array of length 4, the first element is the upper left minimum horizontal coordinate, the second element is the upper left minimum vertical coordinate, the third element is the lower right maximum horizontal coordinate, and the fourth element is the lower right maximum vertical coordinate.

[0030] This embodiment also provides a three-dimensional ground penetrating radar group live broadcast system based on WebSocket, which adopts the three-dimensional ground penetrating radar group live broadcast method based on WebSocket as described above, and includes: Establish a connection module, start the WebSocket service, establish a connection between the acquisition device specified by the client to be monitored and the server, and save the association relationship between the acquisition device and the server after the connection is established; Radar data upload and analysis module: the server receives radar data uploaded by the acquisition device and analyzes the radar data into time-series radar maps, video frames, AI interpretation results and vehicle positioning information; Information sending module: the server sends the message to the client corresponding to the specified acquisition device through the WebSocket service; Real-time dynamic display module, after the client receives the message, it searches for the time corresponding to the message on the timeline and collects the motion trajectory in the playlist to achieve real-time dynamic display of multi-dimensional collected information.

[0031] In summary, this embodiment first starts the WebSocket service, and establishes a connection between the acquisition device to be monitored specified by the client and the server, and saves the association relationship after the connection is established; then after the server receives the radar data uploaded by the acquisition device, the radar data is parsed into time-series radar maps, video frame images, AI interpretation results and vehicle positioning information; finally, the message is sent to the corresponding client through the server, the time corresponding to the message is found on the timeline, and the motion trajectory is collected in the playlist to realize the real-time dynamic display of multi-dimensional acquisition information. Therefore, the present invention can realize a three-dimensional ground-penetrating radar group live broadcast method and system based on WebSocket, can realize that multiple clients can monitor the ground-penetrating radar data of different acquisition devices in real time, and can also monitor the acquisition data from multiple latitudes, including radar maps, video images, real-time positions and interpretation results, to meet the actual application requirements of multi-dimensional real-time monitoring systems.

[0032] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A three-dimensional ground penetrating radar group live broadcast method based on WebSocket, It is characterized in that The following steps are involved: Step S1, start the WebSocket service, establish a connection between the acquisition device specified by the client to be monitored and the server, and save the association relationship between the acquisition device and the server after the connection is established; Step S2: The server receives the radar data uploaded by the acquisition device and parses the radar data into time-series radar maps, video frames, AI interpretation results and vehicle positioning information; Step S3, the server sends the message to the client corresponding to the designated acquisition device through the WebSocket service; Step S4, after receiving the message, the client searches for the time corresponding to the message on the time axis, and collects the motion trajectory in the playlist to achieve real-time dynamic display of multi-dimensional collected information.

2. According to the WebSocket-based three-dimensional ground penetrating radar group live broadcast method of claim 1, It is characterized in that The step S1 comprises the following sub-steps: Step S101, setting the link address of the WebSocket service through the instruction ws / {equipmentId} / {clientId}, wherein equipmentId is the acquisition equipment ID, which is used for the corresponding client to specify the acquisition equipment to be monitored; clientId is the client ID; Step S102, the client creates a new WebSocket object, sets the link address, and then calls the connect() function to open and establish a connection; Step S103: the server associates the acquisition device ID with the client ID in a one-to-one relationship to obtain the association relationship, and saves the association relationship.

3. According to the WebSocket-based three-dimensional ground penetrating radar group live broadcast method of claim 2, It is characterized in that The step S102 includes the following sub-steps: Step S1021, the client initiates a WebSocket protocol handshake by sending an HTTP request to the server. The HTTP request includes header information, an Upgrade header field, a Connection header field, and a Sec-WebSocket-Key header field, which are used to identify this handshake; Step S1022, the server returns an HTTP 101 status code through the WebSocket protocol, indicating that the protocol switch is successful. At the same time, the server includes a Sec-WebSocket-Accept header in the response; Step S1023, after the handshake is successful, the connection between the client and the server is upgraded to a WebSocket protocol connection, so as to perform two-way data transmission through the WebSocket protocol connection.

4. The three-dimensional ground penetrating radar group live broadcast method based on WebSocket according to any one of claims 1 to 3, It is characterized in that The step S2 comprises the following sub-steps: Step S201, the acquisition device collects each frame of data, records the track number trace, time time, acquisition device ID, camera image videoFrame and GPS positioning coordinate position of the frame data, and packages the 3D radar data after collecting a preset track number and uploads it to the server; In step S202, the server performs AI interpretation processing on the uploaded three-dimensional radar data to obtain the interpretation results aiResults and the radar map; the radar map includes the vertical vertical section ascan, the horizontal section bscan and the vertical cross section cscan of the three-dimensional radar map, and converts the radar map into JSON format for representing the message messag, and associates it with the time time recorded in step S201 to obtain the time series radar map.

5. According to the WebSocket-based three-dimensional ground penetrating radar group live broadcast method of claim 4, It is characterized in that In step S202, the process of AI interpreting the uploaded data is as follows: first, the three-dimensional radar data of the preset number of lane numbers is divided into the horizontal axis of the x direction, the vertical axis of the y direction, and the horizontal front-back axis of the z direction, and the two-dimensional data of the horizontal and vertical sections of the x direction are taken out, and the largest data is set to max, where each data value value is calculated into a value within a range of 0 to 255 according to the formula value*(255 / max), and converted into RGB grayscale color rgb(value,value,value); then, a png image of the same size is created according to the length and width of the two-dimensional data, and the two-dimensional data is compared with the two-dimensional data. The colors of the data coordinates are set one by one to the pixels of the corresponding png image to form the horizontal section bscan of the radar map; and the vertical vertical section ascan corresponding to the y-axis and the vertical cross-section cscan of the z-axis are generated in the same way; then, the horizontal section bscan is adjusted to an image of size 224*224, and the pre-trained AI model is called to output the results recognized by AI as the interpretation result aiResults, which includes the position coordinate information bbox, score score and recognition type class.

6. The three-dimensional ground penetrating radar group live broadcast method based on WebSocket according to claim 4, It is characterized in that In step S202, the process of converting the radar map into JSON format is implemented by the instruction [{time:100000000,trace:100,aiResults:[{class:"hole",score: 90,bbox:[1,2,3,4]}],ascan:"image / png;base64",bscan:"image / png;base64",cscan:"image / png;base64",videoFrame:"image / png;base64",position:{lon: 123.33,lat:23.33}}], where time is Unix The total number of milliseconds from the beginning of the epoch to the current moment; aiResults is the AI ​​interpretation result array, where class is the interpretation type; hole indicates a hole; score is a score indicating the accuracy of the result; bbox is the frame position information on the radar map, with the top left being the origin position, the first bit indicating the minimum x-axis position, the second bit indicating the minimum y-axis position, the third bit indicating the maximum x-axis position, and the fourth bit indicating the maximum y-axis position; videoFrame is the video frame, encoded in base64ASCII string; position is the longitude and latitude coordinates of the vehicle's GPS positioning, where lon indicates longitude and lat indicates latitude.

7. The three-dimensional ground penetrating radar group live broadcast method based on WebSocket according to any one of claims 1 to 3, It is characterized in that In step S3, when the server sends the message to the client corresponding to the designated acquisition device, synchronous message sending is adopted. The implementation process of synchronous message sending is as follows: first, the message message is added to an ordered array messageList. The array subscript of the ordered array messageList starts from 0. The first subscript added is 0, and the subsequent subscripts are increased by 1 in sequence. Then, the message with subscript 0 is taken out by first-in-first-out and last-in-last-out methods. After the message is taken out, the subscripts of all subsequent data are reduced by 1; then, the taken out message message is sent to the client; finally, it is determined whether the client returns a successful reception signal. If not, resend is selected after waiting for a preset time, and the next message message with subscript 0 is not taken out and sent until the client returns a successful reception.

8. The three-dimensional ground penetrating radar group live broadcast method based on WebSocket according to any one of claims 1 to 3, It is characterized in that The step S4 comprises the following sub-steps: Step S401, after receiving the message message, the client first adds the message message to the play sequence playSequence, and initializes a timeline for playback. The start time startTime of the timeline is the time time of the first message message of the play sequence playSequence, and the end time endTime of the timeline is the time time of the last message message of the play sequence playSequence. When the total time endTime-startTime is within 10 seconds, buffering is paused and waited. When the total time endTime-startTime is greater than 10 seconds, the timeline starts to play from the start time startTime at the set play speed, and the play speed is speed. The current play time curTime advances speed / 1 second every 1 second. Step S402, monitor the change of the current play time curTime, search the play sequence playSequence by binary search according to the current play time curTime, find the message message whose time time is less than the current play time curTime, take out the radar map and video screen in this part of the message message, and display them in the client window; Step S403, the interpretation result aiResults is drawn into a yellow highlighted box in the radar map according to the coordinate position of the position coordinate information bbox to indicate that a hidden danger is found there; Step S404, add a vehicle model to the map according to the longitude and latitude coordinates of the vehicle's GPS positioning to display it on the map so as to view the location of the vehicle in real time; and gather the GPS positioning coordinates position of all messages in the play sequence playSequence to draw the vehicle's driving trajectory on the map to obtain the multi-dimensional acquisition information of the three-dimensional ground penetrating radar displayed dynamically and in real time.

9. The three-dimensional ground penetrating radar group live broadcast method based on WebSocket according to claim 8, It is characterized in that In the step S402, the implementation process of finding the message message whose time time is less than the current play time curTime in the message message by the binary search method is as follows: first, in the play sequence playSequence, determine the pointer left and the pointer right, which are used to point to the positions of the first element and the last element of the ordered array messageList respectively, and calculate the middle position mid by the formula mid=Math.floor((left + right) / 2), where Math.floor is rounded down; then compare, if the target value is equal to the element value of the middle position mid, the search is completed, and the index of the middle position mid is returned; If the target value is less than the value of the element at the middle position mid, adjust the pointer right to mid - 1 and continue searching in the left half; if the target value is greater than the value of the element at the middle position mid, adjust the pointer left to mid + 1 and continue searching in the right half; repeat the above steps until the target value is found or the search range is empty; In the step S403, the implementation process of drawing a yellow highlighted box in the radar map is as follows: according to the position coordinate information bbox in the element of the interpretation result aiResults, according to the upper left and lower right coordinate positions, the specific position and length and height of the box are located on the horizontal slice bscan, where the length is the value of the third element of the position coordinate information bbox minus the first element, and the height is the value of the fourth element of the position coordinate information bbox minus the second element. The line width of the box is set to 2px, and the color is RGBA (255, 255, 0, 1) yellow, and then a yellow highlighted box is drawn to prompt the hidden dangers there.

10. A 3D ground penetrating radar group live broadcast system based on WebSocket. It is characterized in that The method for live broadcasting of three-dimensional ground penetrating radar based on WebSocket as claimed in any one of claims 1 to 9 is adopted, and includes: Establish a connection module, start the WebSocket service, establish a connection between the acquisition device specified by the client to be monitored and the server, and save the association relationship between the acquisition device and the server after the connection is established; Radar data upload and analysis module: the server receives radar data uploaded by the acquisition device and analyzes the radar data into time-series radar maps, video frames, AI interpretation results and vehicle positioning information; Information sending module: the server sends the message to the client corresponding to the specified acquisition device through the WebSocket service; Real-time dynamic display module, after the client receives the message, it searches for the time corresponding to the message on the timeline and collects the motion trajectory in the playlist to achieve real-time dynamic display of multi-dimensional collected information.

Images